benchmarking METHODOLOGY

The methodology of the umlaut connect Fixed-Line Broadband Benchmark is based on a sophisticated crowdsourcing approach. It was carefully designed to evaluate and objectively compare the performance of fixed-line networks from the users’ perspective.

The umlaut connect Fixed-­Line Broadband Benchmark in the Netherlands is based on a sophis­ticated crowdsourcing approach. 

DATA GATHERING

The analyses are based on data gathered in the 24 weeks between calendar week 29 (mid-July) until calendar week 52 (end of December), 2021. In the process, a total of 3,048,708 samples from 53,003 Dutch fixed lines were evaluated. 

For the data collection, umlaut has integrated a back­ground diagnosis ­pro­cess into more than 1000 diverse Android apps. If one of these applica­tions is ­installed on the end-user’s phone and the user ­authorizes the background analysis, data collection takes place 24/7, 365 days a year. ­Samples are generated in specific intervals (from one second up to 15 minutes and sent daily to umlaut‘s cloud servers, where the data is further processed. 

By filtering the network access technology to those samples collected via Wi-Fi (in contrast to mobile network connections) and identifying the network operator, the collected samples can be confined to fixed-line connections. Using a complex set of rules and comprehensive checks, umlaut hardens the validity of the evaluations. Among other things, data recorded when the smartphone battery is low is filtered out, as are measurements when the transfer volume is too low or extremely low data rates indicate interference. The influence of the mobile devices is likely to be small. The Wi-Fi speeds achievable on current smartphones are usually significantly higher than the total data rates shown. 

PASSIVE DATA RATES

The passive gathering of the data rates observed for down­loads and uploads take place in the back­ground while the user‘s employ everyday applications on their ­devices such as web browsing, messaging or gaming. In order to classify the observed speeds, umlaut has defined four application-related speed classes:

“Basic Internet“ requires a minimum of 2 Mbps, “HD Video“ requires 5 Mbps, “UHD Video“ requires 20 Mbps and “Highspeed“ requires 50 Mbps. The observed passive download speeds make of 10% of the total result, the upload speeds contribute 25% to the total result.

ACTIVE data rates

As the passively observed speeds depend on the demands of the applications actually used, umlaut‘s platform also initiates active speed measurements from time to time. They can determine the maximum performance available on the end device, thus reflecting the user-oriented fixed network experience – including additional influencing factors such as the performance of the Wi-Fi router used. The tables indicate the active download data rate that exceeds 90% of the measured values. This parameter thus shows how good the user experience of a network operator is across the board. What is possible at the peak is shown by the “maximum speed“, which the best 10% of the connections have achieved or even topped. The active speed measurements make up 40% of the overall rating.

latencies

The latency measurements are carried out in the samle cycles which are performed for the active download tests – “pings“ take place in direct succession to the downloads and make up 25% of the total score. The results are also ­assigned to an application-related class: Roundtrip times up to 100 ms are sufficient for “OTT Voice“, less than 50 ms qualify a sample for “Standard Gaming“ and less than 20 ms for “Highend Gaming“. The tables in this report show the percentage of the examined connections that were able to achieve the required threshold values in the different classes or did even better.

ConclusioN

The joint company Vodafone-Ziggo is the winner of the 2022 umlaut connect Fixed-Line Broadband Benchmark for the Netherlands. KPN and T-Mobile also achieve very good results, Tele2 receives the overall grade “good“.

With very strong results overall and particularly in the categories of the active download tests and the passive upload observations, Ziggo is the overall winner with the grade “very good“. In the local analyses of larger Dutch provinces, Ziggo even receives the rarely awarded grade
“outstanding“ in Gelderland, Noord-Holland and Utrecht.

KPN achieves a very good second rank, showing particularly strong results in the active and ­passive download assessments as well as in the observed latencies. Above that, KPN achieves particularly high data rates on its fibre lines – showing the effects of recent investments in this area. Furthermore, KPN is the local champion in the pro­vince of Limburg, also with the grade “outstanding“. 

T-Mobile, rated as the operator of the fixed network lines of the same name, achieves also a very good ­overall result. The operator scores particu­larly well in the assessment of latencies in the classes of Standard as well as Highend Gaming is also local champion in the province of Overijssel, where it even receives the ­grade “outstanding“.

Although being part of T-Mobile now, Tele2 is evaluated separately based on the operator designation of its fixed-line connections. Overall, Tele2 ranks fourth, but still achieves a good result. It shows particular strength in the latency evaluations and manages to pass its parent company in Gelderland, Noord-Holland and Utrecht.