Data access in big cities
LTE has been designed to offer mobile data access with very high speeds and a very high reliability. Can the Dutch network operators fulfill this promise?
In big cities, each of the four Dutch network operators claims to offer almost complete 4G coverage. In order to measure the quality and reliability of mobile data services, the four P3 teams visited a total of ten Dutch cities each of which has more than 100,000 inhabitants.
But in densely populated areas a large number of cells with comparably small diameters is necessary to provide high speed to every user. The real-life test scenarios applied by P3 (see also page 8) there- fore concentrated on accessing both live and static web pages as well as downloading and uploading files and watching YouTube videos.
Web page access
In order to test the stability and speed of website access, the benchmark uses a mixture of ten popular live web pages as well as one static web page. In big cities all four operators showed excellent performance, with T-Mobile and KPN even accomplishing rare 100,0 percent success ratios when accessing the static test page. Further, average session times were pleasantly short, with a close lead by T-Mobile, a strong midfield consisting of KPN and Tele2 and a slight backlog for Vodafone.
File uploads and downloads
A similar picture could be seen in the upload and download disciplines. Success ratios of 100,0 per- cent or close are a clear indicator of the very high performance levels of the Dutch mobile networks. Although a casual observer might take for granted that almost every file transfer just works, this is in fact not so common – as can be seen when comparing the Dutch results to those of the benchmarks that P3 conducted in other countries.
A little more deviation can however be seen when taking a closer look at the actual download and upload speeds. In order to point out these differences, transfer rates have been measured in two different ways:
With files of fixed size the overall speed of the network was tested, including the time it takes to initiate a speedy data stream. This is important in cases where many small files will be transferred.
Additionally, the “10 seconds measurements“ showed the sustained performance once a stable transmission was achieved. It gives insight on how the network behaves when large files like high-res photos or videos are uploaded or downloaded. The value given for “90% faster“ is what data rate can be expected most of the time.
Interestingly, when downloading small files, KPN and Tele2 achieved slightly higher maximum data rates (“10% faster“), while T-Mobile excelled with slightly higher sustainable speeds (“90% faster“). In the upload category, Tele2 can keep up with occasional top speeds, but the “90% faster“ value shows that its network does not constantly perform at such a high level. Also, in the Vodafone network the upload tests revealed somewhat slower data rates in comparison to the top duo T-Mobile and KPN.
YouTube in standard and high definition
As YouTube and similar video services are the major source of traffic in mobile networks today, the benchmark examines them separately. Again, the stability or success ratios and start times as well as successful playouts without interruptions were on a very high level. As might be expected they dropped a little when accessing the more demanding HD videos, but none of the four Dutch operators showed any weaknesses worth mentioning in this discipline.
Data Access in small cities
While providers usually perform superbly in metropolitan areas, they often lag behind in smaller cities. How noticeable is this effect in the Dutch networks?
In addition to the ten large cities considered on the previous page, the four P3 measurement cars also visited a considerable number of smaller Dutch communities on their test routes. The map on the first page of this report shows which smaller cities were taken into account for this benchmark.
In total, the test cars drove through cities that accomodate about 4.3 million people. Compared to the roughly 17 million inhabitants of the Netherlands, this gives the P3 connect Mobile Benchmark a high degree of statistical validity.
Web page access
In smaller towns and rural areas, mobile network coverage is typically less dense than in the metropolitan areas. So it could be expected that the measurement results and scores drop somewhat in comparison to the bigger cities. It is however striking, how close the values and scores turned out to be – especially when looking at the top three operators, T-Mobile, KPN and Vodafone.
Their success ratios and average session times only rank fractions behind their results in the metropolises. And even Tele2, which by their own account is still working on extending its 4G network coverage to a nationwide footprint, scored convincingly well in this category.
File uploads and downloads
When examining file transfers outside of the big cities, the success ratios and data rates again remained at a very high level. While the maximum and average data rates of uploads and downloads in the KPN, Vodafone and Tele2 networks dropped to some extent in smaller cities, T-Mobile manages to keep them at almost the same level as in the big cities.
Also, when comparing the throughput rates, T-Mobile plays in a league of its own, followed by KPN and in third position by Vodafone. As already observed in the major cities, Tele2 manages to almost keep up with the three stronger candidates in the top 10 percent categories and even outruns Vodafone when downloading large files. But when the majority of cases (“90% faster“) is taken into account, the smallest Dutch operator falls behind. For example, when it comes to uploading smaller files, Tele2 only achieved an average throughput of 925 kbit/s, while in the T-Mobile network, files would complete their journey with an average of 7752 kbit/s. Similarly, download rates (which are even more important for the majority of applications) were in most cases considerably higher in the T-Mobile network than via Tele2.
All Dutch operators (as far as they were considered in last year’s P3 connect Mobile Benchmark) including Tele 2 were however able to improve themselves in almost all disciplines.
YouTube in standard and high definition
The benchmark also indicates that watching videos on smartphones while staying in smaller cities is a very nice experience. As the upload and download results already suggested, video playback works extremely well in the Dutch networks. All four candidates accomplish top values and success ratios.
In both of the video categories (SD as well as HD), T-Mobile and Vodafone take a narrow lead, with KPN and Tele2 following closely. With success ratios and “video playouts without interruptions“ all scoring at or over 99 percent – the only exception being Tele2 with a close 98.6% in the YouTube HD discipline – the results are top notch. This is especially remarkable as the more demanding high-definition (HD) videos obviously did not make much of a difference in comparison to the smaller and less bandwidth consuming standard definition (SD) videos.
All in all, the measurements and scores achieved in smaller cities are impressive evidence for the high performance of the Dutch mobile networks compared to those of many other European countries.
Data on connecting roads
While driving we increasingly demand connectivity for navigation, messaging or just pure entertainment. How do the Dutch networks fulfill these expectations?
A mobile network cell covering a road usually only reaches a relatively small number of customers. This makes the deployment of mobile networks along roads and motorways a costly venture. In order to shed more light on this aspect, the four measurement cars also covered a total of 5,200 kilometres on the connecting roads between Dutch cities – about 1,300 kilometres per car. Whilst driving there, the cars permanently examined the coverage, reliability and data performance of all four candidates’ networks.
Web page access
When comparing the results obtained on connecting roads with those of smaller cities, it does not come as a surprise that performance indicators drop to some extent. But once more it is amazing how close to each other these values actually are. When examining the access to live and static web pages, the resulting success rates and average session times were almost identical to their counterparts on the left page – the results obtained in small cities. So in the benchmarking it made almost no difference whether the test cars were located within a smaller city or driving on the road.
Only Tele2 shows a minor gap between these two environments. But their 97.2 percent success ratio is still a good result – and this operator has not even finished its nationwide 4G deployment.
File uploads and downloads
This general picture is also confirmed regarding file transfers. Each of the four Dutch operators scores almost the same on the connecting roads as in the small cities. This of course means that T-Mobile once more takes the lead concerning data rates, KPN and Vodafone again rank in the midfield, and Tele2 trails behind. But in most of the cases within our benchmark, each of the four Dutch networks provided reliable connectivity on the road – a benefit that many other European mobile network providers can not claim to this extent.
YouTube in SD and (with a little surprise) HD
The observations of the web and file categories are also mostly true for the video tests. Generally the results were very good and did not rank considerably worse on the connecting roads than in small cities.
The heavier load of the HD videos caused somewhat smaller success ratios in the Tele2 network and – somewhat surprisingly – to a lesser extent also in the Vodafone network.
However, as long as video consumption on the road is restrained to standard definition (SD) content, all four Dutch networks once more showed a convincingly high stability and quality.
Hakan Ekmen, Managing Director P3 Communications "By conducting comparable measurements in more than 40countries, we have established ourtesting methodology as a de facto industry standard. The results of our tests in the Netherlands show that the Dutch operators outperform the European networks in terms of stability, quality and performance."