Part 3 of 3 iPad 3 Wi-Fi Performance Analysis – Throughput Analysis (Graphical CDF Statistics)
In the previous two posts – iPad 3 Wi-Fi Reception Analysis – Transmit Power and iPad 3 Wi-Fi Performance Analysis – Throughput – we talked about the differences in power and throughput between iPad versions and a couple of other standard devices. We used some standard statistics like Average and Standard Deviation to show our results.
This post will end in a more graphical version of this very same data. But first a quick description of how this CDF or Cumulative Distribution Function works. It’s not too long – so just hang in there. This is going to be worth your time. Really!
When looking at collected data, sometimes it seems to be quite random in nature. Looking at this random data, folks can make mistakes in analysis. One method we use to help ‘clean up’ this random data is to first sort the collected data from high to low, and graph according to percentage. This allows us to see graphically the differences between data sets.
As an example, I’ve put together the following sample data sets. Each has the exact same Maximum, Minimum and Average… but obviously, much different results. This is the value of this sorting method, it allows one to quickly see differences in data.
The first is a graph showing the two sets of data, fairly random looking. Both look like they are quite similar in nature, both fairly random, and yet they both have the exact same average.
But when you take this same information and sort it first, you can see distinct differences in the resulting graphs. One set of data is much more consistent than the other. Even though they both have the same averages.
The only difference between these two graphs is the data has been sorted high to low, and then plotted with a percentage on the bottom. This percentage allows you to say things like, “The red line shows a 10 or greater throughput 90% of the time”.
We’d like to see very flat lines, showing customer experiences to be fairly consistent across the board. The higher the lines the higher the client’s throughput results.
A line with its curve toward the bottom left represents a fairly low consistent result. A diagonal line represents high variability – more inconsistency. A line with the curve in the upper right represents consistently higher results.
Another way to use these ‘sorted’ graphs is to look at the 50% line – this represents the ‘average’ someone would achieve. The 80% line on the bottom represents that 80% of all collected data meets or exceeds this number.
This is a good telltale sign for following the 80/20 rule. Don’t waste too much time and money trying to fix the last 20% - put the bulk of your resources towards getting the 80% where you need it to be.
This way of plotting data is known as the “cumulative distribution function” (CDF) in statistics.
Now lets use this new CDF statistical method to look back at our iPad data. This will allow us to visually compare results – in contains the Max, Min, and Average – as well as a visual representation of the Standard Deviation. I think you’ll find this much easier to use in evaluation of various scenarios.
Like before, we’ll start with the 2.4GHz Tx, then 2.4GHz Rx.
A couple of things to focus on, first the flatter the line, the more consistent the data set. In the 2.4GHz range – all of these devices perform quite well, and very much like each other, very little differentiation between devices. A very consistent user experience above 85% of the time. Though you can still see the slight improvement, as the iPad’s get newer.
Now on to the 5GHz results.
Note the MacBookAir Tx result is nearly dead flat. This connotes a very stable connection. You can also see the MacBookAir really shines in Rx compared with the iPads – again the multiple spatial streams kick in and give a higher MCS – thus higher net throughput.
Note: Think of your Enterprise Wireless network… and how the adoption of BYOD will affect your throughput compared with having all laptops on your network… hmmm.
You will also see the consistent improvement as Apple released newer iPads. Improvement in Wi-Fi performance is just another reason to upgrade. This is especially pronounced in the 5GHz frequencies. Specifically where we are trying to move as many devices as possible to free up resources in the 2.4GHz band.
This graph shows an iPad 3 at 95% confidence has a nearly flat result – meaning a very consistent user experience.
Using the CDF sortable data as your graphing starting point can add much to the discussion when comparing throughput results.
Using only RSSI can be deceiving, it is all about throughput.
In the 2.4GHz band – all devices are pretty much equal. The differences really come through when you move up to 5GHz.
So if you are contemplating moving to the new iPad – now you have some concrete proof to support your desire for that wonderful new Retina display – you’ll also be getting the best possible Wi-Fi!
Sidenote: I tested with a couple of the new iPads – I didn’t experience any Wi-Fi issues at all in my testing. But I have seen where others are posting they haven’t had as good of experience. I’m not sure why the discrepancy… perhaps different production batch lots? Please leave comments with your experience using the new iPad 3.