You might wonder why the numbers you see in our Wireless Benchmark charts are so different from those that NETGEAR obtained. The main reason is that NETGEAR's results all reflect the total throughput from four simultaneous test streams, while all of the charts show data for single test streams.
802.11n can produce higher total throughput when handling multiple simultaneous connections than it can from a single connection. We see this in the simultaneous up and downlink test that are run in Location A. This is reported in the reviews, but not charted (note to self, think of adding this chart).
NETGEAR also ran their tests in an RF-tight chamber to keep other wireless traffic from affecting results. But this multipath-free environment plus close-range testing (3 M) also adds a bit of throughput boost. Finally, NETGEAR uses IxChariot's High Performance Throughput test script, which can also bump up throughput a bit more.
Although I don't have an RF-tight test chamber, I do have a test environment free of neighboring networks and all of the other pieces required to run a check on NETGEAR's test results. This is because NETGEAR loaned me one of their Dell 5410's loaded with the same test scripts they used to get the results shown in the table in Figure 1, so that I could run my own tests.
I ran the test scripts with the loaner Dell and the WNDR4500 review unit set up in the usual 10 foot apart test location A. Test scripts with data were sent for both 20 MHz (Up to 217 Mbps) and 40 MHz (Up to 450 Mbps) bandwidth modes for both bands. Channel 1 was used in 2.4 GHz and Channel 36 used in 5 GHz.
Figure 6 compares the original NETGEAR data (top) for a 2.4 GHz band, 40 MHz mode uplink test, also run with the E5410 loaner (middle) and my Lenovo X220i. Note that all results shown are the total of four identical test scripts run simultaneously. I also expanded the vertical scale to separate the traces.
My 245 Mbps result didn't match NETGEAR's 265 Mbps, which NETGEAR said was due to throughput enhancement provided by their RF-tight test chamber. But they said the result I obtained was in line with what they'd expect for open air testing. The 221 Mbps I got from running the NETGEAR test script on my Lenovo X220i was only 10% lower than my Dell results.
Figure 6: NETGEAR E5410, SNB E5410, SNB X220i - 2.4 GHz, 40 MHz, uplink
NETGEAR's chart (Figure 1) shows that the X220i turns in a particularly crappy result (120 Mbps) for 5 GHz downlink. So I ran that test and Figure 7 confirms that the Lenovo results are, in fact, pretty bad at 121 Mbps! There is also quite a bit of difference between NETGEAR's E5410 results and mine (18% lower), which I have to attribute to the ol' RF-tight test chamber effects.
Figure 7: NETGEAR E5410, SNB E5410, SNB X220i - 5 GHz, 40 MHz, downlink
I wanted to see if the same relative results could be obtained with single test streams. For this test, I again used 2.4 GHz, but switched to 20 MHz mode and downlink. Figure 8 shows that the Lenovo still lagged behind the Dell, this time by 20%.
Figure 8: SNB E5410, SNB X220i - 2.4 GHz, 20 MHz, downlink
Next up was to check how distance (lower signal level) affected results. I stayed with the 2.4 GHz band and 20 MHz bandwidth mode, but ran simultaneous up and downlink tests and moved the laptops to my Location D. Figure 9 shows that once again the Lenovo turned in lower throughput, about 35% lower for this test. Note that each plot trace show the total of one uplink and one downlink test run simultaneously.
Figure 9: SNB E5410, SNB X220i - 2.4 GHz, 20 MHz, downlink, Location D
For my last trick, er, test, I decided to see if the the X220i produced lower throughput than the E5410, when used with a dual-stream N router. I used my house NETGEAR WNDR3700v1, left in its central utility room location and set the laptops on a desk in my office.
Figure 10 shows the laptop (Location A) and WNDR3700 locations, which are about 35 feet apart and have a three wall diagonal path between them. There was no other wireless activity during testing.
Figure 10: WNDR3700 test location
Table 1 shows 2.4 GHz tests with the Lenovo turning in slightly lower throughput in the 2.4 GHz band except for the uplink test, which was unusually higher. I have no explanation for this outlier.
|Test||Dell E5410||Lenovo X220i||% Difference|
Table 1: Dual stream test summary - 2.4 GHz, 20 MHz mode
Table 2 shows 5 GHz results, with the Lenovo consistently producing 30 - 40% lower throughput.
|Test||Dell E5410||Lenovo X220i||% Difference|
Table 2: Dual stream test summary - 5 GHz, 20 MHz mode
I've essentially come full circle to the point made by NETGEAR's table of results that lead off this article: laptop design can significantly affect your wireless performance. My tests confirm NETGEAR's results, which show a loss of best case performance. But I've also shown that the effects will be seen for single connections under strong and lower level signal conditions for three-stream 802.11n. And even if you're running the dual-stream N that most of us are, your wireless performance will still be affected by the design decisions made by your notebook maker.
Special thanks go to NETGEAR for sharing their data with me, allowing me to share it with you and for the loan of the Dell E5410.