Just how fast is that SSD?
Every now and again, resetting the benchmark charts is necessary. Now is such a time, for several reasons. First, we have a new testbed (Skylake) that enables potentially higher performance, particularly for M.2 PCIe drives. Second, some of the benchmarks we were using have gotten a bit long in the tooth and needed an update. We want to have a good selection of both drives and benchmarks for use in future reviews, so we've wiped the slates clean and will have several SSD reviews over the coming weeks. First things first, let's talk about our SSD test bed.
Initially, we had intended to use the same LGA2011-3 test bed that we use for graphics cards reviews, but then we ran into a problem. The Gigabyte X99-UDH4 motherboard has NVMe support and an M.2 slot, but the M.2 slot only gets x2 PCIe 2.0 lanes and bandwidth. That puts the brakes on faster M.2 drives, capping throughput at less than 1000MB/s. What's the point in splurging on a high-end M.2 NVMe drive if you can't even utilize its potential? With Skylake having recently launched, we had a second option: move to a Z170 motherboard. The ASUS Z170-A is more of a mainstream offering, but it does support an M.2 slot with x4 PCIe 3.0 connectivity, which is enough for current SSDs. With that in mind, here's our new SSD test system:
One of the key goals is to provide a selection of reference points. To that end, we've selected three of the top performing options. Setting the high water mark, the Intel SSD 750 1.2TB is one of the first NVMe enabled drives you can buy. There's a drawback in that you have to use a PCIe slot (x4 PCIe 3.0 for maximum performance), but if you have the space, it will provide all the performance you need from your storage subsystem.
The reality, of course, is that most users are still on SATA, and many are running RAID 0 as a way of improving throughput and/or capacity while keeping costs down. The 2x250GB Samsung 850 EVO provides a great reference point as a "bang for the buck" solution, with a price currently sitting south of $200. That's a far cry from the $439 you'll pay for the 1TB Samsung 850 Pro, which is arguably the fastest SATA drive you can currently buy, though it's also half the capacity. Note also that a single 500GB 850 EVO only costs $162, so you're paying a bit extra for RAID 0—though two 500GB drives actually cost less than a single 1TB drive.
For those that want a single high-capacity SSD, our final baseline SSD reference point is the Samsung 850 Pro 1TB. Sporting Samsung's V-NAND and with a high-performance controller, the 850 Pro remains the fastest SATA drive across a broad range of benchmarks. It also has a high endurance rating, roughly 6,000 Program/Erase cycles, which is far more than the TLC drives that are becoming increasingly common.
The rest of the system is more than enough to ensure there are no other bottlenecks for storage, though in some cases they're obviously overkill. (Hello, Mr. 850W PSU!) The be quiet! case, CPU cooler, and power supply keep noise levels down, and we use the same system for other testing at times, so having the option of running a GPU or two is important.
Meet the Benchmarks
Our benchmarks include a mix of real-world and synthetic testing. Each test is run multiple times, ensuring we have repeatable results. In many instances, the first run may show higher (or lower) performance, but subsequent testing will usually show results within a narrow range. Here's what we're using along with the settings for each:
The latest version of AS SSD (1.8.5636.37293) provides theoretical sequential and random transfer rates, and typically represents the best-case and worst-case performance you'll see from an SSD. We use 10GB of data for the sequential and 4K-64 thread results and 1GB for the 4K-single thread results. We looked at several other similar pieces of software, specifically ATTO 3.05 and CrystalDiskMark 4.1.0, but the data provided was largely redundant with AS SSD.
IOmeter is our second theoretical benchmark, again looking at sequential and random transfer rates. We run 10 tests of mixed read/write performance, from 100 percent read/0 percent write to 0 percent read/100 percent write in 25 percent increments. We do this for both 100 percent sequential and 100 percent random IO; we then average (technically, we use the geometric mean so that all results are given equal weight) the five test runs for sequential and random performance to give an aggregate value. The big difference with IOmeter is that each test is run for five minutes, which means the SSDs get a bigger workout than the usual 10–30 second load that you'll see from AS SSD and similar utilities (25 minutes of total testing per average score). Some drives are able to maintain a high transfer rate for shorter periods of time, and this should stress the hardware enough to give a realistic measurement of sustained performance.
For the "real world", we have a file copy test. This test copies just shy of 20GB of data (the contents of our Steam Batman: Arkham Origins folder) from one folder on the drive to another folder, so the workload is split 50/50 between reads and writes. Since this test uses core Windows functionality, the results are likely to differ from the theoretical testing the other benchmarks provide. We use Windows PowerShell to perform the copy and time how long it takes for the copy to complete; we then calculate the MB/s based on the amount of data copied.
Finally, we have PCMark 8's Storage test, with two metrics: the overall Storage score, as well as the Storage Bandwidth score. Rather than a pure test of storage performance, PCMark 8 uses traces of real-world applications and runs these on the test drive. What's interesting to note with PCMark 8 Storage is that even substantially faster drives tend to deliver diminishing returns. We'll see this as significantly higher bandwidth results but only marginally higher overall scores.
For most users, PCMark 8 Storage is a good representation of the sort of performance you will actually get from your device. Rarely are desktop users maxing out transfer rates on an SSD for one minute, let alone several minutes; instead, most applications will hit the storage hard for a few seconds and then wait on the user. For example, consider a game where loading a level takes 5–15 seconds, followed by the user playing that level for long periods of time; a fast SSD might load a few seconds quicker than a slow drive, but after the initial work, the SSD goes on siesta. If AS SSD is a best-case result, the PCMark 8 score is more of a worst-case result. What's interesting is that the bandwidth result of PCMark 8 Storage matches up pretty well with our other tests, again illustrating how faster storage only gets you so far (unless you're doing a lot of file copying).
Setting the Stage
With the benchmarks and hardware established, here's what our initial testing results look like. This is a starting point, and while the three drive configurations listed here will remain, we will include additional drives as we go forward with testing and reviews. We've gone ahead and highlighted the Intel SSD 750 NVMe drive in our charts; it should come as little surprise that this is the drive to beat right now when it comes to flat-out smoking storage performance. Our reviews of newer SSDs will generally fall into one of four categories: contending with the SSD 750, somewhere between the 850 Pro and the SSD 750, somewhat close to the 850 Pro, or significantly slower than the 850 Pro. There's plenty of information to cover, so let's quickly run through each benchmark and discuss what the results mean.
First off, the AS SSD sequential throughput is generally the best-case result; accessing large files will generally correlate with these figures. Sequential read speeds are quite a bit higher than sequential write speeds, as writing data to NAND is a more involved process. The next two results are for purely random file accesses, but in this case AS SSD is only using a queue depth of one (a single thread); it's about as bad as it can get, and the throughput reflects this. Even the mighty SSD 750 only musters 36MB/s, and it's effectively tied with the EVO 850 RAID 0 for read speeds; write speeds, on the other hand, still heavily favor the robust controller in the Intel drive. Last up for AS SSD, we have random IO again, but this time with a queue depth of 64 (64 threads). This gives the drives lots of data to deal with and the controllers can usually find parallel accesses that allow them to optimize performance. Better controllers and faster NAND help here, and RAID 0 can also boost throughput. It's worth pointing out, however, that most consumer workloads will never come anywhere near a queue depth of 64; in fact, even a queue depth of 5-10 is uncommon except for short bursts.
Next up, IOmeter shows some similar results to AS SSD, but throughput drops off quite a bit due to the amount of data that's accessed over the length of the test. Consider that the 850 Pro 1TB averages over 500MB/s reads and writes for 25 minutes; that's about 750GB of data being accessed, and roughly 375GB of writes. Hard drives can actually generate some decent numbers in the sequential tests, and we measured 94MB/s write on a Seagate 3TD drive. It's the random access patterns where SSDs are often a couple of orders of magnitude (100x) faster than hard drives; the same Seagate drive scored a dismal 0.50MB/s in our random testing.
If you're worried about burning through all of an SSD's program/erase cycles, don't be. The 850 Pro 1TB is rated for roughly 6000TB of writes. Even with 5X write amplification (which is normally under 2X in client workloads), you could still write over 500GB of data per day for five years without having the drive go kaput. In a more realistic scenario, you could write 1.5TB of data every day for five years without running into problems, or if you're like a typical client user and you're pushing less than 100GB of writes per day, it would take 82 years to burn through the P/E cycles on the NAND. Needless to say, it's far more likely some other element on the SSD would give out first, and in 10 years the 850 Pro will be a dinosaur even if it's still running.
With all of the massive throughput results so far, you might think copying files on an SSD would show similar results. We're dealing with nearly 9000 files totaling 20GB, however, and many of these are smaller files. This leads to a real-world scenario where performance falls somewhere between the pure sequential and pure random performance. Intel's NVMe drive still wins out by a large margin, followed by the RAID 0 850 EVO, then the solo 850 Pro. Don't jump to conclusions about the 850 Pro being outclassed, however—we're looking at three of the fastest SSD storage options currently available; there are plenty of SSDs that don't do very well in these tests. If you want a fun comparison, running this same file copy test on a 3TB Seagate hard drive resulted in throughput of 67MB/s; it's still a mostly sequential IO test, so the HDD isn't absolutely terrible, but it definitely feels slow compared to a good SSD.
Last but not least, PCMark 8 Storage shows two very conflicting views. In terms of pure bandwidth (throughput), the numbers are similar to our file copy test. This is what happens when the bottleneck is your storage subsystem. But if you're doing lots of other things, which is how most people use their PCs, the overall PCMark 8 Storage score reflects this. The SSD 750 delivers 80 percent higher throughput compared to a single 850 Pro, but in the overall Score metric it's only 1.6 percent faster. This is why, for many users, going out and spending a lot of money on the fastest SSDs isn't really necessary. If you're just surfing the Internet, playing some games, and doing everyday Microsoft Office work, an SSD will still feel a lot snappier than a hard drive… but even a slow SSD will feel about the same as the fastest SSD.
Storage Matters
With the stage now set, we're ready to start posting new SSD reviews. Ideally, we love the performance offered by NVMe drives, and we're certainly looking forward to Intel and Micron's XPoint Technology. They've told us that we should see consumer and enterprise class SSDs using XPoint in 2016, which is awesome… but we'll believe it when we see it. Unfortunately, for the vast majority of users, the cost of current NVMe drives is very high, plus you need a compatible motherboard. That means SATA is still a viable option, and in many cases it may simply be the best solution.
Do you want a 1.2TB Intel SSD 750 taking up a PCIe slot, or would you rather save $600 and give up a bit of capacity and a moderate amount of real-world performance by going with a 1TB 850 Pro? Or save an additional $100 and move to the 1TB 850 EVO? Our benchmarks also show that sticking two SSDs together in RAID 0 can definitely improve performance over a single drive, so if you're not space constrained that's a compelling alternative to expensive hardware.
As we go forward with SSD reviews, these drives are the heavyweights that you need to beat. If you make an NVMe drive that costs as much as the Intel SSD 750, it needs to at least equal it on performance and features. Alternatively, less expensive NVMe offerings that clearly beat SATA drives but may not reach the top of the charts are worth considering. On the SATA side of things, the Samsung 850 Pro and 850 EVO deliver great performance, with the EVO priced extremely competitively. Drives either need to beat Samsung on price, features, and/or performance to stand a chance.
Given the newness of NVMe and M.2 drives, along with their associated cost and hardware requirements, there's still a big market for SATA drives. Perhaps by the time Cannonlake rolls out, we'll see pricing on NVMe drives reach the point where they're no longer out of reach of mainstream users. And if we're lucky, we'll also have an interface to our storage devices with enough bandwidth to run multiple NVMe drives at full performance. With 3D-NAND, XPoint, and other new technologies, the next couple of years in the SSD arena are sure to prove exciting.
