Non-Volatile Memory Express (NVMe) is a specification for accessing solid-state drives (SSDs) attached through the PCI Express (PCIe) bus. It gives lower storage latency and much higher queue depths. Even though this technology is most relevant for server workloads, there are also some affordable new NVMe devices meant for client devices and workloads. A good primer on NVMe is available here.

The use case from nvmexpress.org is that

“NVM Express is architected from the ground up for Non-Volatile Memory (NVM). NVM Express significantly improves both random and sequential performance by reducing latency, enabling high levels of parallelism, and streamlining the command set while providing support for security, end-to-end data protection, and other Client and Enterprise features users need. NVM Express provides a standards-based approach enabling broad ecosystem adoption and PCIe SSD interoperability.”

NVMe is being pushed as a modern replacement for the old Advanced Host Controller Interface (AHCI) that most flash storage devices are still using, and all indications are that NVMe will really start to become more popular and more affordable in 2015/2016.

Windows Server 2012 R2 and Windows 8.1 have a native NVMe driver that allows NVMe devices to be automatically recognized by Windows. This driver works, but does not offer the best performance. I wrote about my experiences with the native NVMe driver last October. Microsoft has also released a hotfix to Windows Server 2008 R2 and Windows 7 that gives native NVMe support to the operating system.

Anandtech has had similar results with several different NVMe devices. Their information (from Samsung) was that

“the performance difference was due to the Microsoft NVMe driver creating FUA (Force Unit Access) I/O write commands. These FUA commands bypass the DRAM cache on the SSD and directly write to the flash, increasing the response time and also lowering bandwidth. For the same access traces, this situation does not happen with the Microsoft AHCI driver.”

This sounds pretty similar to the difference between write-back and write-through caching for RAID controllers. If you have any NVMe storage devices, you should make absolutely sure that you are using the vendor supplied NVMe driver rather than the generic Microsoft NVMe driver. My fear is that it will be very common for many server administrators to simply install their NVMe device, start the server, and then think everything is ok, since Windows recognized the device and it seems to be working.

There are a lot of recent tests of new NVMe storage devices to whet your appetite for this technology. Here are some reviews and tests of client devices:

PCIe SSD Roundup – Samsung SM951 NVMe vs. AHCI, XP941, SSD 750 and More!

Intel SSD 750 Review

Intel 750 series SSD review: Storage so fast, only the highest-end PCs can keep up 

Intel SSD 750 Series NVMe PCI Express SSD Review

Intel 750 Series 1.2TB NVMe PCIe SSD Review

Here are  some reviews of server devices:

Intel SSD DC P3700 Review: The PCIe SSD Transition Begins with NVMe

Intel SSD DC P3700 800GB and 1.6TB Review: The Future of Storage

Intel SSD DC P3700 Review (800GB) – NVMe for Enterprise…and Enthusiasts?

Hopefully, Microsoft will improve the performance of their native NVMe driver in a future update for Windows Server 2012 R2 and Windows 8.1. I certainly hope the native NVMe driver performs better in Windows 10 and “Windows Server 2016”. I would love to see Microsoft’s Jose Barreto weigh in on this subject!