Is 2020 the Year NVMe-oF Finally Takes Off?

Article By : Gary Hilson

Is 2020 the year NVMe over Fabrics (NVMe-oF) finally takes off?

Is 2020 the year NVMe over Fabrics (NVMe-oF) finally takes off?

As an extension of the somewhat mature non-volatile memory express (NVMe) protocol, the NVMe-oF specification uses NVMe to connect hosts to storage across a network fabric. NVMe-oF supports the transfer of data between a host computer and a solid-state storage device or system over a network. Using a NVMe message-based command, these transfers can be done via Ethernet, Fibre Channel (FC), or InfiniBand.

A key value proposition of the fabric is it can connect distributed pools of storage (most likely an NVMe SSD, although other persistent memory could potentially be used down the road). Furthermore, with the ability to connect using TCP, it can be done over a distance. The most recent revision of the specification, NVMe-oF 1.1, includes support for the TCP transport binding, which makes it possible to use NVMe-oF across a standard Ethernet network without having to make configuration changes or implement special equipment.

This ability may be what makes this the year that NVMe-oF truly takes off, while extending the core value of NVMe — unlocking the full benefits of NAND flash. It’s something that couldn’t be done by architectures originally built for hard disk drives with protocols such as iSCSI.

The original NVMe specification was published nearly a decade ago. It takes advantage of a computer’s Peripheral Component Interconnect Express (PCIe) bus to transfer data locally. According to the NVM Express organization that oversees the protocols, NVMe-oF is 90% similar to the original protocol. Introduced five years later, NVMe-oF found an early champion in Micron Technology, which opted to move ahead of the standard with its own SolidScale architecture. It was created for low-latency, high-performance access to compute and storage resources and specifically addresses CPU underutilization in the data center. NVMe SSDs deployed in application servers at the time were on average using less than 50% of their IOPS and capacity.

The NVM Express roadmap has steadily expanded the original specification with additions with NVMe-oF, which with TCP transport binding support, provides the ability to scale storage over distances. (Image source: NVM Express) (Click on the image for enlarged view)

The company has since opted to sunset SolidScale, according to Joe Steinmetz, storage system architect and fellow with Micron, as ultimately it didn’t make sense for the company to potentially be competing with some of its own customers. The company will focus instead on its core strengths as an SSD and memory supplier within the NVMe-oF ecosystem. As with many technologies, its adoption is taking longer than expected, but he said 2020 is looking as though it may be the year for NVMe-oF and there’s sufficient alignment within the industry to make it so.

There are barriers to adoption, however. While the inclusion of TCP and a non-RDMA transport was deemed beneficial, it did create some confusion, according to Steinmetz. “That is one reason why NVMe over Fabric has been a bit delayed from when I think a lot of the industry expected to be there sooner,” he said. Another significant barrier is the software ecosystem and depending on how the customer plans to use NVMe-oF, that ecosystem is still in its infancy, added Steinmetz.

While adding TCP did solve the distance challenge, and it helps that TCP is everywhere, it doesn’t negate the applicability of RDMA RoCE, with which many of the storage OEMs have a great deal of experience. “TCP is useful, needed, and will eventually drive adoption. I don’t see RoCE going away,” said Steinmetz.

For the immediate future, Steinmetz sees all-flash storage array vendors as the first movers with NVMe-oF because they want to unlock the value of the expensive flash for customer to enable full access to the high performance and low latency. “We see a lot of applications and early adopters around all flash arrays.”

Subscribe to Newsletter

Leave a comment