At this week’s NVIDIA GPU Technology Conference, the company announced its groundbreaking Bluefield -2X DPU (Data Processing Unit), which combines key elements of the existing Bluefield DPU and NVIDIA’s Ampere GPU. The new DPU enables the use of AI to perform real-time security analytics, and identify abnormal traffic and malicious network activities. The building-blocks NVIDIA needs to enable high-performance and AI workloads in the data center and edge are coming together—this includes the GPU for application-specific workloads, the DPU to facilitate data I/O processing, and finally, the CPU for compute, as NVIDIA seeks to complete its acquisition of ARM.
The DPU, a terminology coined by NVIDIA, falls within Dell’Oro Group’s definition of the Smart NIC market that we track in the Ethernet Controller and Adapter research. Smart NICs are fully programmable network interface cards designed to accelerate key data center security, networking, and storage task functions, offloading valuable CPU cores to run business-oriented applications.
According to Dell’Oro Group’s latest forecast, the Smart NIC market will grow at a 26% compound annual growth rate, from $270M in 2020 to $848M by 2024, vastly outpacing the overall Ethernet controller and adapter market. We believe that this new class of Smart NIC with integrated AI that NVIDIA has introduced, to be potentially disruptive, and could expand the range of applications that have been available to Smart NICs.
As Cloud and Enterprise data centers continue to scale, we believe that Smart NICs could be a solution in achieving high network throughput, low latency, and minimal packet loss demanded by emerging applications such as high-performance computing and AI. However, there are some notable constraints vendors would need to address before we see stronger adoption of Smart NICs:
- The scalability of Smart NICs depends on key considerations such as price and power consumption. While we are still awaiting details from NVIDIA, I believe that the inclusion of both the ARM processor and GPU in the Bluefield-2X DPU could result in higher unit cost and power consumption compared to that of alternatives. NVIDIA announced that future generations of the Bluefield, such as the Bluefield-4X DPU, will have an integrated ARM and GPU cores, which could result in total cost of ownership improvements.
- Most Smart NICs shipped deployed are based on the ARM architecture, including that of the NVIDIA Bluefield DPU family. However, other Smart NIC vendors such as Intel, Napatech, and Xilinx have released FPGA-based solutions that have demonstrated benefits in adapting to a wide range of applications in conjunction with AI inferencing applications. I predict that both ARM-based and FPGA-based solutions will coexist and be optimized for different use-cases.
- Extensive engineering resources and lead-time are required to bring Smart NICs to market. While the major Cloud service providers have the engineering resources devoted to Smart NIC application development, the smaller Cloud service providers and enterprises do not. It is crucial for vendors to provide value-added services and application development toolkits to customers for software implementation. NVIDIA announced the availability of the NVIDIA DOCA SDK, which enables developers to rapidly create applications and services on top of the Bluefield DPU.
Ultimately, customers will need to find a balance between the benefits Smart NICs could bring, along with the aforementioned considerations. However, I am excited that the vendor community is bringing new innovations to the market that could give customers more choices to implement the solutions that fit their requirements. As servers continue to become more commoditized overtime, Smart NICs could shift more control of the data center architecture back to the customers.
