2020 has been a tumultuous year in which the industry has to reevaluate its data center deployment strategy. While COVID-19 and the ensuing recession did weigh down on projected 2020 data center capex growth to just 2%, the slowdown in spending was not as much as originally feared. Some Cloud service providers have continued to expand their infrastructure to support increased internet usage and work-from-home dynamics, while a great deal of uncertainty persists in other industry sectors. Our 2021 outlook is more optimistic, with a data center capex projection of 10%. We identify the following key trends that could shape the dynamics of the data center capex in 2021.

Cloud spending to return to higher growth:

This may not be a surprise given the surge in demand for Cloud services throughout the pandemic. But we project that all of the Top 10 Cloud service providers will increase their data center capex in 2021 by double-digit growth as they revert to an expansion cycle. Data center suppliers such as processor, memory, storage, and optics vendors have positive sentiment going into 2021 and have been proactively expanding capacity.

Soft Enterprise IT spending will persist:

Overall enterprise growth is forecast for tepid growth in 2021. While high-end enterprises are likely to invest in a hybrid Cloud strategy, small and medium enterprises have been making a secular shift to Cloud computing. This trend has materialized, simply because it is less expensive for smaller enterprises to rely on Public Cloud, as opposed to building and operating their own data centers. We expect this trend to accelerate in light of the macroeconomic uncertainties created by the pandemic.

System pricing expected to be higher, creating upside revenue growth for vendors:

While we may see some deflationary commodity pricing in 1H21, inflationary commodity pricing could return in 2H21 as global demand increases, driving system average selling prices higher. Furthermore, Intel’s new processor platform, Ice Lake, which will ramp in 2021, will enable deeper memory, more storage, and faster interconnects, and could drive up server cost. Accelerated compute servers, which can be many times the cost of a general-purpose server, should see greater adoption as well.

Accelerated computing further materializes:

As the number of artificial computing (AI) applications and use-cases increases, so will the deployment of accelerated compute servers with specialized processors, such GPUs, FPGAs, and custom ASICs, along with enhanced cooling designs such as liquid cooling. These specialized processors are designed to handle AI inference and training workloads much more efficiently than general-purpose processors such as CPUs. Smart NICs and data processing units (DPUs) are innovations that will nicely complement CPUs in increasing the efficiency and flexibility of the data center.

Intel will continue to dominate the data center CPU market, despite new entrants:

While AMD has made share gains and Intel’s data center business slipped 2020, we project Intel to retain a strong leadership position going into 2021. Intel still has a commanding share among the Top 10 Cloud service providers, and this is a market that will undergo an expansion in 2021 with the ramp of the new Intel Ice Lake processor platform. Nevertheless, we expect all the major vendors to increase their offerings of Intel and AMD x86 for enterprise servers. Furthermore, we believe that there are opportunities for ARM as well in niche markets and applications.

The 3^rd AI Hardware Summit took place virtually earlier this month and it was exciting to see how quickly the ecosystem has evolved and to learn of the challenges the industry has to solve in scaling artificial intelligence (AI) infrastructure. I would like to share highlights of the Summit, along with other notable observations from the industry in the area of accelerated computing.

The proliferation of AI has emerged as a disruptive force, enhancing applications such as image and speech recognition, security, real-time text translation, autonomous driving, and predictive analytics. AI is driving the need for specialized solutions at the chip and system level in the form of accelerated compute servers optimized for training and inference workloads at the data center and the edge.

The Tier 1 Cloud service providers in the US and China lead the way in the deployment of these accelerated compute servers. While the deployment of these accelerated compute servers still occupies a fraction of the Cloud service providers’ overall server footprint, this market is projected to grow at a double-digit compound annual growth rate over the next five years. Most accelerated server platforms shipped today are based on GPUs and FPGAs from Intel, Nvidia, and Xilinx, the number of new entrants, especially for the edge AI market, is growing.

However, these Cloud service providers, or enterprises deploying AI applications, simply cannot increase the number of these accelerated compute servers without addressing bottlenecks at the system and data center level. I have identified some notable technology developments that need to be addressed to advance the proliferation of AI:

• • Rack Architecture: We have observed a trend of these accelerated processors shifting from a distributed model (i.e., one GPU in each server), to a centralized model consisting of an accelerated compute server with multiple GPUs or accelerated processors. These accelerated compute servers have demanding thermal dissipation requirements, oftentimes requiring unique solutions in form-factor, power distribution, and cooling. Some of these systems are liquid-cooled at the chip level, as we have seen with the Google TPU, while more innovative solutions such as liquid immersion cooling of entire systems are being explored. As these accelerated compute servers are becoming more centralized, resources are pooled and shared among many users through virtualization. NVIDIA’s recently launched A100 Ampere takes virtualizing to the next step with the ability to allow up to seven GPU instances with a single A100 GPU.
  • CPU: The GPU and other accelerated processors are complementary and are not intended to replace the CPU for AI applications. The CPU can be viewed as the taskmaster of the entire system, managing a wide range of general-purpose computing tasks, with the GPU and other accelerated processors performing a narrower range of more specialized tasks. The number of CPUs also needs to be balanced with the number of GPUs in the system; adequate CPU cycles are needed to run the AI application, while sufficient GPU cores are needed to parallel process large training models. Successive CPU platform refreshes, either from Intel or AMD, are better optimized with processing inference frameworks and libraries, and support higher I/O bandwidth within and out of the server.
  • Memory: My favorite session from the AI Hardware Summit was the panel discussion on memory and interconnects. During that session, experts from Google, Marvell, and Rambus shared their views on how memory performance can limit the scaling of large AI training models. Apparently, the abundance of data that needs to be processed in memory for large training models on these accelerated compute servers is demanding greater amounts of memory. More memory capacity means more modules and interfaces, which ultimately degrades chip-to-chip latencies. One proposed solution that was put forth is the use of 3D stacking to package chips closer together. High Bandwidth Memory (HBM) also helps to minimize the trade-off between memory bandwidth and capacity, but at a premium cost. Ultimately, the panel agreed that there needs to be an optimal balance between memory bandwidth and capacity within the system, while adequately addressing thermal dissipation challenges.
  • Network Connectivity: As these accelerated compute nodes become more centralized, a high-speed fabric is needed to ensure the flow of huge amounts of unstructured AI data over the network to accelerated compute servers for in-memory processing and training. These connections can be server-to-server as part of a large cluster, using NVIDIA’s NVlink and InfiniBand (which NVIDIA acquired with Mellanox). Ethernet, now available up to 400 Gbps, is an ideal choice for connecting storage and compute nodes within the network fabric. I believe that these accelerated compute servers will be the most bandwidth-hungry nodes within the data center, and will drive the implementation of next-generation Ethernet. Innovations, such as Smart NICs, could also be used to minimize packet loss, optimize network traffic for AI workloads, and enable the scaling of storage devices within the network using NVMe over Fabrics

I anticipate that specialized solutions in the form of accelerated computing servers will scale with the increasing demands of AI, and will comprise a growing portion of the data center capital expenditures. Data centers could benefit from the deployment of accelerated computing, and would be able to process AI workloads more efficiently with fewer, but more powerful and denser accelerated servers. For more insights and information on technology drivers shaping the server and data center infrastructure market, take a look at our Data Center Capex report.

Unlike many enterprises that have been migrating their IT infrastructure to the Public Cloud in recent years, the financial sector continues to be an important contributor to purchasing and deploying IT, which includes data center infrastructure such as servers, storage, and networking equipment. Last year, we estimated that roughly 24% of the on-premise enterprise server installed base among Fortune 2000 firms to be attributed to the financial sector, with high-frequency trading among a major use-case. We expect the financial sector to gain share in the on-premise enterprise server installed base, as traditional enterprises in the areas of oil & gas, offline retail, travel, and hospitality, and healthcare are projected to reduce spending on IT this year due to weak economic outlook. Growth in those traditional enterprise industries tends to mirror that of the broader economy, and could face a sluggish road to recovery ahead.

On Premise Server Unit Installed Base by Fortune 2000 Industry Segment (2019)

 

Other observations related to data center capex trends in the financial sector:

• Data Center IT is moving to the Public Cloud (i.e. AWS, Microsoft Azure, and Google Cloud), and the pandemic has accelerated Cloud adoption as companies prefer an operating expenditure-based IT consumption model to conserve capital in uncertain economic times, and the ease in which cloud applications could be provisioned to a remote workforce. However, we expect the financial sector will be one of the last enterprise sectors to move the workload to the Cloud. Applications such as high-frequency trading will continue to be owned and operated by these enterprises due to proximity requirements and demands an ultra-low latency network. Regulations and data sovereignty requirements also dictate that certain data and workloads need to remain in on-premise data centers (and not move to the Cloud). We estimate that over 80% of the workloads for the financial sector have remained on-premise, versus, 20% of the workload outsourced to the Cloud.

• Some of these major financial institutions such as JP Morgan and Goldman Sachs own and operate their own data centers, which are also complemented by colocation data centers that are operated by Equinix for instance. In the first half of 2020, we estimate a 2% average Y/Y growth in revenue generated by the financial sector among the colocation providers. That is versus an estimate of 5% average Y/Y growth in 2019. While the 1H20 growth rate is conservative, the financial sector is expected to outperform that of other enterprise sectors when it comes to colocation infrastructure spending this year.

• The large financial institutions are innovators of server and data center architecture. While some have offloaded some requirements to the Public Cloud, they continue to invest in their own data centers. Some of the large firms operate hundreds of thousands of servers and would be able to benefit from economies of scale a Cloud service provider would. The more servers that these firms have, the more transactions and customers they can support. While this segment still purchases IT equipment from branded server and storage system vendors such as HPE, Dell, and IBM, some of the more sophisticated firms have customized servers in their data centers in an effort to streamline their architecture. They have also deployed accelerators such as Smart NICs (network interface cards) to reduce network latency necessary for high-frequency trading and to ensure secure connections. Specialized servers, equipped with accelerators such as GPUs for instance, are also deployed for AI-centric applications pertaining to predictive analysis and risk management computations.

In order to get additional insights and outlook for spending on servers and other data center center equipment by Cloud and Enterprise segments, please check out our Data Center Capex report.

 

We just finished our first Data Center Capex July 2020 5-year Forecast Report. Below are some highlights from the report. If you need to access the full report, please contact us at dgsales@delloro.com.

Data center capex, which includes capex for servers and other data center infrastructure equipment, is forecasted to grow at a 6% CAGR to just over $200 B over the next five years. Growth is forecasted to be mixed depending on the customer segment. The Cloud, which already accounts for more than 60% of the worldwide data center capex, will continue to gain momentum over Enterprise/On-premise data center deployments. Edge data centers deployed over Telco networks could emerge in the longer-term horizon.

Capex on servers, which generally accounts for nearly half of the data center capex, may be influenced by the following factors:

• • Change in server unit demand from Cloud capacity and digestion cycles.
  • Market volatility of commodity pricing of components such as memory.
  • Server refresh cycles, which could prompt the replacement of aged servers and drive new deployments, could impact server architecture and pricing.

Servers also drive the demand for auxiliary data center infrastructure equipment such as networking switches, storage systems, and facilities.

The COVID-19 pandemic is expected to profoundly disrupt global demand for data center infrastructure equipment in 2020. Impacted vertical industries, especially brick-and-mortar retail, travel, hospitality, and small and medium enterprises, have seen a pull-back in IT spending as they wait for the business climate to stabilize. As enterprises seek to conserve capital, Public Cloud, which offers a flexible and consumption-based infrastructure, could help meet the growing demands of remote work and distance learning. The COVID-19 pandemic and the ensuing recession may have the long-lasting effect of accelerating the permanent migration of certain industries and workloads to the Cloud.

Market and Technology Trends to Watch Out For

• The Top 4 U.S. Cloud service providers—Amazon, Facebook, Google, and Microsoft—are positioned to continue their momentum of expansion over the next five years. Servers will continue to be consolidated in fewer mega Cloud data centers that could potentially provide greater capacity than the same number of servers spread out across thousands of Enterprise data centers.
• The Top 4 U.S. Cloud service providers have been prolonging the useful life of servers in an effort to lower server depreciation expense while maintaining high efficiencies and reliability of their server fleet.
• The Intel server processor refresh cycles have historically influenced IT spending. While the major Cloud service providers typically ramp server capacity outside of the processor refresh cycle, the upcoming Intel 10 nm Whitley server platform refresh due later this year could generate an uplift on server spending. Viable alternatives to Intel processors, AMD EPYC and ARM, for server and storage system applications are starting to materialize in certain markets.
• Various open-source organizations have come together to share and standardize best practices in the design of efficient, scalable, and sustainable data center infrastructure. The Open Compute Project (OCP), in particular, has introduced various technological innovations in the areas of server and server connectivity, rack architecture, and networking switches, which could shape the future development of data center infrastructure.

To learn more about the COVID-19 impact on Data Center Infrastructure and Server spending, please click here to watch my latest video.

About the Data Center Capex 5 Year Forecast Report

Dell’Oro Group’s Data Center Capex 5-Year Forecast Report details the data center infrastructure capital expenditures of each of the ten largest Cloud service providers, as well as the Rest-of-Cloud, Telco, and Enterprise customer segments. Allocation of the data center infrastructure capex for servers, storage systems, and other auxiliary data center equipment is provided. The report also discusses the market and technology trends that can shape the forecast. Highlights from the Server and Storage System Report (now discontinued) was transitioned to this report. Click here to learn more about the report or contact us (dgsales@delloro.com) for the full report.

 

Related video about Data Center Capex:

---

Sign up to Dell’Oro Analyst Talk channel at BrightTalk to watch the full video