March 2025 - Dell'Oro Group

NVIDIA’s Vision for the Future of AI Data Centers: Scaling Beyond Limits

At NVIDIA GTC, Jensen Huang’s keynote highlighted NVIDIA’s growing presence in the data center market, which is projected to surpass $1 trillion by 2028, in reference to Dell’Oro Group’s forecast. NVIDIA is no longer just a chip vendor; it has evolved into a provider of fully integrated, rack-scale solutions that encompass compute, networking, and thermal management. During GTC, NVIDIA also announced an AI Data Platform that integrates enterprise storage with NVIDIA accelerated computing to enable AI agents to provide real-time business insights to enterprise customers. This transformation is redefining how AI workloads are deployed at scale.

GTC2025 keynote Jensen Huang — Source: Nvidia GTC 2025

The Blackwell Platform: Optimized for AI Training and Reasoning

The emergence of NVIDIA’s Blackwell platform represents a major leap in AI acceleration. Not only does it excel at training deep learning models, but it is also optimized for inference and reasoning—two key drivers of hyperscale capital expenditure growth in 2025. Reasoning models, which generate a significant number of tokens, operate differently from conventional AI models. Unlike traditional AI that directly answers queries, reasoning models use “thinking tokens” to process and refine their responses, mimicking cognitive reasoning. This process significantly increases computational demands significantly.

The Evolution of Accelerated Computing

The unit of accelerated computing is evolving rapidly. It started with single accelerators, progressed to integrated servers like the NVIDIA DGX, and has now reached rack-scale solutions like the NVIDIA GB200 NVL72. Looking ahead, NVIDIA aims to scale even further with the upcoming Vera Rubin Ultra platform, featuring 572 GPUs interconnected in a rack. Scaling up AI clusters introduces new challenges in interconnects and power density. However, as compute nodes scale into the hundreds of thousands (and beyond), the industry needs to address several key challenge:

1) Increasing Rack Density

AI data centers aim to pack GPUs as closely as possible to create a coherent compute fabric for large language model (LLM) training and real-time inference. The NVL72 already features extremely high density, necessitating liquid cooling for heat dissipation. With further scaling, interconnect distances will increase. The question arises: will copper cabling remain viable, or will the industry need to transition to optical interconnects, despite their higher cost and power inefficiencies?

2)The Shift to Multi-Die GPUs

To boost computational capacity, increasing GPU die size has been one approach. However, with the Vera Rubin platform, GPUs have already reached the reticle limit, necessitating a shift to multi-die architectures. This will increase the physical footprint and interconnect distance, posing further engineering challenges.

3) Surging Rack Power Density

As GPU size and node count increase, rack power density is skyrocketing. NVIDIA’s GB200 NVL72 racks already consume 132 kW, and the upcoming Rubin Ultra NVL572 is projected to require 600 kW per rack. Given that AI data centers typically operate within a 50 MW range, fewer than 100 racks can be housed in a single facility. This constraint demands a new approach to scaling AI infrastructure.

4)Disaggregating AI Compute Across Data Centers

As power limitations become a bottleneck, AI clusters may need to be strategically distributed across multiple data centers based on power availability. This introduces the challenge of interconnecting these geographically dispersed clusters into a single virtual AI compute fabric. Coherent optics and photonics-based networking may be necessary to enable low-latency interconnects between data centers separated by miles. NVIDIA’s recently introduced silicon photonics switch may be part of this solution, at least from the standpoint of lowering power consumption, but additional innovations in data center interconnect architectures will likely be required to meet the demands of large-scale distributed AI workloads.

The Future of AI Data Centers

As NVIDIA continues to innovate, the next generation of AI data centers will need to embrace new networking technologies, reimagine power distribution, and pioneer novel solutions for high-density, high-performance computing. The future of AI isn’t just about more GPUs—it’s about building the infrastructure to support them at scale.

Related blog: Insights from GTC25: Networking Could Tip the Balance in the AI Race

With a wave of announcements coming out of GTC, countless articles and blogs have already covered the biggest highlights. Rather than simply rehashing the news, I want to take a different approach—analyzing what stood out to me from a networking perspective. As someone who closely tracks the market, it’s clear that AI workloads are driving a steep disruption in networking infrastructure. While a number of announcements at GTC25 were compute related, NVIDIA made it clear that implementations of next generation GPUs and accelerators wouldn’t be made possible without major innovations on the networking side.

1) The New Age of AI Reasoning Driving 100X More Compute Than a Year Ago

Jensen highlighted how the new era of AI reasoning is driving the evolution of scaling laws, transitioning from pre-training to post-training and test-training. This shift demands an enormous increase in compute power to process data efficiently. At GTC 2025, he emphasized that the required compute capacity is now estimated to be 100 times greater than what was anticipated just a year ago.

2) The Network Defines the AI Data Center

The way AI compute nodes are connected will have profound implications on efficiency, cost, and performance. Scaling up, rather than scaling out, offers the lowest latency, cost, and power consumption when connecting accelerated nodes in the same compute fabric. At GTC 2025, NVIDIA unveiled plans for its upcoming NVLink 6/7 and NVSwitch 6/7, key components of its next-generation Rubin platform, reinforcing the critical role of NVLink switches in its strategy. Additionally, the Spectrum-X switch platform, designed for scaling out, represents another major pillar of NVIDIA’s vision (Chart). NVIDIA is committed to a “one year-rhythm”, with networking keeping pace with GPU requirements. Other key details from NVIDIA’s roadmap announcement also caught our attention, and we are excited to share these with our clients.

3) Power Is the New Currency

The industry is more power-constrained than ever. NVIDIA’s next-generation Rubin Ultra is designed to accommodate 576 dies in a single rack, consuming 600 kW—a significant jump from the current Blackwell rack, which already requires liquid cooling and consumes between 60 kW and 120 kW. Additionally, as we approach 1 million GPUs per cluster, power constraints are forcing these clusters to become highly distributed. This shift is driving an explosion in the number of optical interconnects, both intra- and inter-data center, which will exacerbate the power challenge. NVIDIA is tackling these power challenges on multiple fronts, as explained below.

4) Liquid-Cooled Switches Will Become a Necessity, Not a Choice

After liquid cooling racks and servers, switches are next. NVIDIA’s latest 51.2 T SpectrumX switches offer both liquid-cooled and air-cooled options. However, all future 102.4 T Spectrum-X switches will be liquid-cooled by default.

5) Co-packaged Optics (CPO) in Networking Chips Before GPUs

Another key reason for liquid cooling racks is to maximize the number of GPUs within a single rack while leveraging copper for short-distance connectivity—”Copper when you can, optics when you must.” When optics are necessary, NVIDIA has found a way to save power with Co-Packaged Optics (CPO). NVIDIA plans to make CPO available on its InfiniBand Quantum switches in 2H25 and on its Spectrum-X switches in 2H26. However, NVIDIA will continue to support pluggable optics across different SKUs, reinforcing our view that data centers will adopt a hybrid approach to balance performance, efficiency, and flexibility.

6) Impact on Ethernet Switch Vendor Landscape

According to our AI Networks for AI Workloads report, three major vendors dominated the Ethernet portion of the AI Network market in 2024.

However, over the next few years, we anticipate greater vendor diversity at both the chip and system levels. We anticipate that photonic integration in switches will introduce a new dimension, potentially reshaping the dynamics of an already vibrant vendor landscape. We foresee a rapid pace of innovation in the coming years—not just in technology, but at the business model level as well.

Networking could be the key factor that shifts the balance of power in the AI race and customers appetite for innovation and cutting-edge technologies is at an unprecedented level. As one hyperscaler put it during a panel at GTC 2025: “AI infrastructure is not for the faint of heart.”

For more detailed views and insights on the AI Networks for AI Workloads report, please contact us at dgsales@delloro.com.

Conditions improved in the second half, but overall, it was a challenging year for the telecom suppliers. Preliminary findings suggest that worldwide telecom equipment revenues across the six telecom programs tracked at Dell’Oro Group—Broadband Access, Microwave & Optical Transport, Mobile Core Network (MCN), Radio Access Network (RAN), and SP Router & Switch—declined 11% year-over-year (YoY) in 2024, recording the steepest annual decline in more than 20 years (decline was >20% in 2002), propelling total equipment revenue to fall by 14% over the past two years. This remarkable output deceleration was broad-based across the telecom segments and driven by multiple factors, including excess inventory, challenging macro environment, and difficult 5G comparisons.

In 4Q24, stabilization was driven by growth in North America and EMEA, which nearly offset constrained demand in Asia Pacific (including China).

The full-year decline was uneven across the six telecom programs. Optical Transport, SP Routers, and RAN saw double-digit contractions, collectively shrinking by 14% in 2024. Microwave Transport and MCN experienced a more moderate combined decline in the low single digits, while Broadband Access revenues were fairly stable.

Similarly, regional developments were mixed in 2024. While the slowdown was felt across the five regions — North America, EMEA, Asia Pacific, China, and CALA — the deceleration was more pronounced in the broader Asia Pacific region, reflecting challenging conditions in China and Asia Pacific outside of China.

Supplier rankings were mostly unchanged globally, while revenue shares shifted slightly as both Huawei and Ericsson positions improved. Overall market concentration was stable with the 8 suppliers comprising around ~80% of the worldwide market in 2024.

Rankings changed outside of China. Initial estimates suggest Huawei passed Nokia to become the #1 supplier, followed by Nokia and Ericsson. Huawei’s revenue share outside of China was up 2 to 3 percentage points in 2024, relative to 2021, while Ericsson is down roughly two percentage points over the same period/region.

Market conditions are expected to stabilize in 2025 on an aggregated basis, though it will still be a challenging year. The analyst team is collectively forecasting global telecom equipment revenues across the six programs to stay flat.

A Landmark Acquisition in Cybersecurity History

In one of the most significant cybersecurity acquisitions ever, Google announced its intention today to purchase Wiz, a fast-growing cloud-native security firm, for an unprecedented $32 B. This historic deal dwarfs other notable cybersecurity transactions in recent history, including Thoma Bravo’s acquisition of Proofpoint for $12.3 B in 2021 and Broadcom’s $10.7 B purchase of Symantec in 2019. Google’s aggressive move marks a strategic milestone, reinforcing its commitment to cybersecurity after entering the space significantly in 2022 with its $5.4 B acquisition of Mandiant.

Strategic Rationale: Why Wiz and Why Now?

Understanding Google’s strategic rationale behind this deal requires recognizing the surging growth of enterprise cloud investments, coupled with a notable lag in cloud security spending. According to our recent Cloud Workload Security Quarterly Report covering the CNAPP (Cloud-Native Application Protection Platform) market, enterprise cloud spending skyrocketed from approximately $81 B in 2020 to an estimated $285 B in 2024, representing an impressive 5-year compounded annual growth rate (CAGR) of 29%. However, security investments have not kept pace, presenting a significant opportunity for vendors like Wiz that provide comprehensive cloud-native security solutions.

CNAPP, as defined in our Dell’Oro Group reports, is a unified platform that combines software security, deployment security, and runtime security technologies to secure the entire lifecycle of cloud-native applications. This platform approach fosters essential collaboration among development, security, and operations teams to protect applications and data throughout their deployment cycles effectively. Wiz, an innovative pure-play vendor, exemplifies this integrated approach, rapidly capturing market share with near triple-digit revenue growth rates.

Interestingly, Palo Alto Networks’ recent decision to reboot its CNAPP strategy, shifting from Prisma Cloud to Cortex Cloud, underscores the evolving competitive landscape. As detailed in my recent blog, Palo Alto Networks’ rebranding signals a necessary pivot toward deeper integration and a more cohesive security offering that directly addresses customer challenges around fragmented security management and operational complexity. Google’s acquisition of Wiz strategically positions it to avoid similar pitfalls, acquiring a purpose-built CNAPP solution with better cohesion from day one, potentially accelerating adoption among enterprise customers.

Valuation Concerns and Regulatory Risks

Yet, at $32 B, the valuation Google placed on Wiz raises critical questions about market dynamics and valuation metrics in cybersecurity. For context, Google’s purchase price slightly surpasses Zscaler’s current market capitalization of approximately $30 B, despite Zscaler having significantly higher annual revenue of $2.4 B. Furthermore, the price represents a substantial premium compared to cybersecurity giants like Palo Alto Networks (market cap: $126 B, revenue: $8.6 B) and Fortinet (market cap: $84 B, revenue: $6.0 B). Meanwhile, we estimate Wiz’s annual revenues were between $300 and $400 million in 2024. Although Wiz’s exceptional growth rate—94% year-over-year according to our Q2 2024 CNAPP report—partially justifies the valuation premium, it inevitably raises the question: Has Google overpaid?

Additionally, regulatory scrutiny in the technology sector has intensified, exemplified by the Department of Justice’s recent blockage of HPE’s $14 B acquisition of Juniper, despite approval by other global regulatory authorities. Google’s Wiz acquisition, at over twice the value of the blocked deal, is sure to attract rigorous antitrust examination, potentially complicating or delaying the transaction. Google’s willingness to navigate this regulatory environment underscores its confidence in the strategic necessity of securing a market-leading CNAPP platform to compete head-to-head not just against cloud service providers like Microsoft and Amazon Web Services but also standalone cybersecurity leaders.

Google aims to achieve recognition as a leading security vendor and replicate Microsoft’s success, which leveraged its dominant position in endpoint via Windows to build a $20 B annual cybersecurity business across endpoints and the cloud. Google believes that a similar leadership role can now be achieved in cybersecurity purely from a cloud perspective, marking a significant strategic pivot toward securing recurring revenue from cloud workloads.

Synergies and Market Opportunities

Despite valuation concerns and regulatory risks, Google’s aggressive move could be precisely what the company needs to solidify its cybersecurity portfolio and enhance the appeal of Google Cloud. Wiz’s impressive AI-driven security features will significantly bolster Google’s capabilities, enhancing its appeal to enterprises increasingly deploying AI workloads in multi-cloud environments. Moreover, the opportunity to leverage Google’s expansive cloud infrastructure and customer base promises substantial synergies that could rapidly accelerate Wiz’s revenue growth beyond current projections.

Industry observers and participants will closely monitor how this landmark deal influences competitive dynamics, growth trajectories, and customer perceptions in the CNAPP market. I plan to publish my next CNAPP market share report covering 2024 within the next month. It has been a tight race between Palo Alto Networks, CrowdStrike, and Wiz—stay tuned!

Ultimately, Google’s Wiz acquisition underscores a pivotal moment for cybersecurity valuations and strategic priorities, reflecting an industry evolving rapidly in response to enterprise needs for robust, integrated cloud security solutions. While the road ahead is challenging—given valuation expectations and regulatory hurdles—the strategic fit between Google and Wiz is compelling. If executed well, this deal could set a new benchmark for cloud-native security, ultimately benefiting enterprises worldwide by accelerating innovation and elevating the overall security posture in the digital economy.

It was an intense week in Barcelona. After 50+ meetings during and before the event, below are some initial key takeaways.

RAN outlook remains tepid
Open RAN marketing is morphing
Vendor concentration will likely increase
Near-term AI RAN driven by cost efficiencies and performance improvements

Somber RAN Outlook

The RAN forecast remains unchanged, but downside risks persist. One of our primary objectives was to assess whether the 0% CAGR RAN forecast issued in January 2025 still holds. Our preliminary analysis indicates that our long-standing message remains valid—regional imbalances will continue to impact the RAN market in the near term, while the underlying fundamentals shaping the long-term trajectory will continue to exert pressure on the market.

Since RAN spending is constrained by capex, and capex is tied to operators’ revenue growth, the entire wireless industry is urgently seeking new revenue streams to break the cycle of increasing data consumption without corresponding increases in revenue. While we encountered numerous discussions and demos centered on charging premiums for guaranteed or enhanced performance, service providers recognize the difference between monetizing “fun” content and business-critical applications. For example, Uber may be willing to pay extra at airports to ensure timely pings for its users. However, expectations remain low for consumers to pay extra for faster video uploads from congested areas or an improved gaming experience. While AI may drive the development of new applications and content, it is unlikely to fundamentally change consumers’ willingness to pay for “fun” content.

With limited justification for revising carrier topline growth expectations, the focus remains on mobile data traffic growth and performance differentiation. Video accounts for approximately three-fourths of total mobile traffic but still represents a small fraction of the total time users spend streaming on cellular networks. As mobile data traffic growth slows, the industry is increasingly looking for a new device that could shift user behavior and, ultimately, increase video consumption. While a future dominated by smart glasses—where data is continuously recorded and uploaded—would present significant network challenges, we have to spread out the probabilities of any new device for the masses gaining traction.

The general sentiment from the event is clear: the slowdown in data traffic growth, combined with ongoing struggles to monetize consumer connectivity, remains a significant challenge. In this post-peak 5G rollout environment, even flat RAN projections are seen as an optimistic.

Open RAN Losing Marketing Steam

Open RAN is happening (>67% of Ericsson’s 2025 deliveries will be Open RAN prepared), but its marketing power is fading. Incumbent RAN suppliers prefer the Cloud RAN term, while the smaller suppliers are starting to look past Open RAN. Whether this is because the commonly used HHI (Herfindahl Hirshman Index) market concentration gauge was similar in 2024 as in 2018 when the O-RAN Alliance was formed, the original Open RAN multi-vendor vision is morphing, the entire RAN equipment market is down around $9 B, RAN outlook is flat, or the smaller suppliers are tired of waiting for larger Tier1 multi-vendor projects, the outcome is the same – the meaning of Open RAN is changing and marketing departments are aware (multiple suppliers are now looking to shift the message/focus).

In our 5-year forecasts, we track and show Open RAN, vRAN, Cloud-RAN, and multi-vendor RAN. However, our 10-year outlook consolidates the tracking/terms and only shows Cloud RAN.

Vendor Concentration Expected to Increase

Open RAN and Cloud RAN are unlikely to alter the long-term RAN concentration trajectory. After improving between 2020 and 2022—partly due to Open RAN adoption and market share shifts among the top five suppliers—the RAN HHI index rebounded in 2024. While we do not forecast HHI, historical trends suggest that market concentration is on the rise.

Although history is not always the best predictor of future outcomes, several factors indicate that a highly concentrated RAN market by 2030 is a strong possibility. These include recent RAN market developments, the scale required to sustain a competitive RAN portfolio, the ratio of greenfield to brownfield deployments (including FWA, enterprise 5G, and MBB), the challenges faced by smaller suppliers, and ongoing discussions about potential M&A activity.

AI RAN — Performance and Efficiency Gains in the Driver Seat

The use of intelligence in the RAN is not new—both 4G and 5G deployments rely heavily on automation and intelligence to replace manual tasks, manage increasing complexity, enhance performance, and control costs. What is new, however, is the rapid proliferation of AI in both consumer and enterprise domains, along with a shifting mindset toward leveraging AI in cellular networks. More importantly, the scope of AI’s role is expanding, with operators now looking beyond efficiency gains and performance improvements, cautiously exploring whether AI could also unlock new revenue streams.

Given the growing interest in AI RAN, it is no surprise that definitions and interpretations of AI vary across the industry. As the ecosystem gains a deeper understanding of AI’s value in RAN, definitions and expectations will likely continue to evolve.

Currently, the industry’s broader perspective aligns with the AI RAN vision outlined by the AI-RAN Alliance. At a high level, AI is expected to add value in three key areas: asset utilization, application growth, and RAN efficiency improvements. From an operator’s standpoint, AI offers the potential to either boost revenue or reduce capex and opex.

One of the observations in Barcelona was that near-term AI activity is primarily focused on cost savings and efficiency rather than topline growth. For example, China Mobile reported a 30% reduction in MTTR using AI-based O&M, Verizon shared field data indicating a 15% cost savings with Samsung’s AI-powered energy savings manager (AI-ESM), and an Ericsson AI-RAN demo at MWC demonstrated a 20% increase in throughput using AI to optimize performance in non-ideal radio conditions. Similarly, T-Mobile is evaluating how its collaboration with Nokia on AI-RAN can enhance network performance and efficiency.

With revenue growth stagnating, operators are exploring new revenue streams, showing interest in NVIDIA’s latest edge computing initiatives. However, they are also keenly aware of power, energy, and cost constraints at the cell site. The macro-RAN market, valued at approximately $30 billion, supports 1 to 2 million base stations annually, leaving little flexibility in DU pricing. For vRAN to compete with purpose-built RAN, server and acceleration costs must decrease rather than increase. While a GPU-driven RAN-only business model currently has limited viability, the potential for multi-purpose RAN supporting both RAN and non-RAN workloads presents a larger TAM.

That said, our overall impression is that the AI service provider (AI SP) vision—where carriers sell unused AI capacity at scale—remains somewhat farfetched for now. However, as costs and energy consumption decrease, the concept could have more potential in the future.

In short, it was another eventful show in Barcelona with a reasonable balance between hype and reality, perhaps because the RAN market is down nearly $9 B and the outlook remains tepid. Still, the event was also a reminder that there is a lot of innovation and activity underneath the flat topline trajectory. We did not cover all the MWC topics in this blog, but we will likely share more updates in the future.

Contact Us