The defining infrastructure story of 2024 has been the scramble for GPU capacity to train and serve AI models. Out of that scramble a new class of provider has emerged — specialized GPU clouds, often called “neoclouds,” that do one thing: rent out fleets of high-end accelerators at scale. Names like CoreWeave, Lambda, and Crusoe have gone from relative obscurity to commanding serious attention and capital, positioning themselves against the established hyperscalers by being narrower, faster, and more GPU-focused. The phenomenon is worth examining both for what it offers and for the risks it carries, because the neocloud model is a genuinely different proposition from the general-purpose cloud most organizations are used to.
What a Neocloud Is
A neocloud is a cloud provider specialized almost entirely around GPU compute for AI and high-performance workloads. Where a hyperscaler offers a sprawling catalog of hundreds of services — databases, queues, identity, networking, analytics, and much more — a neocloud offers a comparatively narrow menu centered on access to large fleets of current-generation GPUs, the high-speed interconnect needed to cluster them for distributed training, and the storage and networking to feed them.
The specialization is the point. By focusing narrowly, neoclouds aim to deliver GPU capacity faster, at larger contiguous scale, and often at lower cost than the general-purpose clouds — which have to balance GPU supply against every other demand on their capital and capacity. For an organization whose need is specifically “a large cluster of GPUs, available soon, for training or inference,” a neocloud can be a more direct and more available answer than a hyperscaler’s GPU instances, which through 2024 have frequently been capacity-constrained and subject to queues and quotas.
Why They Emerged Now
Several conditions converged to create the opening.
Acute GPU scarcity. The demand for high-end accelerators for AI training and inference has, through 2024, persistently outstripped supply. The most sought-after data-center GPUs were difficult to obtain, and the hyperscalers’ own GPU capacity was heavily rationed. That scarcity created room for providers who could secure GPU allocations and rent them out, sometimes to customers who could not get sufficient capacity from the major clouds.
The economics of specialization. Building and operating GPU clusters — with their specific demands for power, cooling, and high-speed networking — is a distinct discipline. A provider that does only this can optimize for it in ways a generalist cannot, from datacenter design to the software stack for cluster scheduling.
Capital availability. The intensity of investor interest in AI infrastructure has made capital available to providers positioned as GPU-cloud specialists, funding the large hardware purchases the model requires. There are signs through late 2024 that the most acute scarcity of the prior period is beginning to ease at the margins, and a next generation of accelerators has been announced, though it is not yet shipping in volume — which makes the timing and durability of today’s capacity bets an open question.
The Trade-Offs
The neocloud proposition is attractive for the right workload, but it comes with trade-offs that organizations should weigh deliberately rather than be swept past by the urgency of securing GPUs.
What you gain. Availability of GPU capacity when hyperscalers are constrained; often lower per-GPU cost; large contiguous clusters suited to distributed training; and a provider whose entire focus is the workload you are running.
What you give up. The breadth and maturity of the hyperscaler ecosystem. A general-purpose cloud surrounds its GPU instances with managed databases, identity, networking, security tooling, compliance certifications, and a decade-plus of operational hardening. A neocloud typically offers far less of this surrounding infrastructure. If your AI workload needs only raw GPU compute and you supply the rest, that is fine; if it is entangled with managed services, the gap matters.
Concentration and counterparty risk. This is the trade-off that deserves the most scrutiny. Neoclouds are younger, narrower, and more financially leveraged than the hyperscalers. Their business model depends on continued high demand for GPU rental and on the value of the hardware they have bought on credit. A shift in GPU economics — easing scarcity, a new hardware generation that devalues current inventory, or a softening of AI-infrastructure demand — could pressure providers whose finances assume the current conditions persist. An organization that builds critical workloads on a single neocloud is taking on counterparty risk of a kind that the established hyperscalers, with their diversified businesses, do not carry to the same degree.
A Familiar Structural Pattern
The neocloud phenomenon rhymes with patterns the distributed-systems and infrastructure community has seen before. Specialized providers emerging to serve a scarce, high-demand resource — and the question of how to use them without becoming dependent on any one of them — is a recurring shape. The prudent architectural response is also familiar: treat specialized providers as interchangeable suppliers of a commodity (GPU-hours) rather than as platforms to build deeply into, so that the workload can move if a provider’s economics or availability change. This is the same portability discipline that applies to the general cloud, with the counterparty-risk dimension heightened because the providers are younger and less diversified.
Keeping the workload portable across GPU clouds means leaning on architecture-agnostic abstractions — containerized training and inference, orchestration that can target multiple providers, data and checkpoint storage that is not locked to one provider’s proprietary services. The narrower service surface of a neocloud actually helps here: there are fewer proprietary services to get entangled with than on a hyperscaler, which makes the raw-compute relationship easier to keep at arm’s length.
Guidance for Organizations
- Match the provider to the workload. If your need is raw GPU compute and you can supply the surrounding infrastructure, a neocloud can be an excellent, available, cost-effective fit. If your AI workload is deeply coupled to managed services, a hyperscaler’s integrated offering may be worth its premium.
- Treat GPU capacity as a commodity to be sourced, not a platform to be locked into. Architect training and inference to be portable across providers. Avoid building critical, hard-to-move dependencies on a single neocloud’s proprietary surface.
- Assess counterparty risk explicitly. For any workload of consequence, weigh the financial durability of a younger, narrower provider. Consider multi-sourcing for critical capacity rather than depending on one provider.
- Watch the hardware cycle. GPU economics are in flux, with scarcity easing in places and a new hardware generation announced but not yet shipping in volume. The relative attractiveness of neoclouds versus hyperscalers will move with that cycle; revisit the decision as conditions change rather than treating today’s calculus as fixed.
The rise of the neocloud is a direct product of the GPU scarcity that has defined AI infrastructure through 2024. Whether the category proves to be a durable structural feature of the cloud landscape or a phenomenon of a particular supply-constrained moment is a question the next year or two will answer, as the hardware cycle turns and the current scarcity evolves. For now, neoclouds are a real and useful option for GPU-intensive work — best approached with the same portability discipline, and a sharper eye on counterparty risk, that prudent infrastructure planning applies to any single-supplier dependency.
Further Reading
- Cloud Native Computing Foundation — orchestration and portability projects relevant to keeping AI workloads movable across providers.
- Open Compute Project — open hardware and datacenter designs, including for the high-density GPU infrastructure that neoclouds operate.