The US is seeing an unprecedented increase in data center capacity to keep up with the growth of AI. According to the Department of Energy, US data center energy use will grow from 60 TWh in 2016 to as much as ~580 TWh in 2028, primarily because of the addition of GPUs and other AI computing chips [1]. A separate report from McKinsey puts the projected demand of AI data centers at 120 GW, against a realistic buildout of 65–90 GW [2], suggesting a serious shortfall of 30–55 GW by 2030. In lay terms: we need a lot of compute, and we’re not building it fast enough. To close this shortfall in the near term, we need faster software. We can’t afford to rely only on building more data centers; we have to make the most of the infrastructure we already have.

A Department of Energy (DoE) report [1] projects US data center energy use rising from 60 TWh to as much as 580 TWh by 2028, primarily because of the addition of GPUs and other AI computing chips.

This is the setting in which the National Science Foundation (NSF) launched a $21M challenge to address AI efficiency head-on and show material impact within two years [3]. The NSF describes the challenge this way:

The challenge invites proposals that accelerate the commercial adoption of translation-ready solutions to dramatically improve the efficiency of at-scale AI/Machine Learning systems and data centers. The initiative aims to enhance capacity-building and competitiveness of U.S.-based AI companies by reducing the cost of training and inference, addressing near-term limitations on data center capacity, and accelerating the time to market of new models.

Most of this new compute capacity is going toward inference, driven by test-time scaling laws, agentic AI, multi-agent frameworks, and reinforcement learning (RL) training, where the primary workload is RL rollouts that are dominated by inference calls. That is why this solicitation is such a good fit for our work at Makora, where we are optimizing the world’s inference, autonomously, on all hardware.

Makora’s pitch to the NSF centered on our vision for AI performance optimization: automated, continuous, and multi-hardware. We set concrete goals, backed by a strong track record of optimizing performance across hardware platforms, including NVIDIA and AMD GPUs, Google TPUs [blog], and AWS Trainium [blog]. Below are a few of our performance optimization proof points.

Figure 2 · Option A: original collage

As benchmarked by Artificial Analysis, Makora consistently delivers the fastest performance (in output tokens per second) as soon as a new AI model is released, including DeepSeek, Kimi, Qwen, MiniMax, MiMo, and Gemma models.

Makora’s automated optimization stack achieved a 5× performance improvement over vLLM’s TPU fork on TPU v7 for Qwen-3.5 397B, approaching or outperforming NVIDIA B200 GPUs depending on batch size.

Makora’s automated optimization stack deployed several Qwen-3.5 variants on AWS Trainium-2 chips, significantly improving performance over publicly available baselines.

We also signed up several “catcher” organizations that are willing to take our technology and deploy it on their infrastructure to improve efficiency. Our catchers include hyperscalers, cloud companies, and neoclouds that we are already discussing commercial engagements with. They gave us their vote of confidence by sending letters of intent to the NSF describing the opportunity to improve their data center efficiency, how our technology can be integrated into their infrastructure, and why Makora is the right platform.

The project kicks off now. We are excited to partner with the NSF and with our catcher organizations. By the end of 2027, Makora will have materially contributed to US data center efficiency, both by growing our MakoraInference token service tenfold and by deploying our software directly with our catchers on different AI hardware platforms.

Read the official NSF press release here.

[1] Shehabi et al. 2024 United States Data Center Energy Usage Report, Lawrence Berkeley National Laboratory. escholarship.org/content/qt32d6m0d1/qt32d6m0d1.pdf

[2] Tai et al. Powering AI: How real is the risk of overbuilding? McKinsey & Company. mckinsey.com/…/powering-ai-how-real-is-the-risk-of-overbuilding

[3] NSF-supported STRIDE Ventures launches AI Efficiency Challenge to invest up to $21 million over two years to accelerate deployment-ready innovations to strengthen U.S. AI infrastructure, NSF. nsf.gov/tip/updates/nsf-supported-stride-ventures-launches-ai-efficiency