AI Infra Summit 2026: Lightbits Rewrites Tokenomics with Inferra, an Intelligent KV Cache Orchestration Engine

Tokenomics has a new name: Inferra by Lightbits – an intelligent KV cache orchestration engine that dramatically improves GPU utilization, enabling long-context and multi-session AI workloads – makes its public debut at the AI Infra Summit on September 15th.Built by Lightbits Labs, inventors of the NVMe over TCP storage protocol, Inferra tackles one of AI inference’s most pertinent challenges: the memory bottlenecks created by exploding KV caches with long context lengths in LLMs — enabling neoclouds and enterprise organizations to run Inference faster, scale bigger, and spend less money on AI infrastructure.

“The AI inference market will top $117 billion this year, forcing legacy vendors designed for model training and ingest to scramble and re-architect,” said Avigdor Willenz, co-founder and chairman, Lightbits Labs. “Inference is a completely different paradigm – retrofitting legacy training and storage systems to solve the KV cache bottleneck simply doesn’t work. It was built for a different era. We engineered Inferra from the ground up specifically to solve the GPU efficiency problem, and it has already proven successful in customer beta programs. Disrupting the status quo with tech that raises the bar for data infrastructure performance and efficiency is in our DNA. Applying those same principles to Inference that we applied when inventing NVMe over TCP makes Inferra uniquely positioned to deliver significant value for this market.”

Inferra maximizes hardware utilization, eliminating idle GPU cycles and redundant context recomputation, enabling organizations to run multi-session clusters with strict SLA guarantees. By virtualizing GPU memory across memory and storage tiers and transforming the KV cache into an intelligent, persistent data layer, Inferra fundamentally changes the economics of inference.

“Our collaboration with Lightbits has demonstrated a clear path to maximizing AI infrastructure efficiency,” said Yaniv Fdida, CPTO, OVHcloud. “With Inferra’s intelligent KV cache tiering, we were able to demonstrate substantial GPU utilization gains, paving the way to providing our customers a more scalable and cost-effective foundation for their AI agent and RAG workloads.” 

Key AI inference workload breakthroughs on display:  

• Infinite HBM Virtualization & 16x Session Density: Extends and shares the KV cache beyond limited GPU memory, enabling organizations to run multi-session clusters with strict SLA guarantees. Serve up to 16x more concurrent inference sessions on existing GPU infrastructure – without costly hardware upgrades
• >100x Latency Improvement & 10M-Token Context Windows: Drastically lowers Time-to-First-Token (TTFT) and Time Per Output Token (TPOT) by proactively and predictively pre-fetching attention states from storage rather than recomputing them, enabling next-gen AI assistants and multi-session agents that require context windows extending well beyond GPU memory limitations
• AI-Native Security and Isolation: Delivers consistent SLAs with secure tenant isolation and intelligent KV cache management across shared inference infrastructure

AI infrastructure teams, hyperscalers, and GPU cloud providers are invited to visit booth 219 at the AI Infra Summit for live demonstrations and to learn how Inferra delivers superior token economics and unbounded LLM scale. 

“Scaling long-context inference workloads requires rethinking how data moves between the GPU, high-performance memory and storage. Results from testing Lightbits’ Inferra engine in the Solidigm AI Central Lab, leveraging Solidigm’s D7-PS1010 enterprise NVMe SSDs, showcased how network attached storage with intelligent software layer virtualization can break through the memory wall for large context, agentic AI. This collaboration empowers hyperscalers and cloud providers to achieve superior tokenomics while running multi-session workloads with uncompromising SLAs,” said Avi Shetty, VP, ecosystem, solutions & market enablement, Solidigm.

“We’re seeing strong momentum as enterprises and AI infrastructure providers move beyond model training and focus on the economics of inference. Organizations need technologies that maximize utilization of large-scale AI inference environments. We see Inferra as an innovative solution that provides inference and token efficiency. We look forward to helping customers evaluate its potential,” added Alexey Stolyar, CTO, ICC.

“We’re reinventing inference by shipping the world’s first KV cache acceleration engine that eliminates idle GPUs and power context windows of up to 10M tokens on commodity hardware. By delivering Smart Infinite HBM Virtualization across memory tiers, Inferra enables significant latency improvements with no hardware upgrades. Inferra delivers instant payback and generates net positive savings from day one. It eliminates the stalls that traditionally limit long-context performance, so Neoclouds and enterprises can serve larger models and longer conversations at lower infrastructure cost,” said Ramesh Chettuvetty, SVP, AI product and business, Lightbits Labs.

Rewrite your own tokenomics! See the difference Inferra can make to your top line and profit margins; try the Pod Efficiency Analyzer today.