Beyond Transformers: Manifest AI‘s Brumby-14B and the Dawn of Power Retention
For nearly a decade, the transformer architecture has reigned supreme in the world of artificial intelligence, powering breakthroughs in natural language processing, image generation, and beyond. But a new contender has emerged, challenging the transformer’s dominance with a radically different approach. Manifest AI, a research organization focused on building truly bright systems, recently released Brumby-14B-Base, a 14 billion parameter model built on their innovative “Power Retention” mechanism. This release isn’t just another model launch; it’s a potential inflection point, signaling a shift in how we think about building and scaling large language models (LLMs).
As someone who’s been deeply involved in the evolution of AI architecture for over fifteen years – from the early days of recurrent neural networks to the transformer revolution and now, potentially, beyond – I’ve been closely following Manifest AI’s work. The implications of Power Retention are significant, and warrant a detailed examination. This article will delve into the technical details, the broader vision driving Manifest AI, the initial industry reaction, and what this all means for the future of AI.
The Core Innovation: Power Retention – A New Approach to Long-Context Modeling
The Achilles’ heel of transformers has always been their quadratic scaling with context length. as you feed a transformer more data (longer prompts, larger documents), the computational cost explodes, making long-context understanding prohibitively expensive.This limitation hinders applications requiring reasoning over extensive data,like complex document analysis,detailed code understanding,or truly engaging long-form conversations.
Manifest AI’s Power Retention tackles this problem head-on. Instead of relying on the attention mechanism - the core of the transformer – which compares every token to every other token, Power Retention maintains a persistent “memory” of past activations. Think of it like a continuously updated sketch of the conversation or document, rather than constantly re-evaluating everything from scratch.
“We’re trying to model the intelligent processes that generate output, not just the artifacts of intelligence,” explains Jacob Buckman, founder of Manifest AI. This philosophical shift is crucial. Conventional models focus on mimicking patterns in data; Power Retention aims to emulate how information is maintained and processed over time, mirroring aspects of human cognition.
Technical Underpinnings and Practical Implementation
The engineering behind Brumby-14B is equally compelling. The core Brumby kernels are written in Triton, a programming language designed for high-performance GPU computing, ensuring compatibility with both NVIDIA and AMD hardware. This broad compatibility is a significant advantage, fostering wider adoption and accessibility.
Manifest AI also leverages its in-house Vidrial framework for specialized CUDA kernels, further optimizing performance. While integration with popular inference engines like vLLM is still underway (“We have not yet integrated Power Retention into inference engines, but doing so is a major ongoing initiative at manifest,” Buckman notes), the team is actively working to streamline deployment.
Importantly, Buckman dismisses concerns about instability frequently enough associated with recurrent architectures. “We have not found this difficulty to be exacerbated in any way by our recurrent-state architecture. In fact, context-parallel training and GPU partitioning for multi-user inference both become substantially cleaner technically when using our approach.” This suggests a robust and scalable implementation.
The $4,000 Foundation Model: Challenging the Status Quo
The launch of Brumby-14B was accompanied by a bold claim: a “$4,000 foundation model.” This instantly sparked debate within the AI community. Meta researcher ariel (@redtachyon) argued the claim was misleading, pointing out that the model was built by fine-tuning the existing Qwen model, rather than training from scratch.
Buckman acknowledged the criticism, explaining that the $4,000 figure represented the cost of retraining the model, and the initial tweet was part of a larger thread detailing the methodology. “It’s not like I was being deceptive about it,” he clarified.
While the specifics of the cost are debatable, the underlying message is clear: Power Retention significantly reduces the computational resources required to achieve competitive performance. This has profound implications for democratizing access to cutting-edge AI research. It suggests that smaller organizations and independent researchers can now experiment at a scale previously reserved for tech giants.
Performance and Potential: A Crack in the Transformer’s Wall?
The key takeaway from Brumby-14B isn’t just the cost savings, but the performance. Manifest AI
Keep reading