PrismML’s Compact LLM Aims to Transform AI Utilization for Everyone

PrismML, an AI startup that’s not yet widely recognized, is making waves in the tech community—not for its funding, which stands at a modest $22.25 million in seed investment, but for the innovative technologies it’s developing. The company is challenging the convention that high-performing large language models (LLMs) must be large and cumbersome.

Instead, PrismML aims to create compact reasoning models that can operate efficiently on personal computers and smartphones. Although there are rumors about potential collaborations with Apple, CEO Babak Hassibi has declined to confirm this.

Recently, PrismML unveiled Bonsai 2 27B, a new addition to its model family. This latest iteration compresses Alibaba’s widely used Qwen 3.8 27B model down to a mere 5.9 GB, allowing it to be stored on most PCs and potentially high-end smartphones. This represents an impressive reduction of 9 to 10 times the original memory requirement.

Founded by a collective of researchers from Caltech, PrismML is led by Hassibi, a professor renowned for his expertise in compression technologies. The advisory board includes Ion Stoica, co-founder of Databricks and director of the Sky Computing Lab at Berkeley, which has produced numerous impactful startups.

Investors such as Khosla Ventures and Cerberus Capital back the company, indicating strong interest in its vision and technology.

While PrismML is not alone in the LLM compression space—Multiverse Computing is also working on similar technologies—it distinguishes itself by achieving high compression rates without significant performance loss. According to Hassibi, Bonsai 2 achieves 98% of Qwen’s aggregate benchmark scores, up from 95% for its predecessor, which has already amassed over 11 million downloads. Their even smaller models have reportedly been downloaded an additional 2.6 million times.

Although achieving 100% benchmark performance may not be feasible, the impact of a slight degradation might not significantly hinder practical applications, given the inherent limitations in LLM accuracy and benchmarks.

PrismML attributes its success to a novel method of reducing the “weights” that construct a model. Traditionally, these weights require 16 bits each, but PrismML uses a “ternary” system, simplifying them to just three values: +1, -1, or 0. This innovative reduction drastically decreases the space needed for storage.

The startup’s next goal is to extend this compression technique to even larger models, with hopes of releasing models containing several hundred billion parameters in the coming months. Hassibi believes that larger models offer more opportunities for effective compression without sacrificing intelligence.

Stoica expressed enthusiasm for this technology, highlighting its potential to enable advanced AI capabilities directly on users’ devices. He emphasized that this would not only make powerful tools more accessible but also enhance privacy, as computations would not need to rely on cloud infrastructure.