OpenAI Launches Jev Competitor Amid Intensifying Decision Model Competition

OpenAI has announced the wide availability of its Decisions API model, marking a significant entry into the fast-growing sector of affordable, efficient intelligence. This move comes shortly after the impressive valuation of TypeSafe, a startup founded by a former researcher from OpenAI, which recently reached $7.6 billion.

The Jev model from TypeSafe has pioneered a new category of deep learning models, referred to as “super classifiers,” which can swiftly make choices from a set of options while also providing a confidence rating. Developers have quickly adopted these decision models, or System One models, as a cost-effective alternative to large language models (LLMs) in various workflows.

According to Nikunj Handa, the product lead for OpenAI’s APIs, the team was inspired by the capabilities of Jev. Handa mentioned, “We have like a million use cases for this,” prompting OpenAI to configure a version of ChatGPT Luna that optimizes speed and reduces costs.

“The major improvement with the Decisions API is the reduction in processing time, cutting it down from one to three seconds to about 100 to 180 milliseconds,” Handa explained. He further noted that while specific training may enhance the models in the future, the current implementation of Luna efficiently meets user demands.

The Decisions API was unveiled during OpenAI’s developer day on September 29 and became publicly available shortly thereafter. The updated version includes new voice and visual modalities, alongside efforts to lower latency by optimizing inference resources globally.

Initial trials with clients such as Box, Datadog, Ramp, and Speak have shown that the API significantly enhances throughput, allowing Speak to reduce action times from 1.4 seconds to just 0.13 seconds. OpenAI also utilized this product to bolster the responsiveness of their Dots voice assistant.

Despite the advancements, there remains ambiguity regarding the defining traits of a successful decision model, which seems to vary based on different workloads. Diogo Almeida, CEO of TypeSafe, suggests that direct trials may be more effective than relying on benchmarks, although TypeSafe has released its own rankings with the latest Jev model enhancements.

Public benchmarks such as the AIM System One Index and the Artificial Analysis Decision Index currently favor D1 Ultra, a recently launched decision model by Liquid, and Base Lab’s Elephant-1. The Jev model remains competitive at seventh and third positions on these indices, while OpenAI’s Decision Model ranks eighth and fourteenth, respectively. Notably, TypeSafe’s products are currently the most sought-after on OpenRouter.

A particularly engaging benchmark is the JevMan benchmark, where decision models simulate playing PacMan. The latest Jev model leads this ranking, indicating its effectiveness in making rapid, high-stakes decisions. Felix Wunderlich, the chief growth officer at Opper AI (the creators of JevMan), remarked on the benchmark’s unique challenge of demanding swift decision-making.

OpenAI’s model may have the upper hand in multimodal capabilities, particularly in image classification, as highlighted by Handa: “Luna has demonstrated exceptional calibration.” Users have expressed interest in having more versatility with image outputs through the Decisions API.

Handa also envisions further enhancements in speed that could lead to innovative use cases, speculating about achieving response times around 60 milliseconds, which could revolutionize API calls for each frame in video applications.

As performance continues to improve and costs decrease, the industry might see a transformation in how decision models are utilized, reflecting on the concepts introduced by the economist William Stanley Jevons, who elucidated how efficient resource use can lead to increased ubiquity. Handa believes this expansion will unlock “a crazy set of use cases” previously overlooked.