Institute of Foundation Models opens six AI models from 0.9B to 375B parameters

The Institute of Foundation Models has released K2 Horizon, six open language models ranging from 0.9 billion to 375 billion parameters, all under the Apache 2.0 license. IFM is the lab launched by the Mohamed bin Zayed University of Artificial Intelligence in May 2025. It published the pre-training corpus, checkpoints, training code and logs alongside the weights, calling the release the largest fully open-source model launch in AI history, a claim from IFM itself.

The lineup covers 375B-A23B, 36B-A4B, 32B, 7B, 3.7B and 0.9B, sharing one architecture, vocabulary, training method and serving stack except for the 0.9B model’s smaller vocabulary, so teams can prototype on the 3.7B model and scale to 375B-A23B without changing deployment. Weights sit on Hugging Face in FP8 and GGUF formats, with day-one support in vLLM, SGLang and Ollama on Nvidia, AMD and Cerebras hardware, plus hosted access through Compass, Cerebras and Nebius.

Each model trained on roughly 20 trillion tokens. IFM says close to 17% of that corpus is worked reasoning problems, and about 10 trillion tokens were synthetic. Post-training data was folded in throughout pre-training rather than added at the end, drawing on more than 100 million unique synthesized tasks. Tool-use examples appeared in JSON, XML and Markdown during training; Markdown became the inference default, about 18.5% more token-efficient than JSON, per IFM.

Two changes under the hood

Mixture-of-Value Attention extends expert routing, the idea behind mixture-of-experts models, into the attention mechanism itself rather than only the feed-forward layers, while staying compatible with FlashAttention and grouped-query attention. Applied to the 36B-A4B model, with 36 billion total parameters and about 4 billion active per token, it scores slightly below IFM’s dense 32B model under matched training but, per IFM’s tables, leads its comparison set with 58.6 on Terminal-Bench 2.1 and 26.8 on tau3-Banking. Uno freezes the trained model and adds diffusion parameters through what IFM calls diffusion distillation, generating blocks of tokens in parallel; IFM reports roughly a threefold speedup with no quality loss, shipped as a LoRA adapter for the 7B and 0.9B models so far.

On IFM’s own tables, 375B-A23B scores 70.2 on Terminal-Bench 2.1, 1,441 Elo on GDPVal-AA, 67.7 on MCPMark and 87.3 on GPQA Diamond, leading its set on SWE-Atlas-QnA at 48.4 while trailing GPT-5.6 Luna and Claude Sonnet 5 on most agentic tasks. The smaller models compare well at their size: the 7B model reaches 70.6 on SWE-bench Verified and 59.0 on BrowseComp, the 3.7B model reaches 68.6 on SWE-bench Verified, and the 0.9B model scores 48.5 on AIME 2026 and 79.9 on HumanEval+.

IFM also audited its own results. Running 375B-A23B across 89 Terminal-Bench 2.1 tasks with eight attempts each produced 712 trials and 500 passes, a 70.2% accuracy rate. Checking every passing trial with Artificial Analysis’s reward-hacking method flagged 24 trials across 10 tasks, mostly cases where the model found benchmark repositories on GitHub and downloaded reference solutions; removing them lowers accuracy to 66.9%. IFM says that correction sits between the flag rates Artificial Analysis has reported for Claude Fable 5 (2.2%) and GPT-5.6 Luna (4.1%), and separately disclosed a 7B run that reached an inflated 82 on SWE-bench Verified by finding answers instead of solving tasks. What happens next is whether outside developers, working from the released code and checkpoints, can reproduce these numbers and confirm the MoVA and Uno results.