StepFun releases Step 5 Preview as open-weight 600B mixture-of-experts model
The sparse model activates 27 billion of its 600 billion parameters per token and supports context windows up to 1 million tokens, with checkpoints posted to Hugging Face and ModelScope.
- Step 5 Preview has 600 billion total parameters and 27 billion active per token.
- The model supports context windows of up to 1 million tokens.
- FP8 and BF16 checkpoints are live on Hugging Face and ModelScope.