ERNIE 5.5 Flash Base just dropped! Time to open that project folder. 🚀

Our efficient sparse MoE language model draws on data spanning multiple domains and languages, achieving SOTA on multiple benchmarks among similarly sized base models. Its architecture and training system were designed together for efficient training and inference, with advances in attention mechanisms, expert routing, and training objectives:
- 280B total parameters; 14B activated per token
- Pre-trained on 33.5T tokens
- Up to 1M-token context for fine-tuning and inference
- FP8 and BF16 weights

The weights come straight from pre-training, with no mid- or post-training. Compare methods from the same checkpoint, measure what each adds, and shape the model around your own tasks.