DigiNews

Summary

Baidu released ERNIE 5.5 Flash Base, a 280B-parameter sparse Mixture-of-Experts language model with 14B active per token, pre-trained on 33.5T tokens and capable of long contexts up to 1M tokens for fine-tuning and inference. The post details the architecture (sparse attention, Latent MoE, mHC, MultiMax, ECHO) and training data, and notes availability of weights on HuggingFace for community use and domain adaptation research.