What did Baidu release with ERNIE 5.5 Flash Base? Open weights—but not a finished chatbot
Baidu released a 280-billion-parameter sparse base checkpoint trained on 33.5 trillion tokens, with 14 billion parameters active per token. The weights are downloadable, but they have no instruction tuning or safety post-training, require substantial hardware and come under Baidu's own model licence.
The short version
- What happened
- ERNIE-5.5-Flash-Base has 280 billion total parameters and activates 14 billion per token. Baidu says it was pre-trained from scratch on 33.5 trillion tokens with contexts growing to 128,000 tokens.
- Who it affects
- Researchers with substantial compute gain a common large base checkpoint for controlled adaptation experiments.
- What remains unproven
- Architecture, training and benchmark claims come from Baidu and have not been independently reproduced in this release.
Keep in mind
Architecture, training and benchmark claims come from Baidu and have not been independently reproduced in this release.
The checkpoint has no mid-training or post-training and should not be evaluated as if it were an instruction-following consumer assistant.
The 33.5-trillion-token corpus is described by broad categories rather than a complete data inventory with rights, language and removal denominators.
One-million-token support is a capacity claim, not evidence of perfect recall or reasoning across a million-token input.
The weights use Baidu's own model licence, and practical serving or adaptation requires substantial hardware and engineering.
Source analysis · 2 linked sources. Primary analysed source date: 5 November 2026.
Read the full report · 9 min The evidence, methods, implications and limitations
The answer: Baidu released a large pre-training checkpoint, not a ready-to-use assistant
Baidu has published downloadable FP8 and BF16 weights for ERNIE-5.5-Flash-Base, a text-only sparse mixture-of-experts model with 280 billion total parameters and 14 billion activated for each token. The company says the model was trained from scratch on 33.5 trillion tokens, with the training context gradually extended to 128,000 tokens. Its configuration supports contexts up to one million tokens for later fine-tuning and inference, although the deployment examples use 128,000 by default.
The word 'Base' carries most of the practical meaning. This checkpoint comes directly from the end of pre-training, before mid-training, instruction tuning, reinforcement learning or other post-training. It may continue text, but it has not been shaped to follow ordinary user instructions, refuse unsafe requests, cite sources or behave consistently in conversation. Anyone building a public or high-stakes service would need substantial additional training, evaluation and safeguards. The release is an engineering starting point, not evidence of a finished product.
Calling the weights open also needs qualification. They are available through Hugging Face and ModelScope, with technical deployment instructions, but the model uses Baidu's ERNIE 5.5 Flash Base Model License Agreement v1.0 rather than a standard permissive software licence. Prospective users must inspect those terms for their intended use. Weight access does not establish that the training data are open, that all data rights are independently audited or that the model can be run cheaply on ordinary hardware.[1][2]
How the architecture reduces active compute while keeping a very large parameter pool
ERNIE 5.5 Flash has 43 transformer layers, a hidden size of 4,096 and 64 attention heads. All but the first layer use mixture-of-experts feed-forward blocks. Each has 512 routed experts plus one shared expert, but only the top ten routed experts activate for a token. That routing is why the model can hold 280 billion parameters while using about 14 billion per token. Sparse activation reduces computation compared with using every parameter, but the full weights still need storage and the serving system must move tokens among experts efficiently.
Baidu replaces a single expert router with several independently initialised sigmoid routers. The company says this Multi-Router design better aligns selection scores with expert utility while keeping activated experts, expert compute and communication volume unchanged. Experts operate in a 2,048-dimensional latent space—half the model's hidden width—before one projection returns the result to 4,096 dimensions. These choices aim to reduce dispatch traffic and expert compute, not to make the model small in an absolute sense.
The model also uses four parallel residual streams through manifold-constrained hyper-connections, a piecewise-temperature output head called MultiMax and an ECHO mechanism that reuses the final backbone layer to predict future tokens. Each component is presented as an efficiency or optimisation improvement. Their value is supported mainly by Baidu's own ablations and pre-training comparisons; the release does not provide an independent reproduction of the full 33.5-trillion-token run.[1][2]
The distinction that matters
What 'open base weights' include—and what builders must still add
The release exposes a pre-training checkpoint and serving path, while leaving product behaviour and assurance to downstream teams.
Released
Download FP8 or BF16 weights
End-of-pre-training checkpoint
Architecture
Activate 14B of 280B parameters per token
Sparse compute, large storage footprint
Not included
No instruction or safety post-training
Not a finished chatbot
Builder duty
Fine-tune, evaluate and govern
Capability and risk depend on the derivative
Long context is handled by compressing most layers' view of the past
Only six of the 43 layers retain token-level access to the full context through multi-head latent attention with sparse selection. The other 37 use heavily compressed attention: every 128 previous tokens become one cached entry, while a short recent window remains uncompressed. This greatly shrinks the key-value cache. It also means that most layers see distant text through a compressed representation rather than every original token. The six full-context layers are intended to recover details when needed.
Baidu reports that the six sparse full-context layers were enough to keep performance competitive on six long-context benchmarks. However, one of those comparisons, HELMET-Rerank, was led by another base model, and aggregate benchmark success does not guarantee reliable retrieval in a particular million-token document. The supported context length is a maximum input capacity, not a promise that the model will remember every fact, reason consistently across the entire window or resist adversarial distractors.
For organisations, the relevant question is not simply whether one million tokens fit. They should test the actual documents, languages and retrieval tasks they plan to use, measure accuracy as relevant information moves farther back, and record latency and memory cost. A compressed-attention design may offer a favourable trade-off, but the trade-off should be measured against a dense or retrieval-augmented baseline rather than inferred from the context number alone.[1][2]
What is known—and not known—about the 33.5-trillion-token training corpus
The technical page describes a multilingual corpus spanning web data, code and long-context material. Web content was extracted, filtered and scored for quality. Code sources included repositories and web pages as well as commits, issues, pull requests, reviews, generated tests and continuous-integration-verified fixes. Long-context data combined natural documents, retrieved distractors and synthetic examples for retrieval, citation-grounded question answering and multi-hop reasoning.
Those categories help explain the intended capabilities, but they are not a dataset inventory. The release does not enumerate all domains, languages, licences, removals, opt-out procedures or proportions of synthetic and human-created data. A total token count is therefore not a denominator for copyright compliance, cultural representation, privacy risk or factual quality. Developers inheriting the base checkpoint also inherit uncertainty about what the model may memorise or reproduce.
The model card reports FP8 mixed-precision training with a distributed Muon optimiser across thousands of GPUs, but it does not disclose total energy use, hardware hours or carbon and water impacts. Sparse inference can lower active compute compared with a dense 280-billion-parameter model, yet training 33.5 trillion tokens remains a major industrial undertaking. An efficiency claim should be read relative to the stated comparison, not as evidence that the overall environmental or financial cost is small.[1]
How to read Baidu's benchmark table
Baidu compares ERNIE 5.5 Flash Base with three similarly sized base checkpoints: DeepSeek V4 Flash Base, Hy3 preview Base and MiMo V2.5 Base. The company says all four were run through the same evaluation harness, prompts, few-shot examples and free-form-answer judge. ERNIE leads many listed exam, code and retrieval rows—for example, 93.3 on five-shot MMLU and 82.3 pass-at-one on HumanEval+—but it does not lead every measure. ChineseSimpleQA, BBH with chain-of-thought and HELMET-Rerank illustrate rows where another model scores higher.
A same-harness comparison is more informative than copying unrelated numbers from separate model cards. It is still a first-party evaluation by the model's developer. The release does not report independent red-team testing, contamination analysis for every benchmark, statistical uncertainty, human preference evaluation or real deployment outcomes. Some free-form benchmarks use an LLM judge, adding another model's decisions to the measurement. Small score differences should not be treated as settled superiority without repeated external testing.
Most importantly, these are base-model scores. A later post-trained ERNIE assistant could behave very differently, and organisations fine-tuning the checkpoint may improve one domain while degrading another. The release creates a common baseline that researchers can build on and audit. It does not establish the safety, reliability or cost of those future derivatives.[1][2]
What the release means for developers and what would change the assessment
For well-resourced research teams, the release provides a rare large Chinese-English base checkpoint with substantial architectural disclosure and code paths for SGLang and vLLM. It may reduce the need to repeat pre-training when studying instruction tuning, domain adaptation or agent training. Smaller teams face a different reality: the official examples assume multi-GPU distributed serving, the full checkpoint is enormous and further training adds data, compute and safety work. Downloadable does not mean accessible at equal cost.
Independent evaluation should test raw base-model loss and downstream adaptation under matched compute, verify long-context retrieval at multiple distances and languages, and inspect security, memorisation and misuse risks after post-training. Researchers should also compare the licence with their intended commercial or public-sector use and document every additional dataset. A public deployment needs instruction following, abuse testing, monitoring and incident response beyond anything demonstrated here.
The assessment would strengthen if external groups reproduce the efficiency and benchmark advantages, Baidu publishes fuller training-data and environmental disclosures, and downstream builders show safe, useful performance in real tasks. It would weaken if the million-token claim fails on realistic retrieval, licensing limits block expected reuse, or independent tests reveal contamination or unstable behaviour. Today the defensible conclusion is narrower: Baidu has released a technically ambitious, genuinely downloadable base checkpoint, but the evidence does not support calling it a finished, independently validated or low-cost AI service.[1][2]
Keep following the evidence
Join The Impact Brief founding list for selected AI reports, their effect on people and the evidence limits. Read a sample first.
Continue the story
Read next
Continue exploring foundation models: compare the finding and its evidence limits.
Does structure-guided AI improve RNA prediction? An open 21-million-pair model led several benchmarks—but biology still needs experiments
Nature's structRFM pairs RNA sequences with predicted secondary structures during pre-training and releases its data, weights and code. It improved several secondary-, tertiary- and function-prediction benchmarks, but the structural labels are computational, long RNAs remain difficult and benchmark accuracy is not a therapeutic result.
Read the next reportExplore the full source and evidence record
What this means for people
- Researchers with substantial compute gain a common large base checkpoint for controlled adaptation experiments.
- Smaller organisations may still be excluded by storage, multi-GPU serving and further-training costs despite public weight access.
- People affected by downstream systems depend on each builder's data choices, safety work and monitoring; Baidu's base release does not validate those future uses.
Global context
The release adds a major Chinese-developed base model to a field increasingly shaped by downloadable weights from several regions. That can broaden research choices and reduce dependence on closed APIs, but access remains unequal because large-model serving and further training require expensive infrastructure. Language coverage, local regulation, licence terms and data governance will determine whether the checkpoint is useful in a given country or institution.
What the evidence does not yet show
- Architecture, training and benchmark claims come from Baidu and have not been independently reproduced in this release.
- The checkpoint has no mid-training or post-training and should not be evaluated as if it were an instruction-following consumer assistant.
- The 33.5-trillion-token corpus is described by broad categories rather than a complete data inventory with rights, language and removal denominators.
- One-million-token support is a capacity claim, not evidence of perfect recall or reasoning across a million-token input.
- The weights use Baidu's own model licence, and practical serving or adaptation requires substantial hardware and engineering.
What to watch next
- Independent benchmark and long-context evaluations under matched compute and prompts.
- Publication of a post-trained ERNIE 5.5 Flash and clear separation from this base checkpoint.
- External audits of training-data provenance, memorisation, security and misuse risks.
- Real hardware, latency and energy measurements for 128,000- and one-million-token serving.
- How the custom licence affects commercial, academic and public-sector derivatives.
Living evidence record
Impact record IAI-04Q908G
Evidence stage
Announced
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Not yet
Record status
Monitoring
Last checked
6 November 2026
Source trail
2 direct sources across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 6 November 2026
Research question, topics and regions
Topics: Open weights · Foundation models · Model efficiency · Long context · Model licensing · Infrastructure
Regions: China · Asia & Middle East · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.