XiaomiMiMo/MiMo: MiMo: Unlocking the Reasoning Potential of Language Model – From Pretraining to Posttraining

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Unlocking the Reasoning Potential of Language Model
From Pretraining to Posttraining

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

This code repository is licensed under the Apache2.0 License.

Currently, most successful RL works, including open-source research, rely on relatively large base models, e.g., 32B models, particularly for enhancing code reasoning capabilities. Moreover, it was widely considered that achieving uniform and simultaneous improvements in both mathematical and code capabilities within a small model is challenging. Nonetheless, we believe that the effectiveness of the RL trained reasoning model relies on the inherent reasoning potential of the base model. To fully unlock the reasoning potential of language models, efforts must focus not only on post-training but also on pre-training strategies tailored to reasoning.

In this work, we present MiMo-7B, a series of models trained from scratch and born for reasoning tasks. Our RL experiments from MiMo-7B-Base show that our model possesses extraordinary reasoning potential, even surpassing much larger 32B models. Additionally, we perform RL training on a cold-started SFT model, resulting in MiMo-7B-RL, which demonstrates superior performance on both mathematics and code reasoning tasks, matching the performance of OpenAI o1-mini.

We open-source MiMo-7B series, including checkpoints of the base model, SFT model, RL model trained from base model, and RL model trained from the SFT model.
We believe this report along with the models will provides valuable insights to develop powerful reasoning LLM that benefit the larger community.

Models are avaliable at https://huggingface.co/XiaomiMiMo

MiMo-7B series

1. [Recommended] We official support inference with MiMo-MTP using our fork of vLLM.

Example script

from vllm import LLM, SamplingParams

model_path = "/path/to/MiMo"
llm = LLM(
    model=model_path,
    trust_remote_code=True,
    num_speculative_tokens=1,
    disable_log_stats=False
)
sampling_params = SamplingParams(temperature=0.6)

conversation = [
    {
        "role": "system",
        "content": ""
    },
    {
        "role": "user",
        "content": "Write an essay about the importance of higher education.",
    },
]

outputs = llm.chat(conversation,
                   sampling_params=sampling_params,
                   use_tqdm=False)

for output in outputs:
    prompt = output.prompt
    generated_text = output.outputs[0].text
    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")

print("=" * 80)

2. Or, you can register a vLLM loader for MiMo without loading MTP parameters.

You can copy the registry/register_mimo_in_vllm.py to your directory and import it with

import register_mimo_in_vllm

from vllm import LLM, SamplingParams

model_path = "/path/to/MiMo"
llm = LLM(
    model=model_path,
    trust_remote_code=True,
    # num_speculative_tokens=1,
    disable_log_stats=False
)
sampling_params = SamplingParams(temperature=0.6)

Example script

from transformers import AutoModel, AutoModelForCausalLM, AutoTokenizer

model_path = "/path/to/MiMo"
model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_path)
inputs = tokenizer(["Today is"], return_tensors='pt')
output = model.generate(**inputs, max_new_tokens = 100)
print(tokenizer.decode(output.tolist()[0]))

• We recommend using our fork of vLLM which is developed based on vLLM 0.7.3.
• We recommend using empty system prompt.

We haven’t verified MiMo with other inference engines and welcome contributions based on the model definition in the Huggingface repo 💻.

@misc{xiaomi2025mimo,
      title={MiMo: Unlocking the Reasoning Potential of Language Model – From Pretraining to Posttraining}, 
      author={{Xiaomi LLM-Core Team}},
      year={2025},
      primaryClass={cs.CL},
      url={https://github.com/XiaomiMiMo/MiMo}, 
}

Please contact us at mimo@xiaomi.com or open an issue if you have any questions.