Skip to content

Request official evaluation scripts/configs for reproducing MiniCPM5-2B benchmark results #376

Description

@haofeiwang225-boop

Hello MiniCPM Team,

I am trying to reproduce the benchmark results of MiniCPM5-2B for academic research.

I have successfully deployed the released MiniCPM5-2B model following the official instructions. However, I could not find the complete evaluation pipeline used to obtain the benchmark results reported in the model card.

Could you please share the official evaluation scripts and configurations, including:

  1. Benchmark evaluation scripts (MATH-500, MMLU/MMLU-Pro, AIME, HMMT, etc.);
  2. Inference settings:
    • prompt/chat template;
    • whether enable_thinking=True is used;
    • generation parameters (max_new_tokens, temperature, top_p, do_sample, stop criteria, etc.);
  3. Answer extraction and grading scripts;
  4. Dataset preprocessing and evaluation commands.

I especially want to reproduce the reported MATH-500 score (94.6), and I would like to confirm whether the official evaluation uses any reasoning budget, length control, or special stopping strategy for the thinking mode.

Any scripts or configuration files would be greatly appreciated.

Thank you very much.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions