JMMMU-Pro Leaderboard

🌐 Homepage | 🤗 Dataset | 🏆 HF Leaderboard | 📖 arXiv | 💻 GitHub

"Which LMM is expert in Japanese subjects?" 🏆 Welcome to the leaderboard of JMMMU-Pro

We introduce JMMMU-Pro (Japanese MMMU-Pro), Image-based Japanese Multi-discipline Multimodal Understanding Benchmark.
Following the evolution from MMMU to MMMU-Pro, JMMMU-Pro extends JMMMU by composing the question image and question text into a single image, thereby creating a benchmark that requires integrated visual–textual understanding through visual perception.

model_name	prompt	Overall
Gemini3Pro	Direct	87.045
GPT-5.2	Direct	83.333
Qwen3-VL-8B	CoT	47.273
Qwen3-VL-8B	Direct	45.833
Qwen2.5-VL-7B	CoT	45
Qwen2.5-VL-7B	Direct	44.697
Sarashina2.2-V-3B	CoT	42.879
Sarashina2.2-V-3B	Direct	38.03
LLaVA-OV-1.5-8B	CoT	31.97
Phi-4-multimodal	Direct	31.818
InternVL2.5-8B	CoT	31.212
Sarashina2-V-14B	Direct	30.682
Sarashina2-V-14B	CoT	30
LLaVA-OV-1.5-8B	Direct	29.924
Sarashina2-V-8B	Direct	27.879
LLaVA-OV-7B	Direct	27.348
Sarashina2-V-8B	CoT	27.273
Heron-NVILA-Lite-15B	Direct	26.97
Aya-Vision-8B	CoT	26.742
InternVL2.5-8B	Direct	25.076
Phi-4-multimodal	CoT	24.167
Pangea-7B	CoT	23.409
Aya-Vision-8B	Direct	22.424
Pangea-7B	Direct	19.545
LLaVA-OV-7B	CoT	14.091
Heron-NVILA-Lite-15B	CoT	5.303