Gemini 2.5 Pro Experimental (03-25)

条件：Diff setting; AI Studio API model ID gemini-2.5-pro-exp-03-25; default sampling; evaluation date not separately published.

证据：Google's official Gemini 2.5 Pro Experimental (03-25) chart reports: Humanity's Last Exam no tools 18.8%; GPQA diamond single attempt 84.0%; AIME 2025 single attempt 86.7%; AIME 2024 single attempt 92.0%; LiveCodeBench v5 single attempt 70.4%; Aider Polyglot whole 74.0% and diff 68.6%; SWE-Bench Verified 63.8%; SimpleQA 52.9%; MMMU single attempt 81.7%; Vibe-Eval (Reka) 69.4%; MRCR 128k average 94.5% and 1M pointwise 83.1%; Global MMLU (Lite) 89.8%. The chart methodology identifies the model ID as gemini-2.5-pro-exp-03-25, uses default sampling, reports Gemini results as pass@1 unless otherwise indicated, and states that Vibe-Eval uses Gemini as a judge. The announcement text explicitly describes SWE-Bench Verified as using a custom agent setup. Evaluation date is not separately published.

条件：Whole-file setting; AI Studio API model ID gemini-2.5-pro-exp-03-25; default sampling; evaluation date not separately published.

证据：Google's official Gemini 2.5 Pro Experimental (03-25) chart reports: Humanity's Last Exam no tools 18.8%; GPQA diamond single attempt 84.0%; AIME 2025 single attempt 86.7%; AIME 2024 single attempt 92.0%; LiveCodeBench v5 single attempt 70.4%; Aider Polyglot whole 74.0% and diff 68.6%; SWE-Bench Verified 63.8%; SimpleQA 52.9%; MMMU single attempt 81.7%; Vibe-Eval (Reka) 69.4%; MRCR 128k average 94.5% and 1M pointwise 83.1%; Global MMLU (Lite) 89.8%. The chart methodology identifies the model ID as gemini-2.5-pro-exp-03-25, uses default sampling, reports Gemini results as pass@1 unless otherwise indicated, and states that Vibe-Eval uses Gemini as a judge. The announcement text explicitly describes SWE-Bench Verified as using a custom agent setup. Evaluation date is not separately published.

条件：Single attempt; pass@1; AI Studio API model ID gemini-2.5-pro-exp-03-25; default sampling; evaluation date not separately published.

证据：Google's official Gemini 2.5 Pro Experimental (03-25) chart reports: Humanity's Last Exam no tools 18.8%; GPQA diamond single attempt 84.0%; AIME 2025 single attempt 86.7%; AIME 2024 single attempt 92.0%; LiveCodeBench v5 single attempt 70.4%; Aider Polyglot whole 74.0% and diff 68.6%; SWE-Bench Verified 63.8%; SimpleQA 52.9%; MMMU single attempt 81.7%; Vibe-Eval (Reka) 69.4%; MRCR 128k average 94.5% and 1M pointwise 83.1%; Global MMLU (Lite) 89.8%. The chart methodology identifies the model ID as gemini-2.5-pro-exp-03-25, uses default sampling, reports Gemini results as pass@1 unless otherwise indicated, and states that Vibe-Eval uses Gemini as a judge. The announcement text explicitly describes SWE-Bench Verified as using a custom agent setup. Evaluation date is not separately published.

条件：Single attempt; pass@1; AI Studio API model ID gemini-2.5-pro-exp-03-25; default sampling; evaluation date not separately published.

证据：Google's official Gemini 2.5 Pro Experimental (03-25) chart reports: Humanity's Last Exam no tools 18.8%; GPQA diamond single attempt 84.0%; AIME 2025 single attempt 86.7%; AIME 2024 single attempt 92.0%; LiveCodeBench v5 single attempt 70.4%; Aider Polyglot whole 74.0% and diff 68.6%; SWE-Bench Verified 63.8%; SimpleQA 52.9%; MMMU single attempt 81.7%; Vibe-Eval (Reka) 69.4%; MRCR 128k average 94.5% and 1M pointwise 83.1%; Global MMLU (Lite) 89.8%. The chart methodology identifies the model ID as gemini-2.5-pro-exp-03-25, uses default sampling, reports Gemini results as pass@1 unless otherwise indicated, and states that Vibe-Eval uses Gemini as a judge. The announcement text explicitly describes SWE-Bench Verified as using a custom agent setup. Evaluation date is not separately published.

条件：Pass@1; AI Studio API model ID gemini-2.5-pro-exp-03-25; default sampling; evaluation date not separately published.

证据：Google's official Gemini 2.5 Pro Experimental (03-25) chart reports: Humanity's Last Exam no tools 18.8%; GPQA diamond single attempt 84.0%; AIME 2025 single attempt 86.7%; AIME 2024 single attempt 92.0%; LiveCodeBench v5 single attempt 70.4%; Aider Polyglot whole 74.0% and diff 68.6%; SWE-Bench Verified 63.8%; SimpleQA 52.9%; MMMU single attempt 81.7%; Vibe-Eval (Reka) 69.4%; MRCR 128k average 94.5% and 1M pointwise 83.1%; Global MMLU (Lite) 89.8%. The chart methodology identifies the model ID as gemini-2.5-pro-exp-03-25, uses default sampling, reports Gemini results as pass@1 unless otherwise indicated, and states that Vibe-Eval uses Gemini as a judge. The announcement text explicitly describes SWE-Bench Verified as using a custom agent setup. Evaluation date is not separately published.

条件：Single attempt; pass@1; AI Studio API model ID gemini-2.5-pro-exp-03-25; default sampling; evaluation date not separately published.

证据：Google's official Gemini 2.5 Pro Experimental (03-25) chart reports: Humanity's Last Exam no tools 18.8%; GPQA diamond single attempt 84.0%; AIME 2025 single attempt 86.7%; AIME 2024 single attempt 92.0%; LiveCodeBench v5 single attempt 70.4%; Aider Polyglot whole 74.0% and diff 68.6%; SWE-Bench Verified 63.8%; SimpleQA 52.9%; MMMU single attempt 81.7%; Vibe-Eval (Reka) 69.4%; MRCR 128k average 94.5% and 1M pointwise 83.1%; Global MMLU (Lite) 89.8%. The chart methodology identifies the model ID as gemini-2.5-pro-exp-03-25, uses default sampling, reports Gemini results as pass@1 unless otherwise indicated, and states that Vibe-Eval uses Gemini as a judge. The announcement text explicitly describes SWE-Bench Verified as using a custom agent setup. Evaluation date is not separately published.

条件：No tools; Google chart methodology; AI Studio API model ID gemini-2.5-pro-exp-03-25; default sampling; evaluation date not separately published.

证据：Google's official Gemini 2.5 Pro Experimental (03-25) chart reports: Humanity's Last Exam no tools 18.8%; GPQA diamond single attempt 84.0%; AIME 2025 single attempt 86.7%; AIME 2024 single attempt 92.0%; LiveCodeBench v5 single attempt 70.4%; Aider Polyglot whole 74.0% and diff 68.6%; SWE-Bench Verified 63.8%; SimpleQA 52.9%; MMMU single attempt 81.7%; Vibe-Eval (Reka) 69.4%; MRCR 128k average 94.5% and 1M pointwise 83.1%; Global MMLU (Lite) 89.8%. The chart methodology identifies the model ID as gemini-2.5-pro-exp-03-25, uses default sampling, reports Gemini results as pass@1 unless otherwise indicated, and states that Vibe-Eval uses Gemini as a judge. The announcement text explicitly describes SWE-Bench Verified as using a custom agent setup. Evaluation date is not separately published.

条件：Single attempt; pass@1; AI Studio API model ID gemini-2.5-pro-exp-03-25; default sampling; evaluation date not separately published.

证据：Google's official Gemini 2.5 Pro Experimental (03-25) chart reports: Humanity's Last Exam no tools 18.8%; GPQA diamond single attempt 84.0%; AIME 2025 single attempt 86.7%; AIME 2024 single attempt 92.0%; LiveCodeBench v5 single attempt 70.4%; Aider Polyglot whole 74.0% and diff 68.6%; SWE-Bench Verified 63.8%; SimpleQA 52.9%; MMMU single attempt 81.7%; Vibe-Eval (Reka) 69.4%; MRCR 128k average 94.5% and 1M pointwise 83.1%; Global MMLU (Lite) 89.8%. The chart methodology identifies the model ID as gemini-2.5-pro-exp-03-25, uses default sampling, reports Gemini results as pass@1 unless otherwise indicated, and states that Vibe-Eval uses Gemini as a judge. The announcement text explicitly describes SWE-Bench Verified as using a custom agent setup. Evaluation date is not separately published.

条件：Single attempt; pass@1; AI Studio API model ID gemini-2.5-pro-exp-03-25; default sampling; evaluation date not separately published.

证据：Google's official Gemini 2.5 Pro Experimental (03-25) chart reports: Humanity's Last Exam no tools 18.8%; GPQA diamond single attempt 84.0%; AIME 2025 single attempt 86.7%; AIME 2024 single attempt 92.0%; LiveCodeBench v5 single attempt 70.4%; Aider Polyglot whole 74.0% and diff 68.6%; SWE-Bench Verified 63.8%; SimpleQA 52.9%; MMMU single attempt 81.7%; Vibe-Eval (Reka) 69.4%; MRCR 128k average 94.5% and 1M pointwise 83.1%; Global MMLU (Lite) 89.8%. The chart methodology identifies the model ID as gemini-2.5-pro-exp-03-25, uses default sampling, reports Gemini results as pass@1 unless otherwise indicated, and states that Vibe-Eval uses Gemini as a judge. The announcement text explicitly describes SWE-Bench Verified as using a custom agent setup. Evaluation date is not separately published.

条件：128k cumulative/average score; AI Studio API model ID gemini-2.5-pro-exp-03-25; evaluation date not separately published.

证据：Google's official Gemini 2.5 Pro Experimental (03-25) chart reports: Humanity's Last Exam no tools 18.8%; GPQA diamond single attempt 84.0%; AIME 2025 single attempt 86.7%; AIME 2024 single attempt 92.0%; LiveCodeBench v5 single attempt 70.4%; Aider Polyglot whole 74.0% and diff 68.6%; SWE-Bench Verified 63.8%; SimpleQA 52.9%; MMMU single attempt 81.7%; Vibe-Eval (Reka) 69.4%; MRCR 128k average 94.5% and 1M pointwise 83.1%; Global MMLU (Lite) 89.8%. The chart methodology identifies the model ID as gemini-2.5-pro-exp-03-25, uses default sampling, reports Gemini results as pass@1 unless otherwise indicated, and states that Vibe-Eval uses Gemini as a judge. The announcement text explicitly describes SWE-Bench Verified as using a custom agent setup. Evaluation date is not separately published.

条件：1M pointwise score; AI Studio API model ID gemini-2.5-pro-exp-03-25; evaluation date not separately published.

证据：Google's official Gemini 2.5 Pro Experimental (03-25) chart reports: Humanity's Last Exam no tools 18.8%; GPQA diamond single attempt 84.0%; AIME 2025 single attempt 86.7%; AIME 2024 single attempt 92.0%; LiveCodeBench v5 single attempt 70.4%; Aider Polyglot whole 74.0% and diff 68.6%; SWE-Bench Verified 63.8%; SimpleQA 52.9%; MMMU single attempt 81.7%; Vibe-Eval (Reka) 69.4%; MRCR 128k average 94.5% and 1M pointwise 83.1%; Global MMLU (Lite) 89.8%. The chart methodology identifies the model ID as gemini-2.5-pro-exp-03-25, uses default sampling, reports Gemini results as pass@1 unless otherwise indicated, and states that Vibe-Eval uses Gemini as a judge. The announcement text explicitly describes SWE-Bench Verified as using a custom agent setup. Evaluation date is not separately published.

条件：Pass@1; AI Studio API model ID gemini-2.5-pro-exp-03-25; default sampling; evaluation date not separately published.

证据：Google's official Gemini 2.5 Pro Experimental (03-25) chart reports: Humanity's Last Exam no tools 18.8%; GPQA diamond single attempt 84.0%; AIME 2025 single attempt 86.7%; AIME 2024 single attempt 92.0%; LiveCodeBench v5 single attempt 70.4%; Aider Polyglot whole 74.0% and diff 68.6%; SWE-Bench Verified 63.8%; SimpleQA 52.9%; MMMU single attempt 81.7%; Vibe-Eval (Reka) 69.4%; MRCR 128k average 94.5% and 1M pointwise 83.1%; Global MMLU (Lite) 89.8%. The chart methodology identifies the model ID as gemini-2.5-pro-exp-03-25, uses default sampling, reports Gemini results as pass@1 unless otherwise indicated, and states that Vibe-Eval uses Gemini as a judge. The announcement text explicitly describes SWE-Bench Verified as using a custom agent setup. Evaluation date is not separately published.

条件：Custom agent setup; AI Studio API model ID gemini-2.5-pro-exp-03-25; evaluation date not separately published.

证据：Google's official Gemini 2.5 Pro Experimental (03-25) chart reports: Humanity's Last Exam no tools 18.8%; GPQA diamond single attempt 84.0%; AIME 2025 single attempt 86.7%; AIME 2024 single attempt 92.0%; LiveCodeBench v5 single attempt 70.4%; Aider Polyglot whole 74.0% and diff 68.6%; SWE-Bench Verified 63.8%; SimpleQA 52.9%; MMMU single attempt 81.7%; Vibe-Eval (Reka) 69.4%; MRCR 128k average 94.5% and 1M pointwise 83.1%; Global MMLU (Lite) 89.8%. The chart methodology identifies the model ID as gemini-2.5-pro-exp-03-25, uses default sampling, reports Gemini results as pass@1 unless otherwise indicated, and states that Vibe-Eval uses Gemini as a judge. The announcement text explicitly describes SWE-Bench Verified as using a custom agent setup. Evaluation date is not separately published.

条件：Gemini used as a judge; AI Studio API model ID gemini-2.5-pro-exp-03-25; default sampling; evaluation date not separately published.

证据：Google's official Gemini 2.5 Pro Experimental (03-25) chart reports: Humanity's Last Exam no tools 18.8%; GPQA diamond single attempt 84.0%; AIME 2025 single attempt 86.7%; AIME 2024 single attempt 92.0%; LiveCodeBench v5 single attempt 70.4%; Aider Polyglot whole 74.0% and diff 68.6%; SWE-Bench Verified 63.8%; SimpleQA 52.9%; MMMU single attempt 81.7%; Vibe-Eval (Reka) 69.4%; MRCR 128k average 94.5% and 1M pointwise 83.1%; Global MMLU (Lite) 89.8%. The chart methodology identifies the model ID as gemini-2.5-pro-exp-03-25, uses default sampling, reports Gemini results as pass@1 unless otherwise indicated, and states that Vibe-Eval uses Gemini as a judge. The announcement text explicitly describes SWE-Bench Verified as using a custom agent setup. Evaluation date is not separately published.