Last week, Google announced the latest Gemini-1.5-Pro-002 model for developers, and it’s now live in Gemini Advanced for paid subscribers.
Gemini Advanced now uses a “chat optimized version” of 1.5 Pro-002. For developers, Google offered a comparison of improvements across general, math, vision, and code benchmarks compared to the previous version (May 2024):
- “We see a ~7% increase in MMLU-Pro, a more challenging version of the popular MMLU benchmark.”
- “On MATH and HiddenMath (an internal holdout set of competition math problems) benchmarks, both models have made a considerable ~20% improvement.”
- “For vision and code use cases, both models also perform better (ranging from ~2-7%) across evals measuring visual understanding and Python code generation.”
For Gemini Advanced users today, this means “better and more accurate responses for prompts related to math and exploring complex topics that invite thoughtful conversation.” Google credits “continuous model training and valuable user feedback.”
Whether you’re providing detailed, multi-step instructions, or tackling advanced mathematical calculations, Gemini Advanced will help you navigate these challenges with greater ease.
This follows free Gemini being upgraded to 1.5 Flash in July and getting “major latency improvements” in August.
More on Gemini:
FTC: We use income earning auto affiliate links. More.