Back to News
AI Performance Surges on Demanding Benchmarks in 2024
AI Performance Surges on Demanding Benchmarks in 2024
1/15/2025

.

Analysis of 2024 data reveals a significant leap in Artificial Intelligence capabilities, with scores on the MMMU, GPQA, and SWE-bench benchmarks rising by 18.8, 48.9, and 67.3 percentage points respectively. This robust performance indicates enhanced reasoning and problem-solving capacities across complex, multi-modal, and code-centric evaluations. Furthermore, language model agents have demonstrated the capacity to outperform human programmers in time-constrained scenarios, signaling a transformative shift towards autonomous cognitive agents. This progress is underscored by a staggering 280-fold reduction in GPT-3.5-level inference costs between November 2022 and October 2024, coupled with annual hardware cost declines and energy efficiency improvements, paving the way for widespread advanced AI deployment and accessibility.