Google Cloud’s DevOps Research and Assessment (DORA) program this week released its 10th annual report, revealing a complex interplay between artificial intelligence (AI) adoption and software development performance.
The DORA report has focused on different aspects of DevOps and software development over its 10-year history. Both the 2022 and 2023 reports identified culture as key to success, something that hasn’t changed in 2024.
The 2024 Accelerate State of DevOps Report, however, also includes new insights about how AI is helping or, as the case may be, hindering software development. In addition, the new report looks at the emerging practice of platform engineering — and its implications on software development.
Here are key findings of the latest Google Cloud DORA report:
• AI adoption: More 75% of developers now use AI daily, with one-third reporting significant productivity gains, yet 39% remain skeptical of AI-generated code.
• Quality improvements: A 25% increase in AI adoption correlates with 7.5% better documentation, 3.4% improved code quality, and 3.1% faster code reviews
• Performance challenges: Despite productivity gains, increased AI adoption shows a 7.2% decline in delivery stability and a 1.5% decrease in delivery throughput.
• Platform engineering impact: While boosting developer productivity, particularly in large organizations, implementation of platform engineering initially causes temporary performance decreases.
• Developer experience: Unstable priorities and constant pivoting negatively impact performance, even when other positive organizational factors are present.
AI Benefits Not Translating to Better Delivery
In one of the report’s most surprising revelations, researchers found that while AI is improving certain aspects of software development, these gains aren’t resulting in better overall software delivery performance.
“We see AI helping with some different measures of productivity, some of what we traditionally have called capabilities, things like documentation quality, things like developer productivity,” Nathen Harvey, DORA lead and developer advocate at Google Cloud, told ITPro Today. “But we aren’t seeing those translate into better software delivery performance.”
The findings suggest that developers might be missing opportunities to leverage AI effectively. Rather than focusing solely on code generation, the report recommends expanding AI usage across the entire software delivery lifecycle.
“Generating code was never the constraint when it came to software delivery performance,” Harvey explained. “You write the code, but then it’s got to shift down the line, and it goes through feedback cycles, maybe through some testing, through some approval processes, and so forth.”
One hypothesis for the disconnect between AI adoption and performance improvements centers on change management practices. The report suggests that AI-assisted development might be leading teams to abandon proven practices of implementing smaller, incremental changes.
“We have years of data and experience that tell us the best way to improve those software delivery metrics is to ship smaller changes,” Harvey said. “With AI helping us generate code faster, maybe our change sets are no longer small, and we’ve kind of forgotten that lesson that shipping small changes is the path to better software delivery performance.”
Cultural Health and Leadership
The report emphasizes the critical role of organizational culture in software development success, particularly highlighting the impact of leadership decisions on team performance. A key finding shows that constantly changing organizational priorities can negatively affect practitioners.
“Culture happens at the practitioner level, but it doesn’t happen in a vacuum,” Harvey said. “Culture requires investment and dedication from leadership as well.”
Interestingly, the research found that AI initiatives might be inadvertently helping stabilize organizational priorities.
“Organizations are saying we have to prioritize AI. They’re bumping other things out of their priorities and putting AI at the top of that list, which helps stabilize those priorities for teams,” he said.
Industry Evolution and Future Direction
The report reflects significant changes in the software development industry, including a potential shift away from traditional DevOps terminology. While the foundational principles remain valuable, the DORA team suggests that the DevOps label may be reaching the end of its usefulness.
“When I use the word ‘DevOps,’ that opens about as many doors as it closes because people have tried this DevOps transformation thing, and it didn’t necessarily work for them,” Harvey said.
Looking forward, DORA itself is evolving, moving away from its original acronym (DevOps Research and Assessment) while maintaining its focus on helping teams improve.
“What DORA stands for is what our tagline has become — it’s about getting better at getting better,” he said.