The founders of prominent venture capital firm Andreessen Horowitz are observing a plateau in artificial intelligence capabilities, citing data shortages and computational limitations as key barriers to continued rapid advancement.
What Happened: Speaking on a podcast released last week, Marc Andreessen noted that while OpenAI‘s GPT-3.5 was once significantly ahead of competitors, “Sitting here today, there’s six that are on par with that. They’re sort of hitting the same ceiling on capabilities.”
This observation aligns with recent statements from Ilya Sutskever, co-founder of OpenAI and Safe Superintelligence (SSI), who acknowledged that results from traditional AI scaling approaches have plateaued. “The 2010s were the age of scaling, now we’re back in the age of wonder and discovery once again,” Sutskever said in an interview with Reuters.
Despite these challenges, Andreessen Horowitz maintains a significant investment in AI. The firm has even established a GPU lending program to address the critical shortage of computing hardware.
Ben Horowitz highlighted multiple infrastructure challenges beyond chip availability. “Once they get chips we’re not going to have enough power, and once we have the power we’re not going to have enough cooling,” he explained.
See Also: Greg Brockman Returns To Work At ChatGPT’s Parent Company: ‘Back To Building OpenAI’
Why It Matters: Data scarcity presents another significant hurdle. According to the Data Provenance Initiative, between April 2023 and 2024, websites restricted 5% of all data and 25% of high-quality data sources from AI training use. This limitation has led to an unexpected trend: AI companies are hiring human experts to generate training data manually.
“The big models are trained by scraping the internet…and there’s just literally only so much of that,” Andreessen noted. He pointed out the irony that AI companies are now “hiring thousands of programmers and doctors and lawyers to actually handwrite answers to questions” for training purposes.
This situation has created what Andreessen describes as an “AI hiring boom,” contrasting with widespread concerns about AI-driven job displacement.
Read Next:
Image Via Shutterstock
Disclaimer: This content was partially produced with the help of Benzinga Neuro and was reviewed and published by Benzinga editors.
Market News and Data brought to you by Benzinga APIs
© 2024 Benzinga.com. Benzinga does not provide investment advice. All rights reserved.