For the past decade of scaling, we’ve been spoiled by the enormous amount of internet data that was freely available for us to use. This was enough for cracking natural language processing, but not for getting models to become reliable, competent agents. Imagine trying to train GPT-4 on all the text data available in 1980—the data would be nowhere near enough, even if we had the necessary compute. In 2025, our situation when it comes to automating software engineering is no different
— Read on www.mechanize.work/blog/how-to-fully-automate-software-engineering/