One of the most shocking graphs currently is the Stack Overflow’s questions created per month.
So LLMs got pretty good at coding not just by reviewing code, but also by understanding how humans request. And Stackoverflow also has those edge cases and human creativity to find a workaround those edge cases.
Now with LLMs being so good, no one is going to stackoverflow to ask questions, because you could just ask it to the LLM.
But what happens when in the future there is no more training data, where will the LLMs get the creative ideas from? What will they train from?
I guess they will just learn by themselves. When a user asks it to do something and it misses the mark and the user provides it enough prompts and enough evidences and screenshots and explains why it is messing up, it will learn by itself. It won’t need stackoverflow data or quora data etc.
It needed them to get to a point where it is deemed useful by humans, so they interact with it. Once they start interacting with it, it doesn’t need training data, it will learn from the interaction.
See also Arbitrage Decay for the broader pattern of how each technological wave commoditises what came before, and Spatial Intelligence as Next Arbitrage for one defensible answer to “what’s next.”