There’s actually a lot of human intervention in the mix. Data labelers for source data, also domain experts who will rectify answers after a first layer of training, some layers of prompts to improve common answers. Without those domain experts, the LLM would never have the nice looking answers we are getting.
I think the human intervention is going to increase to counter the AI pollution in the data sources. But it may not be economically viable anymore eventually.
There’s actually a lot of human intervention in the mix. Data labelers for source data, also domain experts who will rectify answers after a first layer of training, some layers of prompts to improve common answers. Without those domain experts, the LLM would never have the nice looking answers we are getting. I think the human intervention is going to increase to counter the AI pollution in the data sources. But it may not be economically viable anymore eventually.
This is a nice deep dive of the different steps to make today’s LLMs: https://youtube.com/watch?v=7xTGNNLPyMI