Your point about needing large models in the first place is well taken.
But I still think we would want a curated collection of chat/assistant training data if we want to use that language model and train it for a chat/assistant application.
So this is a two-phase project, the first phase being training a large model (GPT), the second being using Reinforcement Learning from Human Feedback (RLHF) to train a chat application (InstructGPT/ChatGPT).
There are definitely already people working on the first part, so it's useful to have a project focusing on the second.
But I still think we would want a curated collection of chat/assistant training data if we want to use that language model and train it for a chat/assistant application.
So this is a two-phase project, the first phase being training a large model (GPT), the second being using Reinforcement Learning from Human Feedback (RLHF) to train a chat application (InstructGPT/ChatGPT).
There are definitely already people working on the first part, so it's useful to have a project focusing on the second.