According to OpenAI’s paper on GPT-3, how much text was used to train the model?
570GB worth
That’s includes several billion human-written webpages and trillions of words.
It’s also includes over 500 million digitised books and more.
What when into training ChatGPT?
Trillions of words being processed for the equivalent of about 300 years by many supercomputers working simultaneously for months to create 170 billion connections between all those words!
What happens when anyone asks ChatGPT anything?
It calculates through its 170 billion connections between words to figure out which word would be most probable to be placed next to the last word, one word at a time.
What is a problem that has plagued the field of AI for decades which has been partially solved in ChatGPT’s case through the use of positive reinforcement learning?
Alignment- AI aligning in some way to human values when responding to prompts
