Define LLM.
A Large Language Model is an AI model trained on vast text data to understand and generate human-like text.
True or false: LLMs can perform zero-shot learning.
TRUE
Zero-shot learning allows LLMs to perform tasks without prior specific training on those tasks.
What does tokenization refer to in LLMs?
The process of converting text into smaller units called tokens for processing.
Fill in the blank: LLMs use _________ to predict the next word in a sequence.
probabilities
What is fine-tuning in the context of LLMs?
Adjusting a pre-trained model on a specific dataset to improve performance on a particular task.
Define transformer architecture.
A neural network design that uses self-attention mechanisms to process sequential data efficiently.
True or false: LLMs require supervised learning for all tasks.
FALSE
LLMs can also learn from unsupervised and semi-supervised methods.
What is the purpose of self-attention in LLMs?
To weigh the importance of different words in a sentence relative to each other.
Fill in the blank: The _________ layer in LLMs helps to capture long-range dependencies.
attention
What does pre-training involve for LLMs?
Training on a large corpus of text to learn general language patterns before fine-tuning.
Define context window.
The number of tokens the model considers at once when generating or understanding text.
True or false: LLMs can generate creative content.
TRUE
They can produce poetry, stories, and other creative forms based on prompts.
What is the role of embeddings in LLMs?
To represent words or phrases as vectors in a continuous vector space for better processing.
Fill in the blank: LLMs are often evaluated using _________ metrics.
perplexity
What is transfer learning in LLMs?
Using knowledge gained from one task to improve performance on a different but related task.
Define prompt engineering.
The practice of designing inputs to elicit desired responses from LLMs.
True or false: LLMs can understand contextual nuances in language.
TRUE
They can grasp subtleties like sarcasm or idioms based on training data.
What is a decoder in LLMs?
A component that generates output sequences from the encoded representations of input data.
Fill in the blank: _________ is a common challenge faced by LLMs regarding biased outputs.
Bias
What does model size refer to in LLMs?
The number of parameters in the model, affecting its capacity and performance.
Define data augmentation in LLM training.
Techniques used to increase the diversity of training data without collecting new data.
True or false: LLMs can only process English text.
FALSE
LLMs can be trained on multiple languages and can understand various linguistic structures.
What is inference in the context of LLMs?
The process of generating output from a trained model based on new input data.
Fill in the blank: _________ is a method to reduce overfitting in LLMs.
Regularization
