Optimizing Foundation Models

Question 1

What are vector embeddings? How do they play a part in RAG?

Answer 1

Embedding is the process by which text, images, and audio are given numerical representation in a vector space. Embedding is usually performed by a machine learning (ML) model.

In RAG, enterprise datasets, such as documents, images and audio, are passed to ML models as tokens and are vectorized. These vectors in an n-dimensional space, along with the metadata about them, are stored in purpose-built vector databases for faster retrieval.

Question 2

What is a vector database?

Answer 2

1. Compactly stores billions of high-dimensional vectors representing works and entities.
2. They provide ultra-fast similarity searches across these billions of vectors in real time
3. Uses K-NN for searches.
4. AWS Services for VDBs - OpenSearch, pgvector extension in RDS, Kendra

Question 3

What are Agents in an AI system?

Answer 3

Agents interact with the environment to perform intermediary operations
Coordinate multi-step functions
Example of an agent:
A chatbot may have an agent to modify/reset a customer’s password or phone plan
Another agent may send a CSAT survey to the customer when the conversation ends.

Question 4

How do you evaluate a Gen AI system?

Answer 4

1. Human Evaluation - evaluates user experience, contextual appropriateness, creativity, and flexibility.
2. Benchmark datasets - a quantitative way to evaluate generative AI models (e.g. Accuracy, Speed, Scalability).

Question 5

What is involved in creating a benchmark dataset?

Answer 5

SMEs have to do this manually.
They create intelligent questions.
Then they craft answers for them.
These datasets are then used to judge the performance of the model.
A “judge model” could be used to automate this process - i.e. a Judge Model takes the output of the model under evaluation and compares it to the benchmark dataset created by the SME and issue a grading score.

Question 6

What are the benefits of fine tuning?

Answer 6

1. Increases specificity
2. Improves accuracy
3. Reduces biases
4. Boosts efficiency

Question 7

What are the different types of FT?

Answer 7

1. Instruction tuning - involves retraining the model on a new dataset that consists of prompts followed by the desired outputs.
2. Reinforcement learning from human feedback (RLHF): uses a reward model based on human feedback.
3. Adapting models for specific domains - e.g. legal or healthcare
4. Continuous pre-training - the initial training phase is extended to keep the model current.

Question 8

What are the key steps in data preparation for fine tuning?

Answer 8

1. Data curation - more rigorous that for base FM. High impact data, highly relevant, labelled.
2. Labeling - accurate labeling is essential
3. Governance and compliance - specialized data to be handled with care
4. Bias checking - ensure data is balanced and does not introduce any new bias.
5. Feedback integration - integrating user feedback back into the training process.

Question 9

What are a few standard metrics for evaluating LLMs?

Answer 9

1. ROUGE - evaluate automatic summarization of texts, in addition to machine translation quality in NLP; e.g. measure overlaps, unigrams, bigrams and n-grams between machine and human generated reference text. (effectively ensures completeness of information)
2. BLEU - evaluate the quality of text that has been machine-translated from one natural language to another (accurate inclusion of critical features).
3. BERT - evaluate the quality of text-generation tasks; measures the cosine similarity between generated and reference texts. Effectively estimates the semantic appropriateness.