What are the stages in the Generative AI Application Lifecycle?
- Define a use case
- Select a Foundation Model
- Improve Performance
- Evaluate Results
- Deploy the Application
What are the capabilities of Gen AI?
Adaptability
Responsiveness
Simplicity
Creativity and exploration
Data efficiency
Personalization
Scalability
What are the risks of Gen AI?
Regulatory violations
Social risks
Data security and privacy concerns
Toxicity
Hallucinations
Interpretability
Nondeterminism
What is involved in the first stage of Gen AI App LC - i.e. Defining a use case?
- Define the problem
- Gathering relevant requirements
- Aligning stakeholder expectations
In this state, business requirements are translated into technical specs that guide development.
Normally, a business case is done.
What are the most common elements of a business case?
- Use Case Name
- Description
- Actors
- Preconditions
- Success Path
- Alternate Flows
- Post conditions
- Business Rules
- Non functional requirements - e.g. security, performance, usability, etc.
- Assumptions
- Additional Info.
What sort of metrics can be used to set Gen AI application goals?
- Cost Savings
- Time Savings
- Quality Improvement
- Customer Satisfaction
- Productivity Gains
What are the various ways in which Gen AI can help businesses?
- Process Automation - automate repetitive or time-consuming tasks, such as content generation, data analysis, or customer service interactions
- Augmented decision making - enhance decision-making processes by providing insights, recommendations, and decision support
- Personalization - create personalized and customized content that is more relevant for a customer
- Creative content generation - can be valuable for marketing, advertising, entertainment, or educational purposes
- Exploratory analysis and innovation - explore new ideas, concepts, or solutions by generating novel combinations or variations.
What are some of the factors to consider when choosing a pre-trained model for Gen AI?
- Cost - large models can be expensive, need for computational resources for inference. Cost vs Benefits analysis needed.
- Modality - what is needed text generation? image? audio? multi-modal?
- Latency
- Multi-lingual support
- Model size - relates to cost and computational requirements. Larger models are expensive and need more compute but can also solve more complex problems
- Model complexity - simple use cases may not require complex models (e.g. transformer based)
- Customization
- Input/Output Length
- Responsibility Considerations - e.g. potential biases, misinformation risks
- Deployment and integration - compatibility with existing infra
What are the different ways to improve a model’s performance?
- Prompt Engineering
- RAG
- Fine-tuning
- Automation Agents
What is prompt engineering?
- Fastest way to harness the power of LLMs.
- You engineer the prompt in such a way so as to drive the required output
- The phrasing, structure, and content of the prompt can significantly influence the quality, relevance, and characteristics of the generated outputs.
- Prompt engineering aims to optimize the prompts to steer the model’s generation in the desired direction, using the model’s capabilities while mitigating potential biases or undesirable outputs.
What are some of the key aspects of prompt engineering?
- Design - prompt is clear, unambiguous, and context-rich
- Augmentation - incorporate additional info and constraints into the prompt
- Tuning - Iteratively refining and adjusting the prompts based on the model’s outputs and performance
- Ensembling: Combining multiple prompts or generation strategies to improve the overall quality and robustness of the outputs
- Mining: Exploring and identifying effective prompts through techniques like prompt searching, prompt generation, or prompt retrieval from large prompt libraries
What is RAG or Retrieval Augmented Generation?
RAG is a natural language processing (NLP) technique that combines the capabilities of retrieval systems and generative language models to produce high-quality and informative text outputs.
a) Retrieval system - retrieves relevant information from a large corpus knowledge base; retrieval systems themselves can be quite complex with embeddings and vector databases that are built from customer specific content.
2. LLM - large pre-trained model to which the user prompt along with additional retrieved info is passed.
RAG does not alter a model’s parameter weights.
It is good for Q&A systems and generating high quality content.
What is Amazon Bedrock Knowledge Bases?
This is a complete workflow for RAG.
It helps organizations build a knowledge base, retrieve relevant information, and augment prompts to LLMs.
To build a knowledge base, you have to ingest data, turn it into embeddings, and store it in a Vector Database.
What is Fine Tuning?
Training a FM with domain specific data to make it more relevant to the task at hand - e.g. medical research
It does alter a model’s parameter weights and adds additional layers.
There are two ways to fine tune:
a) Instruction fine-tuning - uses examples of how the model should respond to a specific instruction
b) Reinforcement learning from human feedback (RLHF) provides human feedback data
What are Agents?
Agents are software components or entities designed to perform specific actions or tasks autonomously or semi-autonomously, based on predefined rules or algorithms.
In the case of Amazon Bedrock, agents are used to manage and carry out various multi-step tasks related to infrastructure provisioning, application deployment, and operational activities.
Their primary task is task coordination. They can also do additional reporting and logging.
What are the various ways in which FMs can be evaluated?
- Human evaluation : expensive and time consuming
- Benchmark Datasets - e.g. GLUE, SQuAD etc.are benchmark datasets for NLP
- Automated Metrics: e.g. Perplexity (measure of how well the model predicts the next token) and BLUE score (for machine translations)- ROUGE, BLEU, and BERTScore provide assessment of the foundation model’s capabilities
