Lecture 1

Question 1

Goal of Lec 1

Answer 1

• Explain what ML & DL are
• Explain why DL matters
• Explain why efficiency matters
• Set mental framework for the whole course
  👉 This lecture is about “WHY”, not “HOW”.

Question 2

AI (Artificial Intelligence)

Answer 2

• Broad goal: make machines behave intelligently
• Many approaches exist:
  
  • Rule-based systems
  • Symbolic reasoning
  • Expert systems
  • Machine Learning

Question 3

Machine Learning (ML)

Answer 3

Definition (understand, don’t memorize): ML lets computers learn patterns from data, instead of following fixed rules.
Arthur Samuel (1959): “Computers learn without being explicitly programmed”

Question 4

Key contrast

Answer 4

• Rule-based programming
  Rules + Data → Output
• Machine learning
  Data + Output → Rules (model)
  💡 Mental model: ML = learning rules by observing examples, like humans do.

Question 5

Some problems are easy to hard-code

Answer 5

• Sorting numbers
• Arithmetic
• Following instructions

Question 6

Some problems are NOT

Answer 6

• Image recognition
• Face recognition
• Speech understanding
• Language translation

Question 7

Example: Image classification

Answer 7

Task: “Is this image a plane?”
Why rules fail:
* Different angles
* Different lighting
* Different backgrounds
* Different shapes
👉 Too many edge cases → rules explode
💡 Insight: If you cannot describe a task clearly in words, ML is useful.

Question 8

ML is used when

Answer 8

• Patterns exist
• Rules are unclear
• Data is available

Question 9

Examples

Answer 9

• Face recognition
• Voice assistants
• Search engines
• Recommendation systems
• Translation
  💡 Connection to real life: You learned faces and language without explicit rules — by exposure.

Question 10

Standard ML pipeline

Answer 10

1. Collect data + labels
2. Train a model
3. Test on new unseen data

Question 11

Training

Answer 11

• Model adjusts itself to fit data
• No explicit rules written by humans

Question 12

Evaluation

Answer 12

• Check if learning generalizes, not memorizes
  💡 Key word: generalization

Question 13

Strengths

Answer 13

• Pattern recognition
• Object detection
• Face recognition
• Anomaly detection
• Prediction
• Recommendation

Question 14

What ML is NOT good at

Answer 14

• Logical reasoning with guarantees
• Small data problems
• Tasks with strict rules
• Perfect correctness
  💡 Rule of thumb: ML ≠ logic engine
  ML ≈ statistical pattern learner

Question 15

Where Deep Learning fits

Answer 15

AI → Machine Learning → Neural Networks → Deep Neural Networks (DNNs)

Question 16

Other ML approaches

Answer 16

• Kernel methods
• Probabilistic models
• Genetic algorithms
• Symbolic methods
  This course focuses on: 👉 Deep Neural Networks

Question 17

Artificial Neural Network (ANN)

Answer 17

• Inspired by biological neurons
• Consists of:
  
  • Input layer
  • Hidden layers
  • Output layer

Question 18

Deep Neural Network (DNN)

Answer 18

• Neural network with more than 2 hidden layers

Question 19

Why depth matters

Answer 19

• Shallow networks learn simple patterns
• Deep networks learn hierarchical features
  Example (vision):
  
  • Early layers: edges
  • Middle layers: shapes
  • Later layers: objects
    💡 Mental model: Depth = layered abstraction

Question 20

Important terms

Answer 20

• AI → goal
• ML → learning from data
• DL → deep neural networks
• DNN / NN → the model
• CNN, RNN, Transformer → architectures
• LLM → large language-focused DNN
• GenAI → models that generate content
  💡 Memory trick: Architecture = structure
  Model = trained instance

Question 21

Three main reasons

Answer 21

1. Record-breaking performance
   * Vision (ImageNet)
   * Speech recognition
   * NLP (question answering)
2. Huge social impact
   * AlphaGo
   * ChatGPT
   * AI-generated art
   * Autonomous systems
3. Industry adoption
   * Search
   * Translation
   * Healthcare
   * Finance
   💡 Why now?
   * Large datasets
   * GPUs
   * Better algorithms

Question 22

Evidence

Answer 22

• Chatbots
• Self-driving cars
• Code generation
• Image generation
• Multi-agent systems
  💡 Key realization: AI is now infrastructure, not a toy.

Question 23

Why this matters

Answer 23

Deep learning models are:
* Powerful
* Accurate
* Expensive to run

Question 24

Energy definition

Answer 24

Energy = Power × Time
Energy-efficient DL means:
* Faster execution
* Lower power consumption

Question 25

Two main costs

Answer 25

1. Computation cost * Billions of operations * Example: ResNet-152 → 11.3 billion MACs 2. Storage cost * Huge number of parameters * Example: GPT-3 → 175 billion parameters 💡 Mental image: DNNs are like giant factories — powerful but costly.

Question 26

Scaling trends

Answer 26

* More data * Larger models * Higher accuracy * Higher compute cost Known as scaling laws: Accuracy ↑ when: * Model size ↑ * Data ↑ * Compute ↑ Problem: 👉 Energy cost ↑↑↑

Question 27

Two complementary paths

Answer 27

Path 1: Hardware innovation * GPUs * Specialized accelerators * Optimized memory access Path 2: Algorithm innovation * Model compression * Efficient training * Faster inference This course focuses on: 👉 Algorithm + HW/SW co-design

Question 28

Deep models can be

Answer 28

* Fooled by adversarial inputs * Vulnerable to attacks Understanding: * Attacks * Defenses * Robust design Is part of responsible AI

Question 29

You will learn

Answer 29

* ML basics * DNN fundamentals * Training & evaluation * Energy-efficient design * Robustness & security Not just using DL — understanding it deeply.

Question 30

One master story

Answer 30

Rules fail → ML learns from data → DL learns deep patterns → DL is powerful but expensive → Efficiency & robustness are critical

Question 31

One question per section

Answer 31

* Why ML? * Why depth? * Why efficiency? * Why robustness? If you can answer these aloud, you mastered Lec 1.

Question 32

Final One-Line Summary

Answer 32

Deep learning works because the world has patterns, not rules — but making it efficient, scalable, and safe is the real challenge.