what role does machine learning play in automation
ML enables automation by training models to learn from data, allowing systems to make decisions or predictions without being explicitly programmed
what is a machine learning model
a model is the output of a training process that represents learned patterns from data, used to make predictions or decisions
what is devops and how does it relate to automation
- set of practises, tools and cultural philosophy that integrate software development and operations to automate/streamline code build, test, deploy and monitor processes (CI/CD)
- emphasises teamwork, communication and collaboration, and technology automation
- ML detects anomalies, predicts failures and optimises pipelines in workflows
what is MLOps and how is it different from devops
automated process of designing, training, deploying and managing ML models
- involves both data scientists and operations teams
- deals with more complex, data driven systems
what is RPA
robotic process automation
- using software bots to automate routine and repetitive tasks performed by humans on computers
- subset of bpa - automates individual tasks within processes
- can interact with applications, manipulate data, trigger responses
- improves efficiency and reduces errors
- employees focus on more valuable/complex work
- streamline operations, improve productivity
what is bpa
business process automation
- automating entire workflows (complex, multi-step processes that traditionally require input, coordination and oversight)
- arranging multiple tasks, systems, data sources and people through automated workflows
- eg automating customer onboarding process
distinguish between ai and ml
- AI is the capability of a machine to mimic human intelligence (reasoning, discovering new information, inferring data)
- ML is a subset of AI that focuses on the ability of machines to learn from data and make predictions/decisions without being explicitly programmed
what are ml training models
- training involves using data to teach an algorithm to recognise patterns and make predictions/decisions
- supervised, unsupervised, semi-supervised, reinforcement
what is supervised learning
algorithm is trained on labelled data and learns to predict outcomes
- classification (label/category)
- regression (continuous value)
- eg spam detection
what is unsupervised learning
model finds patterns or clusters in unlabelled data without predefined outcomes
- can find hidden patterns, but can be inaccurate/longer training times + need human validation
- eg customer segmentation
what is semi-supervised learning
hybrid model that uses small amount of labelled data with large amount of unlabelled data
- trains on labelled data, then uses predictions on unlabelled data to create new points to add to training data
- cost effective, faster than supervised, more accurate than unsupervised
- but sensitive to noise/errors, computationally complex
- eg medical diagnosis
what is reinforcement learning
agent interacts with an environment, and receives feedback in the form of awards or penalties
- learns to maximise the cumulative reward over time
- useful when difficult to define specific goal/provide labelled data
- solving complex problems, flexibility, finding best sequence of actions to achieve goal
- but significant computational power, effectiveness relies on quality of reward function, requires lots of data to learn
- eg learning to play a game
what are common applications of key ml algorithms
- data analysis and forecasting (examining large datasets to find patterns / using insights to predict future outcomes by applying regression/classification)
- virtual personal assistants (software agents that use NPL and ML to interact with users and perform tasks - reinforcement learning)
- image recognition (ability of software to detect, classify and interpret visual data - CNNS, KNN, neural networks, logistic)
how ml algorithms can be used in data analysis
- trained with subset of data to automatically identify patterns and trends
- handle complex/nonlinear relationships
- used for predictions and forecasting
- learn from new data to improve accuracy over time
what are key types of ml algorithms (list them out)
- linear regression (predict continuous numeric values)
- polynomial regression (predict outcomes with nonlinear relationships - curve)
- logistic regression (predict categorical outcomes - uses sigmoid function)
- KNN (classify new data based on the majority class of its ‘k’ nearest neighbours)
- decision trees (make decisions by splitting data into branches based on features)
- neural networks (complex pattern recognition and predictions - artificial brain)
what are variables in ml
a characteristic, property or feature that represents data and provides information for models to learn
- features are independent input variables used to make predictions
- targets are dependent output variables that the model is trying to predict
- labels are used during training and testing for supervised
what are decision trees and how are they used in ml
- supervised learning model used for classification and regression
- tree-like structure of nodes and branches to represent combinations of decisions and consequences
- root/decision/leaf nodes
- splitting/pruning
- at each node the dataset is split based on value of feature - until stopping criteria are met
- overfitting is when model is too tailored on training data
- goal is to build accurate tree that is generalisable (can make predictions)
what are neural networks and how are they structured
- set of algorithms designed to recognise patterns by mimicking the way the human brain works)
- complex problems, lots of unstructured data
- series of neurons/interconnected nodes that receive inputs, processes, and produces an output
1. input layer receives input (each neuron represents feature of input data)
2. hidden layers perform computations/transformations on input data (connections have weights and thresholds, activation function)
3. output layer produces the final result/prediction (each neuron = possible output class)
describe the cycles or processes involved in neural networks
- training cycle (network learns from data - by comparing output to label and adjusting weights with backpropagation)
- execution cycle is when trained neuron network is used to make real world decisions (inference)
what are weights and biases (in neural networks)
- weights determine the strength of connections between neurons
- biases provide an additional parameter that shifts the activation function’s output
- activation function decides whether neuron should be activated based on weighted sum of inputs and bias
describe linear regression
- supervised learning algorithm for predicting continuous values
- assumes linear relationship
- residuals = actual output - predicted output
- bias and weight (parameters) calculated from training
- simple to implement/interpret, but outliers have big impact
describe logistic regression
- supervised ml algorithm that predicts probability by analysing relationship and classifying data into discrete classes
- predictive modelling (outputs probability of input belonging to category)
- sigmoid curve that maps values between 0 and 1
- works well when data is linearly separable (binary classification problems)
- logistic cost function measures how close prediction is
- gradient descent used to optimise w/b
describe k-nearest neighbour
- supervised learning algorithm for classification and regression
- ‘similar things exist in close proximity’
- instance based learning (training data stored)
- k is number of points the algorithm looks at to make decision (should be odd)
- classification (new point assigned most common class)
- regression (value predicted by taking average)
- performance decreases as data size increases, and sensitive to irrelevant features
what is cost function and cost
- cost function measures how well a regression models’ predictions match actual target values
- calculates total loss (prediction - actual) across all training examples
- aggregates errors across all points into single scalar value
- cost is the sum of the loss for all data points (quantifies error)
- mean square error (average of squares of difference between predicted and actual values)
