Data engineering Flashcards

Question

"No one shall be subjected to arbitrary interference with his privacy, family, home or correspondence, nor to attacks upon his honour and reputation. Everyone has the right to the protection of the law against such interference or attacks." | Where does the quote originate from?

Answer 1

Article 12 of the Universal Declaration of Human Rights (1948)

Answer 2

all organizations in the EU and those that offer goods or services in the EU, or collect and analyze data related to EU residents, regardless of their geographic locations

Answer 3

– accuracy – data minimization – **integrity** and confidentiality – lawfulness, fairness, and transparency – purpose limitation – **storage** limitation

Answer 4

That personal data shall be adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed.

Answer 5

The processing of personal data is not allowed unless it is legally required, the processor has a legitimate interest, or the data subject has consented.

Answer 6

All appropriate measures are required to ensure inaccurate personal information is erased or corrected immediately, considering the nature of the processing.

Answer 7

public interest, science, historical research, or statistical research, following fair, lawful, and transparent practices

Answer 8

– Right of access – Right of **transparent** **information**, communication, and modalities – Right to be notified – Right to data portability – Right to erasure or to be forgotten – Right to know the source of personal data – Right to object – Right to rectification – Right to reject automated individual decision-making – Right to restriction of processing

Answer 9

When data are processed to prevent, investigate, detect, or prosecute criminal offenses, or to prevent public security threats.

Answer 10

the regularion is technology-agnostic

Answer 11

**OneTrust** – comprehensive privacy management **BigID** – AI-powered data discovery and privacy compliance **Ketch** – consent and data rights automation **PrivacyPerfect** – EU-based SaaS for DPOs and privacy officers **SAI360** – integrated GRC with GDPR modules **Securiti** – unified data controls and privacy automation **Transcend** – automated DSARs and consent management **TrustArc** – privacy management and assessments

Answer 12

– incorrect data modification – unauthorized data observation – unavailability of data

Answer 13

165 million

Answer 14

– birth date – email address – encrypted & unencrypted security Qs – hashed password – name – telephone number

Answer 15

for over a year

Answer 16

between 50 and 90 million

Answer 17

security requirements engineering

Answer 18

– **CLASP** (Comprehensive, Lightweight Application Security Process) – **Core** security requirements artifacts – **SQUARE** (Security Quality Requirements Engineering)

Answer 19

Step 1: Agree on definitions Step 2: Identify assets and security goals Step 3: Develop **artifacts** Step 4: Perform risk assessment Step 5: Select elicitation technique Step 6: Elicit security requirements Step 7: Categorize requirements Step 8: Prioritize requirements Step 9: Inspect requirements

Answer 20

– **Authorization** Pattern – **Multilevel** Security Pattern – RBAC pattern – Single Access Point Pattern

Answer 21

Discovering the appropriate data, accessing them, understanding their meaning, and determining how they can be used in a compliant way.

Answer 22

– a set of **processes** and **activities** that involve identifying, defining, monitoring, and enforcing data quality policies – data policies and data governance rules – **roles** and **responsibilities** of people involved in the data governance life cycle – **technology** for implementing data governance

Answer 23

1. Ensuring data is easy to find and access for authorized users. 2. Improving shared understanding of data through **clear definitions and metadata**. 3. Maintaining and enhancing data quality and trustworthiness. 4. Enabling **controlled self-service** use of data by business users. 5. Protecting data through strong privacy and security controls.

Answer 24

– CDO – Data governance council – Data owners – Data stewards

Answer 25

– Data **access** policy – Data **governance** **structure** policy – Data **integrity** policy – Data **usage** policy

Answer 26

Level 1: reliable data processing Level 2: **controlling** the data processing Level 3: data quality **planning** Level 4: system-level **data quality assurance** Level 5: **root cause analyses** of issues + meta-shit data approach reflections

Answer 27

**Accelerated Requirements** Method **Joint Application** Design Structured interviews

Answer 28

Making data more **findable** and accessible Making data more **understandable** Making data more **trustworthy** and improving the quality of data Empowering data users with **self-service** Ensuring data **privacy** and security

Answer 29

Opt-in — the data subject must take a clear, affirmative action (e.g., ticking an unchecked box) to indicate consent. | Pre-checked boxes or implied consent are not valid.

Answer 30

consent by the guardian for the child

Answer 31

reliable data processing

Answer 32

non-discretionary access control | non-DAC ## Footnote The Access Control Survey classifies traditional access control models into two primary categories: 1. DAC (Discretionary Access Control) 2. Non-DAC, which further divides into MAC, RBAC, and ABAC

Answer 33

increase productivity while reducing waste

Answer 34

an open-source, CLI-based software that manages cloud services using IaC

Answer 35

plain text files with an extension of .tf

Answer 36

Azure Resource Manager | the deployment and management service for Azure ## Footnote It provides a management layer that helps us create, update, and delete resources in an Azure account.

Answer 37

YAML e. g. "acidvuca\vlc\.github\workflows\test.yml"

Answer 38

After pushing their changes to a branch (or fork), when the work is ready for review and potential merge into a target branch (for example, main).

Answer 39

- build - deploy - document - package - stage - static analysis - test

Answer 40

Jenkins **controller** (manages configuration, schedules builds, serves the UI) and Jenkins **agents** (execute the actual build jobs, can be static nodes or dynamic cloud-provisioned instances).

Answer 41

♥ data extraction ♥ data ingestion ♥ data preprocessing ♥ data validation, training machine learning algorithms, reporting

Answer 42

Data Factory, Apache Airflow, Apache Nifi, Luigi, AWS Glue

Answer 43

immutability and idempotency of data

Answer 44

☻ replayability ☻ auditability ☻ scalability ☻ reliability ☻ security

Answer 45

A DAG models a pipeline as a graph of tasks with two constraints. Directed means each edge (dependency) flows one way: a task only runs once all of its upstream dependencies have completed. Acyclic means the dependency chain cannot loop back on itself — if A feeds B and B feeds C, then C cannot feed back into A. Together, these properties guarantee a deterministic, topologically sortable execution order with no deadlocks or infinite loops.

Answer 46

a data pipeline orchestrator and an ETL tool

Answer 47

♥ **Operators**, predefined tasks that can be linked together ♥ **Sensors**, operators that wait for external events to occur ♥ **TaskFlow**, a Python function that is packaged as an Airflow pipeline task

Answer 48

A wide variety of data formats can be used, including logs, geolocation information, and social media feeds. | Additionally, SFTP, HDFS, and Kafka are supported.

Answer 49

♦ deployment frequency ♦ **lead time for changes** ♦ change failure rate ♦ mean time to restore service

Answer 50

Dead code is any section of source code that can never be executed, regardless of input, because no control flow path reaches it — for example, code after an unconditional return, or a branch guarded by a condition that is always false. | Compilers typically detect and eliminate it during optimization. ## Footnote The related term dead store refers to code that does execute but whose result is never subsequently used.

Answer 51

the number of confirmed bugs during the development period divided by the software size

Answer 52

* (raw) lines of code (LOC) * percentage of comments (perCOM) * source lines of code (SLOC) * logical lines of code (LLOC)

Answer 53

the number of operators and operands

Answer 54

HV = LTH · log₂(VOC) | LTH = OP + OD (program length) VOC = UOP + UOD (program vocabulary) ## Footnote OP = total number of operators OD = total number of operands UOP = number of distinct operators UOD = number of distinct operands

Answer 55

operators: =, + operands: int, x, 1

Answer 56

the understandability, maintainability, and testability of code

Answer 57

CC = E − N + 2 E is the number of edges and N the number of nodes

Answer 58

A fully covered program requires the same number of tests as its cyclomatic complexity.

Answer 59

how easy is it to support and change the code

Answer 60

MI = 171 − 5.2 · ln(aveHV) − 0.23 · aveCC − 16.2 · ln(aveLOC) + 50 · sin(√(2.4 · perCM))

Answer 61

to track incomplete work or backlogs over a given period, as well as to measure team progress

Answer 62

- **Jaeger** (for microservices) - **OpenCensus** (data viewing on the host + exporting the data to central aggregators) - Prometheus (for containerized applications) - Grafana (to visualize event-driven metrics such as response time, request volume, workloads, network traffic flow etc.)

Answer 63

CREATE, CLONE, RECEIVE

Answer 64

For each change, measure the time between when the change is initiated (or approved) and when it is fully implemented in production. Then calculate the average of these lead times across all changes. | lead_time_i = t_finish,i − t_start,i ## Footnote average lead time: L_bar = (1/N) ∑_i=1^N ( t_finish,i − t_start,i )

Answer 65

A failed change / change failure is a production deployment that results in degraded service or an incident and requires immediate remediation, such as a hotfix, rollback, or fix-forward.

Answer 66

The Change Failure Rate is determined by how many production deployments fail—that is, the number of unsuccessful deployments in production (those that cause incidents or require remediation) relative to the total number of deployments.

Answer 67

**Jenkins** is the open-source automation server commonly used for continuous integration and continuous delivery, allowing teams to automate building, testing, and deploying their software.

Answer 68

difficult to maintain

Data engineering Flashcards

Assist learner to become knowledgeable in data engineering (95 cards)