Path2.Mod1.a - Make Data Available

Question 1

“S is for abS…”

Datastores; what they are

Answer 1

Datastores are abstractions for cloud data sources

Question 2

WS Enc Lay

Datastores; their advantages and why they promote Best Practices

Answer 2

• Attached to Workspaces
• Securely encapsulate and store connection info. This makes it easy to connect to storage services without having to provide connection details in code (Best Practices: No hardcoding sensitive info).
• Serves as a protective layer between users and the underlying storage service (they connect to the datastore, not directly to the storage service)

Question 3

BS FS DLG1 DLG2

Four Datastore Types

Answer 3

• Azure Blob Storage
• Azure File Share
• Azure Data Lake (Gen 1)
• Azure Data Lake (Gen 2)

Question 4

“S comes after R…”, MD

Data Assets; what they are and what you get when you create one

Answer 4

Data Assets are references to where data is stored (datastores, storage services, public URLs, locally stored data). When creating you create a reference to that data with a copy of its Metadata.

Question 5

S&R AC V IO

Data Assets; four advantages they provide

Answer 5

• share and reuse data with team members
• Access data during training without worrying about connection strings or data path
• Versioning the data asset’s metadata
• Use when executing ML tasks as Azure ML Jobs; assets can be parsed as both input or output to a Job.

Question 6

UFI UFO MLT

Three Data Asset Types

Answer 6

• URI file: specific files
• URI folder: points to a folder
• MLTable: either a folder or file that includes a schema to read as tabular data

Question 7

CB IB

Two Datastore Authentication Methods

Answer 7

• Credential-Based - Uses a Service Principal, Shared-Access Signature (SAS) Token or an Account Key
• Identity-Based - Uses Azure AD Identity or Managed Identity (AzureDefaultIdentity())

Question 8

• Azure ML Datastores create the underlying Storage Account on creation if the account doesn’t exist (T/F)
• Datastores are not required when you have access to the underlying data as you can use storage URIs directly (T/F)
• If you have direct access to your data, then connecting directly to a Storage location during Notebook experimentation is preferred because it avoids unnecessary programmatic overhead (T/F)

Answer 8

• False. They are an abstraction over it
• True
• True

Question 9

h(s) a(s) a

Usage for Uniform Resource Identifiers (URIs) wrt Data

Three common protocols for URIs, and the one that doesn’t require authentication

Answer 9

Used to find and access your data. URIs are pointers to the location of your data

• http(s) - public/private data stores in Blob storage or public locations
• abfs(s) - data stores in Azure Data Lake Gen 2
• azureml - datastore (ie an existing storage account on Azure). When referring to a datastore, you won’t need to authenticate; remember that the connection info is stored with it and Azure ML will use it automatically.

Question 10

• Protocol to use when connecting directly to a folder or file, and when to apply authentication
• Protocol to use when connecting to a datastore where the authentication info and connection are stored

Answer 10

• http(s), when the container is set to private
• azureml

Question 11

BC FS, OD

The two built-in Workspace Datastores and how many are created by default.
The DataSet that creates another datastore if used

Answer 11

• Two Azure Storage Blob Containers and two Azure Storage File Shares. Two of each are created with the workspace
• Another datastore is added if you use Open Datasets

Question 12

The default Datastore for Data Assets, the default one for Notebooks, and their respective prefix

Answer 12

• Data Assets =>Blob Container Datastores. Prefixed azureml-blobstore
• Notebooks =>File Share Datastores . Prefixed code-

Question 13

Parquet Files
- What they are
- Why they are better than .csv files for ML tasks
- When reading in, all columns are convert to…

Answer 13

• Parquet Files are columnar file formatted files with optimizations that speed up queries
• Compared to csv files which are in row format, Parquet files are in compressed column format. Since ML tasks deal with Features, which are in tabular data the columns, ML tasks can process these more efficiently
• Nullable columns, for compatibility reasons (ex. Schema Merging)