Unity Catalog

Question 1

4 key areas of Data Governance

Answer 1

1. Data Access Control
2. Data Access Audit
3. Data Lineage
4. Data Discovery

Question 2

Unity Catalog capabilities

Answer 2

• Centralised metadata and user management
• Centralised data access controls
• Data access auditing
• Data lineage
• Data search and discovery
• Secure data with Delta Sharing

Question 3

Data Governance with vs without Unity Catalog

Answer 3

Centralised access control, auditing, lineage, and data discovery across Databricks workspaces.

Question 4

Unity Catalog object model

Answer 4

• Metastore: top-level container for metadata. Registers metadata about data and AI assets and permissions that govern access to them.
• Catalog: used to organise your data and used as the top level in your data isolation schema. Often split by organisational units or software development lifecycle scopes.
• Schema (i.e. database): contain tables, views, volumes, AI models and functions. Organise data and AI assets into logical categories

Question 5

Unity Catalog three-level namespace

Answer 5

• Traditional SQL is managed at the two-level namespace, i.e. SELECT * FROM schema.table
• Unity Catalog manages at the three-level namespace, i.e. SELECT * FROM catalog.schema.table

Question 6

List the 6 types of principals in Unity Catalog

Answer 6

1. Cloud Admin: manage underlying cloud resources
2. Identity Admin: manage users and groups and provision to the account
3. Account Admin: create or delete metastores, manage users and groups, full access to all data objects
4. Metastore Admin: create or drop, grant privileges on, and change ownership of catalogs (and below)
5. Data Owner: owns data objects they created
6. Workspace Admin: manages permissions on workspace assets, restricts access to cluster creation, add/removes users, grants privileges etc

Question 7

List the 5 Identities in Unity Catalog

Answer 7

1. Users
2. Account Administrators
3. Service Principals
4. Service Principals with administrative privileges
5. Groups (e.g. “allusers” includes “analysts” and “developers”)

Question 8

Privileges for Metastore

Answer 8

CREATE CATALOG
CREATE EXTERNAL LOCATION
CREATE SHARE
CREATE RECIPIENT
CREATE PROVIDER

Question 9

Privileges for Catalog

Answer 9

USE CATALOG
CREATE SCHEMA

Question 10

Privileges for Schema

Answer 10

USE SCHEMA
CREATE TABLE
CREATE FUNCTION

Question 11

Privileges for Table

Answer 11

SELECT
MODIFY

Question 12

Privileges for View

Answer 12

SELECT

Question 13

Privileges for External Location

Answer 13

CREATE EXTERNAL TABLE
READ FILES
WRITE FILES
CREATE MANAGED STORAGE

Question 14

Privileges for Storage credential

Answer 14

CREATE EXTERNAL TABLE
READ FILES
WRITE FILES
CREATE EXTERNAL LOCATION

Question 15

Privileges for Function

Answer 15

EXECUTE

Question 16

Dynamic Views (3)

Answer 16

1. Limit access to columns (omit column values from output)
2. Limit access to rows (omit rows from output)
3. Data masking (obscure data in certain fields)

Question 17

Unity Catalog Store Credential

Answer 17

• Enables Unity Catalog to connect to an external cloud storage (e.g. IAM role for AWS S3, Service Principal for Azure Storage)

Question 18

Unity Catalog External Location

Answer 18

Cloud storage path + storage credential
- Self-contained object for accessing specific locations in cloud storage
- Fine-grained control over external storage

Question 19

Managing Owner Permissions (sql)

Answer 19

ALTER SCHEMA schema_name OWNER TO username
ALTER TABLE table_name OWNER TO username
ALTER VIEW view_name OWNER TO username
ALTER FUNCTION function_name TO username

• Only owner can grant access to an object
• User who creates the object becomes its owner, regardless of the owner of the parent object

Question 20

Revoking Permissions

Answer 20

REVOKE [privilege_type] ON [data_object_type] [data_object_name] FROM [user_or_group_name]

e.g.
REVOKE ALL PRIVILEGES ON SCHEMA default FROM alf@melmak.et;
REVOKE SELECT ON TABLE t FROM aliens;

Question 21

Grant Permissions

Answer 21

GRANT privilege_types ON securable_object TO principal

GRANT CREATE ON SCHEMA <schema-name> TO `alf@melmak.et`;
GRANT ALL PRIVILEGES ON TABLE forecasts TO finance;
GRANT SELECT ON TABLE sample_data TO USERS;</schema-name>

Question 22

Unity Catalog best practices

Answer 22

1. Only 1 metastore per geographic region (share data between regions with Delta Sharing)
2. Use catalogs (not metastores) to segregate data, then apply permissions to groups on certain catalogs (USAGE on catalog -> USAGE on schema -> SELECT on Tables)
3. Segregate data by Catalogs. E.g. business unit + environment scope
4. Manage all identities at the account level
5. Use groups rather than users to assign access and ownership to objects
6. Use Service Principals to run production jobs