Benefits of employing an architecture
- Baseline for requirements
- Easy development of new applications
- Reuse architectural assets, products
- Plattform for selecting new products (tools, apps)
- Fewer decisions, hence speed
- Set of architectural standards
- Defines the business context for sustainable BI
- Forces the business to think about the big picture
- Enables analytics across a range of processes
- Avoids premature rush to selecting products
- Restrains the IT function, business power users
The four BI architecture categories
- Each of these categories will have sub layers
- Requirements flow downward
- Implementation flow upwards
of the four types of architecture, where is BPM used
information architecture
Questions that need answers when designing archictures
define information architecture
the structural design of shared information environments;
the art and science of organizing and labelling websites, intranets, online communities and software to support usability and findability;
Basic Process of Information Architecture
- to gather data from inside and outside the enterprise
- transform it into information that the business uses to operate its business today and to plan for the future.
purpose of modelling
Data modelling is about defining the target data structures.
- standardise the process
- reproduce the process
- increase efficiency
- measure the process
define data integration
- combines data from different data sources
- provides users with a unified view of the data
Examples:
- commercial: two similar companies need to merge their databases
- scientific: combining research results from different bioinformatics repositories
Data integration appears with increasing frequency as the volume and the need to share existing data explodes.
Data Integration Framework Building Blocks
- This is a lot more than an Extract, Transform and Loadtool; ETL tools are only one element in a DIF.
- Beware of magic bullets, panaceas and of people who tell you their latest tool will fix all your DI problems
- As in a lot of Computing, we have the triangle of People, Process and Technology. Architecture and Standards are no less important
Describe Data Integration Frameworks (DIF)
A combination of architecture, processes, standards, people and tools used to transform enterprise data into information for tactical reporting and strategic analysis
Data integration framework (DIF) information architecture
- 6 step process
- it’s purpose
- Take data from systems of record,
- integrate it
- put it in the EDW,
- extract from the EDW
- put into data marts or OLAP cubes
- apply BI and analytics
The objective of the architecture is to gather data from inside and outside the enterprise and transform it into information that the business uses to operate its business today and to plan for the future
Data is gathered, transformed using business rules and technical conversions, stored in databases tobemade available to business users for reporting and analysis
2 stages in Data Integration
- Data Preparation (collect)
- Data Franchising (distribute)
Architecture Components
- Data Preparation
- Data Franchising
- Business Intelligence and Analytics
- Data Management
- Metadata Management
Architecture Component:
Data Preparation (6 steps)
- Gather
- Reformat
- Consolidate
- Transform
- Clean
- Store
Architecture Component:
Data Franchising
- Create information for reporting and analysis with BI tools.
- Data further filtered, reorganised, transformed, summarised and/or aggregated, and stored
- Copied from DW to business area data marts or cubes
Architecture Component:
Business Intelligence and Analytics
Deliver data to business users using BI applications
- Reports, spreadsheets, alerts, graphics, analytic applications
Architecture Component:
Data Management
Processes and standards used to define, govern and manage a company’s enterprise information assets
Architecture Component:
Metadata Management
Processes, procedures and policies that define and manage the metadata used by the DIF
Define a Data Mart
The access layer of the data warehouse environment that is used to get data out to the users.
The data mart is usually oriented to a specific business line or team. Whereas data warehouses have an enterprise-wide depth, the information in data marts usually pertains to a single department or business area.
Data Preparation Step 1: Gather Data
Part of data integration:
- gather data from various internal and external sources
- usually mix of custom, package, cloud applications
- transform it according to business and technical rules
- stage it for later steps where it becomes information used by business consumers.
- Staging may not be in permanent physical files in every step of the process.
Data Profiling
Data profiling is about understanding the data in the source system, before going through the data preparation phase.
- Examine the structure, content of data sources
- Perform source system analysis
- Find anomalies, understand data quality
- Feed into design of the data integration workflow
Data Preparation Step 2: Reformat Data
- Convert the data to a common format and schema
- To be fed into a Data Warehouse
- Straightforward if there are schema, column definitions for the source data
- If not, you may need to discover them (use SME)
- All governed by master data in the Reference or Dimension tables
Database Schemas
- Schema is the structure of the database that defines the objects in the database
- In a relational database, the schema defines:
- database’s tables, fields, relationships, indexes, database links, directories, XML schemas, and other elements.
- Set of integrity constraints imposed on a database
Data Preparation Step 3: Consolidate, Standardise, Validate Data
- Provide a single, consistent definition for business users
- Validate by checking dimensions or reference tables
- To see if it conforms to specific business rules
- Reference files are metadata you build up to describe the eventual Data Warehouse
