ETL - basic concepts
The structure of an ETL process: Domains, routines and jobs¶
The structure of the ETL process is organised into several levels:
Structure details¶
The description of each level is summarised in the following table:
| Level | Description |
|---|---|
| 1. ETL domain | An ETL domain is a container macro which includes all the dimensions and dictionaries, SQL models and the parameters related to an ETL rule. |
| 2. Routines and jobs | A routine is the minimum data processing unit that can be run in an ETL process and contains all the information for running the routine and the link containing details for extracting, transforming and loading the data. One or several routines can be defined for every ETL domain. An ETL job allows you to run, sequentially and in a defined order, several ETL routines (which may also belong to different ETL domains) in a single data process. |
| 3. Details | The details of a routine specify how many and which external sources are available, which data processing/transformation rules to apply to the extracted data and lastly how and where the system must load the data that has been previously extracted and transformed. |
IMPORTANT: All the information related to the configuration, routines and details is specific to an ETL domain. As a result, every management window will ask you to enter the ETL domain to work on as an incoming filter.