Endpoint page
Repository > Endpoint
Purpose of the page¶
This page allows you to configure the access to resources outside the application by defining an Endpoint. These Endpoints can be used when reading/writing from the Job ETL and other modules of Tagetik Software S.r.l..
Attributes¶
Page fields
Please find below the most important fields.
| Field | Description |
|---|---|
| Code | It specifies the unique identifier by which the Endpoint will be referenced by the modules ofTagetik Software S.r.l. that use it. |
| Description | Shows the description in full. |
| Endpoint type | Indicates the mode of access to the resource, selectable from URL, Tagetik Data Loader, Tagetik Application, OData V2 service, OData V4 service, REST API, SharePoint, Google BigQuery, Snowflake, MS Fabric. |
URL¶
By selecting the URL type, a resource can be referenced via certain protocols depending on the syntax entered in the URL field.
Default Endpoint¶
It instructs the system that files produced by CCH Tagetik tools (such as Application Import/Export) that require prolonged storage, must be saved to the remote URL reported by the Endpoint. If this option is not enabled for any Endpoint, files produced by CCH Tagetik tools will be saved to the directory set as writable folder.l This option can be enabled on one Endpoint only.
Managed protocols¶
Below is a list of managed protocols, with relevant examples.
| Protocol | Windows URL examples | Unix URL examples |
|---|---|---|
| File | - file://localhost/c | /WINDOWS/ - file:///c |
| Ftp | ftp://host[:port]/url-path | |
| Ftp (secure) | ftps://host[:port]/url-path | |
| Http | http://webdav/files | |
| Https | https://webdav/files | |
| Sftp | sftp://host[:port]/url-path | |
| Smb | smb://host/shared-folder-path |
SFTP protocol¶
The 'Key Exchange' (the process of negotiating the secret key used to encrypt data during the session) used by CCH Tagetik is the'diffie-hellman-group-exchange-sha256'.
The 'host key' algorithms (algorithms used during key exchange to verify that the client is actually communicating with the correct server) managed by CCH Tagetik are the following:ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256,ssh-rsa
Note: the ossh-rsa algorithm is a weak algorithm and its use is not recommended (this algorithm has been deprecated by OpenSSH)
Tagetik Application¶
By selecting the Tagetik Application type, a Tagetik application can be referenced by its specific data source.
The configuration requires the following fields:
| Field | Description |
|---|---|
| Tagetik Application | |
| URL | Tagetik application URL |
| Application datasource | Identifier of a datasource among those defined for the Tagetik application |
Tagetik Data Loader¶
By selecting the Tagetik Data Loader type, it is possible to refer a Tagetik Data Loader previously registered on CCH Tagetik.
Configuration requires:
| Field | Description |
|---|---|
| Tagetik Data Loader | |
| Tagetik Data Loader Name | to be selected from those defined. |
| Tagetik Data Loader source type | the type of source form which data must be extracted/uploaded, either From database or From file |
Note: Once the source type from which data must be extracted/uploaded has been specified, clicking the Update Tagetik Data Loader data button will update the list of datasources or paths configured on the specified Tagetik Data Loader.
Database¶
Selecting the From database type tells Tagetik Data Loader to use the databases configured on it as a resource.
Configuration requires:
| Field | Description |
|---|---|
| Tagetik Data Loader Datasource | the logical name of the datasource from which data must be extracted/uploaded |
File¶
Selecting the From file type tells Tagetik Data Loader to use the repositories configured on it as a resource.
Configuration requires:
| Field | Description |
|---|---|
| Tagetik Data Loader path | the logical name of the path on which the application can run read/write operations. |
OData service¶
By selecting the OData V4 service or OData V2 service type, an OData service can be referenced for reading collections of entities (the EntitySets defined in the service root’s Entity Data Model). For the OData V4 service type, it is also possible to refer a service for reading collections of entities from functions (FunctionImport).
Configuration requires:
| Field | Description |
|---|---|
| URL | the OData service root URL |
| Use direct connection (only for instances in the same region) | It allows the use of a direct connection for saastagetik services in the same cloud region. The flag is only present if: - it is a cloud environment - the system property 'it.grupposervizi.easy.api.internal.host' is available - the url host ends with ".saastagetik.com" . Under the conditions described above, by default, the flag is enabled |
| Pagination size | Maximum number of records in each data packet to be loaded to CCH Tagetik. - > 0: packet-based reading enabled - 0: reads all data simultaneously |
In the event that large amounts of data are to be transferred and a significant memory occupancy is observed on the CCH Tagetik application server side, it is possible to enable the transmission of packet data by setting the pagination size property to a value >0.
The system handles any server-side pagination of OData service (performed via the caption @odata.nextLink) even if the data packet transmission is enabled on CCH Tagetik.
REST API¶
By selecting the REST API type, it is possible to refer to a service that expose data via RESTfull API.
Configuration requires:
| Field | Description |
|---|---|
| URL | the REST API root URL |
| Use direct connection (only for instances in the same region) | It allows the use of a direct connection for saastagetik services in the same cloud region. The flag is only present if: - it is a cloud environment - the system property 'it.grupposervizi.easy.api.internal.host' is available - the url host ends with ".saastagetik.com" . Under the conditions described above, by default, the flag is enabled |
| Response type | Response format (JSON and CSV). |
| JSON: | |
| Path | The path to the object containing the list of data to read, using the JSONPath syntax. |
| Managed array level | The number of nested arrays to be included in the result. |
| CSV: | |
| Header | Indicates the presence of the header row |
| Columns separator | Indicates the separator character of the columns: "," - ";" - " |
| Value delimiter mode | Indicates the value delimitation mode: None, Minimal, Non Numeric, All |
| Value delimiter | Specifies the values delimiter: None, ", ' |
| Row separator | Specifies the row separator character: Carriage return + Line feed (\r\n), Carriage return (\r), Line feed (\n) |
| Pagination | |
| Usage | Indicates where the pagination property is to be specified: None, Header, Query param |
| Property name | Indicates the name of the property that manages the pagination size. |
| Size | Specifies the pagination size |
| Next page | |
| Position in response | Indicates where to retrieve the property defined in the response to access the next page: None, Header, Body attribute |
| Property name | Indicates the name of the property in the response that contains the information for retrieving the next page. If the selected position is Body attribute, the path must be indicated using the JSONPath syntax. |
| Position in request | Indicates where the property should be specified in the request to access the next page: None, Header, Query param, Complete URL |
| Property name | Indicates the name of the property to be specified in the request that contains the information for retrieving the next page. |
Example - JSON¶
Endpoint configuration:
URL: https://company.com/api/employees
Response type: JSON
Path: data.employees
Managed array level: 0
Pagination:
Usage: Query param
Property name: maxSize
Size: 3
Next page:
Position in response: Body attribute
Property name: nextPage
Position in request: Query param
Property name: next
The first request made by the system is: https://company.com/api/employees?maxSize=3
the returned body is:
{ "data":{ "employees":[ {"name":"Shyam", "email":"shyamjaiswal@gmail.com", "role":{"name":"Developer", "level":"Senior"}, "qualifications":[{"id": "AAA","details":{"id":"001", "list":["A01","A02"]}}]}, {"name":"Bob", "email":"bob32@gmail.com", "role":{"name":"Developer", "level":"Junior"}, "qualifications":[{"id": "BBB","details":{"id":"002", "list":["B01"]}}]}, {"name":"Jai", "email":"jai87@gmail.com", "role":{"name":"Manager", "level":"Senior"}, "qualifications":[{"id": "CCC","details":{"id":"003", "list":["C01","C02"]}}]} ] }, "nextPage":"adWOdawe1" }
La seconda richiesta effettuata dal sistema sarà: https://company.com/api/employees?maxSize=3&next=adWOdawe1
The attributes of each employee will be read and the compound objects deployed:
- "name"
- "email"
- "role.name"
- "role.level"
whereas arrays ("qualifications") will be ignored
If the Managed Array Level attribute is set to 1, then the qualifications. id and qualifications.details.id fields will also be added
If the Managed Array Level attribute is set to 2, then qualifications. id, qualifications.details.id and qualifications.details.list fields will also be added
Note: The handling of nested arrays results in the production of more records in the result rows, as they are obtained via Cartesian product.
Example - CSV¶
Endpoint configuration:
URL: https://company.com/api/employees
Response type: CSV
Header: true
Column separator: ,
Value delimiter mode: All
Value delimiter: '
Row separator: Carriage return + Line feed (\r\n)
Pagination:
Usage: Header
Property name: maxSize
Size: 3
Next page:
Position in response: Header
Property name: nextPage
Position in request: Complete URL
Property name:
The first request made by the system is: https://company.com/api/employees con Header: maxSize: 3
the returned body is:
"name","email","role.name","role.level"\r\n "Shyam","shyamjaiswal@gmail.com","Developer","Senior"\r\n "Bob","bob32@gmail.com","Developer","Junior"\r\n "Jai","jai87@gmail.com","Manager","Senior"\r\n
The returned headers are:
Connection: keep-alive Content-Type: text/csv ... nextPage: https://company.com/api/employees?next=adWOdawe1
The second request made by the system will be: https://company.com/api/employees?next=adWOdawe1 con Header: maxSize: 3
SharePoint¶
By selecting the SharePoint type, a SharePoint Site can be referenced. Configuration requires:
| Field | Description |
| Tenant id | SharePoint tenant id |
| Client id | SharePoint application client id |
| Site name | Name of the site on SharePoint. The name is case sensitive and must respect spaces, upper and lower case. |
| Tenant name | (optional) Name of the tenant retrieved from the SharePoint URL: |
Note: The following requirements must be met in order to connect to SharePoint via CCH Tagetik:
-
The minimum permissions (Microsoft Graph scope) associated with the configured client must be:
-
If the 'tenant name' field is filled: Site.Selected
- If the 'tenant name' field is not filled: Site.FullControl.All
- The configured permissions must be of 'Application' type
- Authentication can be by secret or by certificate
For further details on SharePoint configuration, see Microsoft's official documentation
Google Big Query¶
By selecting the Google Big Query type, a Google Big Query project can be referenced.
Configuration requires:
| Field | Description |
| Project Id | Resource project identifier on Google Cloud Platform |
Snowflake¶
By selecting the Snowflake type, a Snowflake database can be referenced.
Configuration requires:
| Field | Description |
| URL | The url to reach Snowflake's service. It must have the following format: https:// |
| User | User identifier Snowflake |
| Warehouse | Identifier of the Warehouse on which the resource is located |
| Database | Identifier of the Database on which the resource is located |
| Scheme | Identifier of the Scheme on which the resource is located |
| Role | Role with which the Snowflake user accesses resources |
MS Fabric¶
By selecting the MS Fabric type, an MS Fabric database can be referenced
Configuration requires:
| Field | Description |
|---|---|
| SQL Endpoint | The SQL Endpoint can be retrieved directly from the settings of the Warehouse Fabric to be managed. To be reported exactly as provided by Fabric. |
| Warehouse | The name of the Warehouse Fabric. |
| Tenant Id | Id of the tenant for which the Service Principal has been defined added to the Workspace Fabric to which the Warehouse to be managed belongs. |
Note: In order to connect to MS Fabric via CCH Tagetik, a Service Principal meeting the following requirements is required:
- Access to the Workspace where the Warehouse to be used is located, with authorisation level at least equal to Contributor
- Secret‑only authentication.
For more details on MS Fabric configuration, please refer to the official Microsoft documentation.
Pretty Good Privacy (PGP) encryption¶
If the URL Endpoints and Tagetik Data Loader use PGP encryption, it will be possible to configure its features in the Data Encryption section.
The Enable PGP option allows enabling the PGP encryption so that files are decrypted when read and encrypted when written using this standard.
Requested information:
| Field | Description |
| Public key | Public key of the key pair generated by the client |
| Secret key | Private key of the key pair generated by CCH Tagetik |
| Secret passphrase | Passphrase used to generate CCH Tagetik keys (Field value is hidden) |
To successfully upload an encrypted file, it is necessary to comply with certain restrictions:
- no signature must be applied after encryption;
- the encrypted file must contain one recipient only.
If the endpoint is only used to read data, only the secret key and secret passphrase pair needs to be configured; if it is only used to write data, only the public key needs to be configured.
Authentication¶
If the service provides an authentication mode, it is possible to select a type from the Available authentication types field in the Authentication tab.
The following types of authentication are currently supported:
| Field | Description |
|---|---|
| None | |
| Valid for all types of Endpoints, when the remote resource does not require any authentication type | |
| Basic Authentication | |
| Valid for URL type Endpoints (not with "file"), OData service type and REST APIs, when the remote resource requires requires an authentication consisting of user and password. For URL-type endpoints with the 'smb' protocol, the username must contain the domain reference (e.g. |
|
| Username | identifies the user name that can access the requested resource |
| Password | identifies the password associated with the user name defined at the previous point |
| OAuth 2 | |
| Valid for Endpoints of type OData service, REST API, and Snowflake, it allows you to configure, using the OAuth2 standard, the issuing of an access token by an authorization server to a third‑party client, subject to the approval of the user who owns the resource being accessed. See OAuth website | |
| Grant type | It can be: Client credentials or Password credentials |
| Client Authentication | Indicates whether to send authentication in the header, body (with Client credentials or Password credentials) or authenticate with a signed JWT with a private key (Client credentials only) |
| Token Endpoint | The url to obtain the token |
| Client Id | OAuth2 Client Id |
| Client Secret | OAuth2 Client Secret |
| Username | Username for flow with password credentials |
| Password | Password for flow with password credentials |
| Kid | Represents the key identifier used if you have multiple keys on the authentication server to obtain a signed JWT with a private key |
| Lifetime | Indicates the time, in seconds, to be added to the current time for configuring the EXP field of the JWT token. Possible values are those in the range [1 - 3600]. |
| Private key in PEM format | File containing the private key in PEM format to obtain a signed JWT with a private key. The private key must be generated with the RS256 algorithm in PEM format without passphrase and must have this format: -----BEGIN PRIVATE KEY----- MII... -----END PRIVATE KEY---- The claims included in the JWT token are: iss, sub, aud, exp, iat, jti (see RFC7523). |
| Scopes | OAuth2 Scopes |
| Resource | OAuth2 Resource |
| Use proxy configuration to obtain the token | Indicates whether the proxy configuration is required to reach the url for obtaining the token |
| Use configured certificates to obtain the token | Indicates whether authentication via certificates is required to reach the url to obtain the token |
| Key Pair | |
| Only valid for Snowflake endpoints. It allows you to configure authentication using your own private key. | |
| Account Identifier | Snowflake account identifier |
| Keyfile | Account's private key file |
| Passphrase | (optional) Passphrase for opening the private key |
| Fingerprint public key | Fingerprint of the associated public key on Snowflake. The fingerprint must be in the format SHA256: |
| Json Key | |
| Only valid for endpoints of type BigQuery. Allows you to configure the ADC key used for the connection | |
| ADC Json Key | ADC key in Json format of the service account used for connection (after saving, the private key field is obscured) |
| Underlying | |
| Valid only for endpoints of BigQuery type. It allows you to connect to Google BigQuery using the Google Cloud environment on which CCH Tagetik is installed. IMPORTANT: beta functionality must be enabled to use this authentication mode. |
Authentication tab for endpoint of SharePoint type
Please find below the most important fields.
| Field | Description |
|---|---|
| Scope | SharePoint Permissions Scope. If not specified, the default value is used: https://graph.microsoft.com/.default |
| Available authentication types | Can be: Via Secretor Via Certificate. Via Secret - Secret value: secret client value Via Certificate, it provides the following options: Certificate Type: can be: PEM (.cer .crt .pem .key) or PKCS#12 (.pfx .p12) Certificate file: contains the certificate in PEM or PKCS#12 format Key File: contains the key of the certificate (the field is only present if the certificate type is PEM) Passphrase: used to generate the certificate (the field is only present if the certificate type is PKCS#12) |
Authentication tab for MS Fabric endpoints
| Field | Description |
|---|---|
| Client Id | Client Attribute Id of the Azure Application defined for the Main Service |
| Secret | Client Secret value of Application Azure (Service Principal) |
Proxy¶
If the Endpoint OData service, REST API and Snowflake are exposed via a Proxy server, this can be configured in the Proxy section.
| Field | Description |
|---|---|
| Server | Proxy server identifier |
| Port | Proxy server port |
| User | Username for authentication on the proxy server |
| Password | Password for authentication on the proxy server |
Custom headers¶
If the Endpoint OData service, REST API and Snowflake require additional headers to be sent in the request, these can be configured in the Headers section.
| Field | Description |
|---|---|
| Key | Key of the custom header |
| Value | Value of the custom header |
Certificates¶
If the OData service and REST API Endpoints require authentication via certificates, these can be configured in the Certificates section. This section is only visible if the protocol used in the url is https.
| Field | Description |
|---|---|
| Certificate Authority | |
| CA file | File containing the CA certificate in PEM format |
| Client | |
| Certificate Type | It can be: PEM (.cer .crt .pem .key) or PKCS#12 (.pfx .p12) |
| Certificate file | File containing the certificate in PEM or PKCS#12 format |
| Key file | File containing the certificate key in PEM format (the field is only present if the certificate type is PEM) |
| Passphrase | Passphrase used to generate the certificate. (the value of the field is hidden) |
Advanced¶
For some endpoint types, the following additional connection parameters can be set
| Field | Description |
|---|---|
| Endpoint type: Snowflake | |
| Query timeout | Setting for Snowflake query timeout (in seconds). This option refers to the processing time of the query by Snowflake. |
| Type of Endpoint: Google Big Query | |
| Use target table | Use a target table for the query result. IMPORTANT: in case the compressed outcome of the query exceeds 10GB, this setting is mandatory) |
| Use existing Project Id | For the definition of the target table, use the Project Id specified in the attributes of the Big Query endpoint |
| Project Id | Definition of the Project Id of the target table. If it has been set that the existing Project Id must be used, the field is not editable. |
| Dataset name | Definition of the name of the target table dataset |
| Table Name | Definition of the name of the target table |
| Write mode | Defines how data is written to the temporary table according to Google BigQuery : - Write Truncate: If the table already exists, BigQuery overwrites data, removes restrictions and uses the scheme given by the query. - Write Truncate Data: If the table already exists, BigQuery overwrites data, but keeps the restrictions and the scheme of the existing table. - Write Append: If the table already exists, BigQuery adds new data to the table and returns the results of the table. - Write Empty: If the table already exists and contains data, the data processing returns an error of 'duplicate' type. |
Connection¶
For each Endpoint, in the Connection section, the Test button can be used to check the validity of the entered information and test the connection to the remote resource. At the end of the test, a message containing details on the test result is displayed in the lower area.