Skip to content

Installation

To install the Tagetik Data Loader, simply unzip the file TagetikDataLoader_ApacheKaraf.zip into a directory (e.g. /TagetikDataLoader/).

Version 8 of the Tagetik Data Loader uses the latest version of the Apache Karaf container, 4.4.6

Inside are two directories:

  • apache-karaf-<version>: contains the Apache Karaf container, the Tagetik Data Loader jars and the default JDBC drivers. The directory can be renamed to apache-karaf.
  • tgk-configuration-<version>: contains the externalisation of the Karaf configuration (/etc) in which the configuration files of the Tagetik Data Loader are also located.

Configuration

Tagetik Data Loader (TDL) requires the configuration of a set of parameters necessary for it to function correctly and fully:

  • Java Home configuration
  • definition of the other Karaf settings (memory, etc.)
  • definition of Karaf configuration
  • general configuration and configuration of the connection to CCH Tagetik
  • authentication configuration
  • logs configuration
  • configuration of datasources for the databases access
  • configuration of paths for accessing file system directories

Java Home configuration

In order to configure Java Home, it is necessary to access the setenv file(setenv.bat if installation is carried out on Windows operating systems) located in /bin and define JAVA_HOME.

The JDK version to be used is 17.0.12. The Tagetik Data Loader 8 has been built and tested with JDK17.0.12 : upgrading to subsequent patches is not discouraged in principle, as they generally contain fixes that do not disrupt compatibility. However, it should be noted that the Tagetik Data Loader is not beforehand tested with every JDK patch beyond version 17.0.12. This occurs only in response to specific requests or reports of issues identified after a JDK update.

E.g. linux: export JAVA_HOME="/usr/lib/jvm/openjdk-17.0.12"

E.g. windows set JAVA_HOME=C:\Progra~1\Java\openjdk-17.0.12

List of tested and supported JDKs:

  • Amazon Corrected 17.0.12
  • Microsoft 17.0.12
  • Oracle 17.0.12
  • SapMachine 17.0.12
  • IBM Semeru 17.0.12
  • Liberica 17.0.12
  • Temurin 17.0.12

Definition of other Java and Karaf settings

To define the other Java and Karaf settings, it is necessary to access the setenv file (setenv.bat if installing on Windows operating systems) located in /bin and add the options. If Karaf is run via Windows service, after running the wrapper installation command, configure the wrapper.conf file available inside the configuration directory. If a custom name is specified inside the wrapper installation command, the file will be karaf-.conf

E.g. linux:

export JAVA_MIN_MEM=256M # Minimum memory for the JVM

export JAVA_MAX_MEM=1024M # Maximum memory for the JVM

export JAVA_PERM_MEM=128M # Minimum perm memory for the JVM

export JAVA_MAX_PERM_MEM=256M # Maximum memory for the JVM

E.g. windows

set JAVA_MIN_MEM=256M # Minimum memory for the JVM

set JAVA_MAX_MEM=1024M # Maximum memory for the JVM

set JAVA_PERM_MEM=128M # Minimum perm memory for the JVM

set JAVA_MAX_PERM_MEM=256M # Maximum memory for the JVM

Karaf configuration definition

It is recommended to externalise the Karaf configuration from the /etc folder to another directory so that all customisations can be managed when updating the Karaf container. Once outsourced, it is possible to set up the configuration directory on Karaf by accessing the setenv file (setenv.bat if installing on Windows operating systems) located in /bin and defining the KARAF_ETC.

E.g. linux: export KARAF_ETC="/home/user/TagetikDataLoader/tgk-configuration"

E.g. windows set KARAF_ETC=C:\Progra~1\TagetikDataLoader\tgk-configuration

General setting and connection to CCH Tagetik

To configure the TDL for both the connection to CCH Tagetik and to change certain parameters relating to its functionality, it is necessary to access the tgk.properties.cfg file in the Karaf configuration directory.

The parameters to be configured are:

  • tgk.agent.polling.interval. Indicates at what intervals (in seconds) the TDL verifies requests for transactions to be taken over. It is recommended not to set up a value lower than 30 seconds
  • tgk.agent.tmpfolder. Indicates the path reserved for the TDL to save temporary resources generated by packet data transmission. The parameter is mandatory (to avoid relying on temporary system folders).
  • tgk.agent.write.db.batchsize. Indicates the number of statements sent simultaneously to the database to be executed at write time (default 10000)
  • tgk.agent.read.db.fetchsize. Indicates the number of records to be retrieved simultaneously from the database being read (if not specified, the default value specified by the jdbc driver in use will be used)
  • tgk.endpoint.address. CCH Tagetik endpoint URL (e.g. https://alpha.saastagetik.com/prod)

If secure data transfer is active, i.e. the Tagetik Endpoint URL has https as protocol, in the absence of specific parameterisation, the TDL will use the default JDK trustor in use. Should you wish to specify a custom server certificate, you can do so via the following parameters:

  • tgk.server.ks.password. Password chosen when generating the server-side certificate. It must be written following the syntax specified in Security and encryption of sensitive data
  • tgk.server.ks.path. Complete path of the server-side certificate

If the communication between TDL and CCH Tagetik is managed by a proxy server, the following can also be configured:

  • tgk.proxy.server. Indicates the IP of the proxy server
  • tgk.proxy.port. Indicates the proxy server port
  • tgk.proxy.user. Indicates the user accessing the proxy server
  • tgk.proxy.password. Indicates the password with which the user defined in the previous property can access the proxy server. It must be written following the syntax specified in Security and encryption of sensitive data

Starting with Tagetik Data Loader version 7.0.0, it is possible to enable payload encryption of data exchanged between the TDL and CCH Tagetik via PGP algorithm.

To do this, it is necessary to configure:

  • tgk.pgp.publickey indicates the full path to the file containing the public key
  • tgk.pgp.privatekey indicates the full path to the file containing the private key
  • tgk.pgp.passphrase indicates the passphrase to use the private key. It must be written following the syntax specified in Security and encryption of sensitive data

If the Endpoint is only used to read data, only the public key needs to be configured; if it is only used to write data, only the secret key and secret passphrase need to be configured. If the TDL is used to write data, only the privatekey with the passphrase on the TDL and the publickey on the Endpoint defined on CCH Tagetik need to be configured.

Example 1 (HTTPS secure transfer without specified custom server certificates)

tgk.agent.pollingInterval=30

tgk.agent.tmpfolder=/home/user/TagetikDataLoader/tgk-configuration/tmp/

tgk.agent.write.db.batchsize=10000

tgk.agent.read.db.fetchsize=

tgk.endpoint.address=https://tagetik.installation.url.com/tagetikcpm

tgk.server.ks.path=

tgk.server.ks.password=

Example 1 (HTTPS secure transfer with specified custom server certificates)

tgk.agent.pollingInterval=30

tgk.agent.tmpfolder=/home/user/TagetikDataLoader/tgk-configuration/tmp/

tgk.agent.write.db.batchsize=10000

tgk.agent.read.db.fetchsize=

tgk.endpoint.address=https://tagetik.installation.url.com/tagetikcpm

tgk.server.ks.path=/home/TDL/certificates/server.truststore

tgk.server.ks.password={TGK_ENCRYPT}server_side_certificate_password{TGK_ENCRYPT}

Example 2 (HTTP transfer)

tgk.agent.pollingInterval=30

tgk.agent.tmpfolder=/home/user/TagetikDataLoader/tgk-configuration/tmp/

tgk.agent.write.db.batchsize=10000

tgk.agent.read.db.fetchsize=

tgk.endpoint.address=http://tagetik.installation.url.com/tagetikcpm

Example 3 (PGP transfer)

tgk.agent.pollingInterval=30

tgk.agent.tmpfolder=/home/user/TagetikDataLoader/tgk-configuration/tmp/

tgk.agent.write.db.batchsize=10000

tgk.agent.read.db.fetchsize=

tgk.endpoint.address=http://tagetik.installation.url.com/tagetikcpm

tgk.pgp.publickey=/home/user/TagetikDataLoade/publickey.asc

tgk.pgp.privatekey=/home/user/TagetikDataLoade/privatekey.asc

tgk.pgp.passphrase={TGK_ENCRYPT}123_pippo{TGK_ENCRYPT}

Authentication configuration

To enable the communication between TDL and CCH Tagetik, it is necessary to define the authentication parameters. They can be found in the file tgk.agent.cfg in the Karaf configuration directory.

The parameters are as follows:

  • tgk.agent.id. Unique ID associated with the TDL instance. It is automatically generated upon the first launch of Karaf.
  • tgk.agent.secret. SecretKey to authenticate the TDL on CCH Tagetik. It must be defined after the first launch of Karaf, once the ID has been automatically generated, and must be defined using the syntax specified in Security and encryption of sensitive data

With these two parameters it is possible to register the TDL on CCH Tagetik and authenticate it upon each request

Log configuration

The TDL log system is nothing more than a customization of the log system of Apache Karaf. The log configuration can be found in the file org.ops4j.pax.logging.cfg in the Karaf configuration directory and is a standard Log4j configuration file.

This file contains the following section with the specified values:

Tagetik agent appender

log4j2.logger.tagetik.name = com.tagetik.cpm

log4j2.logger.tagetik.level=INFO

log4j2.logger.tagetik.additivity=false

log4j2.logger.tagetik.appenderRef.TagetikRollingFile.ref = TagetikRollingFile

log4j2.appender.tagetik.type = RollingRandomAccessFile

log4j2.appender.tagetik.name = TagetikRollingFile

log4j2.appender.tagetik.fileName = ${karaf.data}/log/tagetik.log

log4j2.appender.tagetik.filePattern = ${karaf.data}/log/tagetik.log.%i

log4j2.appender.tagetik.append = true

log4j2.appender.tagetik.layout.type = PatternLayout

log4j2.appender.tagetik.layout.pattern = ${log4j2.pattern}

log4j2.appender.tagetik.policies.type = Policies

log4j2.appender.tagetik.policies.size.type = SizeBasedTriggeringPolicy

log4j2.appender.tagetik.policies.size.size = 16MB

For the meaning of the various options and of the values to be assigned, refer to the Log4j specifications.

Log files are saved by default in data/log/karaf.log

Based on the default settings above, the TDL log file is named tagetik.log and is generated on a daily basis. Moreover, every day the file generated on the previous day is renamed by adding a suffix given by the production date.

Datasource configuration

In case database operations are required such as:

  • reading data to be transferred to CCH Tagetik
  • writing of data extracted from CCH Tagetik

TDL requires .cfg files to be created and configured in the Karaf configuration directory.

Here are some templates of such files:

SqlServer (driver JTDS) - tgk.datasource-mssql-jtds.cfg**

osgi.jdbc.driver.class=net.sourceforge.jtds.jdbc.Driver

pool=dbcp2

xa=false

url=jdbc:jtds:sqlserver://:/

user=

password={TGK_ENCRYPT}{TGK_ENCRYPT}

dataSourceName=

vendor=tagetik

SqlServer (driver Microsoft) - tgk.datasource-mssql-microsoft.cfg**

osgi.jdbc.driver.class=com.microsoft.sqlserver.jdbc.SQLServerDriver

pool=dbcp2

xa=false

url=jdbc:jtds:sqlserver://:;databaseName=

user=

password={TGK_ENCRYPT}{TGK_ENCRYPT}

dataSourceName=

vendor=tagetik

Oracle - tgk.datasource-oracle.cfg**

osgi.jdbc.driver.class=oracle.jdbc.OracleDriver

pool=dbcp2

xa=false

url=jdbc:oracle:thin:::

user=

password={TGK_ENCRYPT}{TGK_ENCRYPT}

dataSourceName=

vendor=tagetik

PostgreSQL - tgk.datasource-postgresql.cfg**

osgi.jdbc.driver.class=org.postgresql.Driver

pool=dbcp2

xa=false

url=jdbc:postgresql://:/

user=

password={TGK_ENCRYPT}{TGK_ENCRYPT}

dataSourceName=

vendor=tagetik

SAP Hana - tgk.datasource-saphana.cfg**

osgi.jdbc.driver.class=com.sap.db.jdbc.Driver

pool=dbcp2

xa=false

url=jdbc:sap://:/?databaseName=

user=

password={TGK_ENCRYPT}{TGK_ENCRYPT}

dataSourceName=

vendor=tagetik

IBM DB2 - tgk.datasource-ibmdb2.cfg**

osgi.jdbc.driver.class=com.ibm.db2.jcc.DB2Driver

pool=dbcp2

xa=false

url=jdbc:db2://:/:retrieveMessagesFromServerOnGetMessage=true;

user=

password={TGK_ENCRYPT}{TGK_ENCRYPT}

dataSourceName=

vendor=tagetik

Netsuite - tgk.datasource-netsuite.cfg**

osgi.jdbc.driver.class=com.netsuite.jdbc.openaccess.OpenAccessDriver

pool=dbcp2

xa=false

url=jdbc:ns://:;ServerDataSource=NetSuite.com;encrypted=1;CustomProperties=(AccountID=;RoleID=)

user=

password={TGK_ENCRYPT}{TGK_ENCRYPT}

dataSourceName=

vendor=tagetik

The datasources passwords must be entered (both upon definition and edit) with the syntax specified in Security and encryption of sentitive data

Adding a custom datasource

From the Karaf command line via the command service:list DataSourceFactory it is possible to display the installed JDBC drivers.

To add a new JDBC driver it is necessary to:

  • copy the [driver_name].jar into the /lib/endorsed folder
  • run the install -s wrap:file:./lib/endorsed/[driver_name].jar command from the Karaf command line
  • add a .cfg file relating to the new datasource into the Karaf configuration directory (like those described above)
  • restart Karaf

The simultaneous presence of two JDBC drivers from the same vendor causes problems at runtime. If it is necessary to install a new JDBC driver (E.g. : a later version), the previous one has to be uninstalled.

Path configuration

In case operations are needed on files such as:

  • file transfer to CCH Tagetik
  • writing files produced by CCH Tagetik

TDL requires the tgk.repository.cfg file to be created and configured in the Karaf configuration directory.

Inside, a mapping list between the logical path name and the real absolute path can be configured.

The syntax is typical of the *.properties files, thus:

=

=

Security and encryption of sensitive data

For keystore and datasource passwords, there is a mechanism for encrypting the values.

To define the value of the password, the following syntax as to be used: ={TGK_ENCRYPT}{TGK_ENCRYPT}

After saving the configuration file, Tagetik Data Loader will automatically encrypt the value of the property and replace it in the same configuration file.

TDL can only handle passwords containing 'ASCII Printable Characters', i.e. ASCII characters 32 to 126 with the following exceptions:

  • The '$' character (ASCII 36) cannot be used as the last character of the password
  • The character '\' (ASCII 92) cannot be used in the password.

Starting Karaf

Once installation and configuration have been carried out, Tagetik Data Loader can be used by starting Karaf via the karaf command(karaf.bat if installation is carried out on Windows operating systems) present in /bin.

Starting as an operational system service

It is also possible to start Karaf as an OS service by executing the command wrapper:install --name "TDL" --display "TDL" --description "TDL" from the Karaf command line and afterwards:

  • for Windows: run from the command prompt the command TDL-service.bat install (or karaf-service.bat install), present in the /bin directory
  • for Linux: follow the instructions shown once you run the wrapper:install command .