ansible-role-infosvr-metadata-asset-manager

This Ansible role helps automate the deployment of metadata using IBM Metadata Asset Manager brokers and bridges.

Are you new to Ansible? Check out this simple introduction.

Requirements

• Ansible version 2.4.x
• Network access to an IBM Information Server environment with 'dsadm' permissions
• Inventory group names need to be set up like the IBM.infosvr role

Role Variables

Refer to defaults/main.yml for documentation and the example below for key variables.

Each broker and bridge has its own specific parameters. These are detailed in the YAML files under vars/, while the minimum requirements for all brokers and bridges are in vars/simple_examples.yml. We aim to eventually support more brokers and bridges, but only those in vars/ for this version should be expected to work.

Example Playbook

This role is meant to be included in other playbooks for metadata deployment through supported brokers and bridges in the Information Server environment. (This is why Ansible v2.4.x is required along with the import_role module.)

This example will:

1. Add DB2 drivers from Information Server to the JDBC configuration.
2. Create a data connection named AutoJDBC (if it doesn't already exist) to connect to a DB2 database called MYDB at myhost.somewhere.com. (requires Information Server v11.7+)
3. Create an import area and run the import for metadata from any files in /data/loadable on the Information Server engine tier (noting that they are found on 'IS-SERVER.IBM.COM').
4. For the DB2INST1 schema of MYDB on the AutoJDBC connection, automatically import any metadata (e.g., tables, columns), perform column analysis, detect term assignments, run data quality analysis, and publish the results to the Information Governance Catalog. (requires Information Server v11.7+)

Note that the order of variable definitions doesn't matter — the role manages execution order automatically to ensure dependencies are handled correctly (e.g., JDBC configuration is completed before data connections or imports).

- import_role: name=IBM.infosvr-metadata-asset-manager
  vars:
    import_areas:
      - name: Simple_LocalFileConnector_ImportArea
        type: LocalFileConnector
        description: "A simple sample (setting only required fields) for a LocalFileConnector import area"
        metadata_interchange_server: myhost.domain.com
        dcn:
          name: LOCALFS
        assets_to_import:
          - "folder[/data/loadable]"
        hostname: "IS-SERVER.IBM.COM"
    jdbc_entries:
      classpaths:
        - /opt/IBM/InformationServer/ASBNode/lib/java/db2jcc.jar
      classnames:
        - com.ibm.db2.jcc.DB2Driver
    data_connections:
      - name: AutoJDBC
        type: JDBCConnector
        description: Data connection for automatically discovering against a JDBC source
        url: jdbc:db2://myhost.somewhere.com:50000/MYDB
        username: db2inst1
        password: "{{ a_password_from_eg_vault }}"
    discover_sources:
      - dcn: AutoJDBC
        project: UGDefaultWorkspace
        target_host: myhost.somewhere.com
        steps:
          - import
          - columnAnalysis
          - termAssignment
          - dataQualityAnalysis
          - publish
        parameters:
          rootAssets: schema[MYDB|DB2INST1]
          Asset_description_already_exists: Replace_existing_description

Possible Variables

import_areas

Use this variable to list complex structures defining import areas for Metadata Asset Manager. If an import area doesn't exist, it will be created; if it already exists, its metadata will be re-imported.

Example structures are available under vars/documented_*.yml, and simpler examples can be found under vars/simple_examples.yml.

data_connections

For v11.7+ only, use this variable to define data connections instead of full import areas. This is useful for leveraging automated discovery capabilities introduced in v11.7+.

odbc_entries

This variable defines ODBC entries for the {DSHOME}/.odbc.ini file, which ensures ODBC connectivity (such as for DataStage and IMAM).

Required keys:

• name: unique name of the ODBC entry
• description: description of the ODBC entry (avoid using = character)
• type: type of ODBC entry (e.g., db2, oracle, etc.)
• database: name of the database (for RDBMS)
• host: hostname or IP address of the data source
• port: port number for the database

Extra properties can be added to an extras entry as key-value pairs.

Examples:

odbc_entries:
  - name: IADB on DB2
    description: Connection to IADB on DB2
    type: db2
    database: IADB
    host: infosvr.vagrant.ibm.com
  - name: Test database on Oracle
    description: Connection to some test data set on Oracle
    type: oracle
    database: TESTDB
    host: infosvr.vagrant.ibm.com
    SID: TESTDB
    extras:
      - QueryTimeout: -1
      - ColumnSizeAsCharacter: 1

jdbc_entries

Use this variable to add JDBC classes to the {DSHOME}/isjdbc.config file for connectivity through JDBC.

Two required sub-keys:

• classpaths: locations of Java classes to add to the CLASSPATH (JDBC drivers)
• classnames: names of the Java classes providing the JDBC drivers

Examples:

jdbc_entries:
  classpaths:
    - "{{ ibm_infosvr_metadata_asset_mgr_install_location }}/ASBNode/lib/java/db2jcc.jar"
  classnames:
    - com.ibm.db2.jcc.DB2Driver

osh_schemas

Use this variable to define DDL files for generating OSH schema files that mirror database tables.

Most parameters are required, except tables — if not specified, an OSH schema will be generated for every CREATE TABLE in the DDL.

This will create:

• A blank data file for each CREATE TABLE statement in the specified dest directory.
• An OSH schema file for each data file, appending .osh as the extension.

Examples:

osh_schemas:
  - ddl: /some/location/MYDB.sql
    structure: "file_format: 'delimited', header: 'false'"
    recordfmt: "delim='|', final_delim=end, null_field='', charset=UTF-8"
    dest: /some/target/directory
    fileext: csv
    tables:
      - TABLE1
      - TABLE2

discover_sources

This variable defines data sources for automatic discovery using the Open Discovery Framework in v11.7+.

Necessary sub-keys for each entry:

• dcn: data connection name to use for discovery
• project: name of the Information Analyzer project for the discovery
• target_host: hostname for the source being discovered

Optional sub-keys include:

• parameters: defines additional behaviors or restrictions for discovery.
• steps: specifies which discovery steps to apply, e.g., import, columnAnalysis, termAssignment, etc.

Examples:

discover_sources:
  - dcn: AutoJDBC
    project: UGDefaultWorkspace
    target_host: myhost.somewhere.com
    steps:
      - import
      - columnAnalysis
      - termAssignment
      - dataQualityAnalysis
      - publish
    parameters:
      rootAssets: schema[MYDB|DB2INST1]
      Asset_description_already_exists: Replace_existing_description

License

Apache 2.0

Author Information

Christopher Grote