Snowflake Connector for ServiceNow®V2 Migration Guide

The Snowflake connector for ServiceNow® is subject to the Connector Terms.

This topic describes how to migrate from the Snowflake Connector for ServiceNow® V1 to the V2 version such that previously ingested data is preserved and the V2 connector continues replicating incremental data changes from where older version finished.

Why upgrade?

The Snowflake Connector for ServiceNow®V2 is based on the latest Native Apps Framework, which is Generally Available on AWS and Azure. Due to underlying changes introduced in the V2 version of the connector you must reinstallation using a new listing based on this new version.

To preserve connector’s state and configuration, please follow this guide.

Before you begin

Review carefully the items below to ensure you are ready to safely reinstall the connector using the new version.

Deprecation notice

The Snowflake Connector for ServiceNow®V2 is generally available, as a result then Snowflake Connector for ServiceNow®, referred to here as V1, is now deprecated. If you wish to preserve your connector’s state and configuration, you must migrate on or before June 23rd, 2024. After this date the prior version of the connector will become unusable and data export will be no longer possible.

Important

Once the deprecated version of the connector is uninstalled from the account, it cannot be reinstalled.

Release notes

Please see release notes to see what has changed in the new version.

Underlying changes

The following changes have been made between the prior version and the current version of the connector:

  • Re-architected from Native Apps V1 framework to Native Apps V2.

  • Connector now updates automatically, and consequently the RUN_UPGRADE procedure has been removed.

  • Ingested data, including flattened views, is no longer owned by the data owner role, but rather by the application. From now on by default you will have access to SELECT privilege via DATA_READER application role. If additional privileges are needed by your data pipelines, they need to be granted explicitly to the data owner role before performing migration.

  • Signatures of many views and procedures shared by the connector’s database have changed. Please review them before migration. If you were using them in custom scripts or data pipelines ensure that they are adjusted.

  • All of the public procedures return now data in the structured JSON format.

  • Dropped serverless support. Virtual warehouse must always be provided.

  • Performance of API calls to ServiceNow® has been improved:

    • Connector now supports ingesting large tables. Previously the limitation was 16 MB for an entire response. Now 16 MB per row.

    • Connector retry mechanism is more accurate which allows it to establish the correct page size for ingesting given table quicker.

  • Review the known limitations section of the new connector version. In particular changes:

    • connector does not support MANAGED ACCESS destination schemas.

    • it’s not possible to execute certain procedures via user-created tasks.

The current version of the connector also includes several new features:

  • Completely new UI experience and installation flow.

  • Better manageability due to new application roles mechanism giving the user access to ADMIN, VIEWER, and DATA_READER roles.

  • Connector’s state and configuration is now regularly saved in the __EXPORT_CONNECTOR_STATE table.

  • Connector supports vanity URLs for ServiceNow® instance when installing from worksheets.

How to upgrade to Snowflake Connector for ServiceNow®V2

This section explains how to safely migrate your data and configuration to the Snowflake Connector for ServiceNow®V2.

Upgrade your current connector instance

We require your connector user version 4.21.0, or later. To check your app version, open Snowsight, create an SQL worksheet and, from the context of the connector database, run:

CALL <connector_v1_app>.PUBLIC.GET_VERSION();
Copy

If your version is 4.21.0 or higher, you can skip this step. Otherwise, run:

CALL <connector_v1_app>.PUBLIC.RUN_UPGRADE();
Copy

Attention

It’s recommended that you upgrade to the latest available 4.x.x version. This ensures that the most recent changes apply to your connector’s state.

Export your data

The procedure creates a new __CONNECTOR_STATE_EXPORT table in the destination schema with an exported state. To perform an export the following conditions must be met:

  • the connector must be stopped.

  • there is no ongoing table reload.

  • the __CONNECTOR_STATE_EXPORT table must not exist in the destination schema. If it exists, before performing the export, the table must be dropped or renamed.

Note

The __CONNECTOR_STATE_EXPORT table contains all information necessary to restore the connector state during reinstallation, but the following information is missing and should be noted:

  • The destination database and schema, warehouse, Data Reader role, ServiceNow URL, secret object, API Integration, and name of journal table (if configured) aren’t exported. This information must be provided when reinstalling the connector.

  • For each ingested table only the newest ingestion state is exported. As a result after the connector state import, historical data and statistics won’t be available.

  1. Determine which table reloads are running by executing the command:

    SELECT * FROM ENABLED_TABLES WHERE RELOADING = TRUE;
    
    Copy

    This command identifies all the tables in the reloading state. You can choose whether to wait until the reloads are completed or cancel them by running:

    CALL CANCEL_RELOAD('<table_name>');
    
    Copy

    Note

    If desired, you will need to reload the table again after reinstalling the connector in the new version. Data concerning table reloads is not exported.

  2. Stop the connector by running:

    CALL STOP_CONNECTOR();
    
    Copy

    Connector state and configuration can be exported only while the connector is stopped. Don’t resume the connector later, as this ensures that no two connectors are running simultaneously, especially if the destination database and schema are going to be reused in the new installation. Stopping the connector will protect you from access failures and additional cost as a result of running two connectors.

  3. Export the connector’s data. The data can be exported with the following procedure:

    CALL EXPORT_CONNECTOR_STATE();
    
    Copy

    Once configuration is exported it’s no longer updated. After each change in configuration or before connector reinstallation we recommend dropping the __CONNECTOR_STATE_EXPORT table and re exporting the state.

At this point __CONNECTOR_STATE_EXPORT table containing the connector’s state and configuration should be visible in the destination schema.

Install Snowflake Connector for ServiceNow®V2

This section focuses on providing the same configuration values used in the previous connector install.

For more information see Installing and configuring the connector with Snowsight in the Snowflake Connector for ServiceNow®V2 documentation.

  1. Ensure that the currently installed connector V1 instance is stopped.

  2. (Optionally) Save currently used configuration values required in the later steps (and are not exported). To retrieve

    • connector configuration values (Journal Table, Data Owner Role, Warehouse, Destination Database, Destination Schema, ServiceNow® instance), run:

      SELECT * FROM GLOBAL_CONFIG WHERE KEY IN ('journal_table_name', 'data_owner_role', 'warehouse_name', 'destination_db_and_schema', 'connection_config');
      
      Copy
    • ServiceNow® username that was used to configure the connector, run:

      SELECT value:secret FROM GLOBAL_CONFIG WHERE KEY = 'connection_config';
      DESCRIBE SECRET <secret_value>;
      
      Copy

      Note

      Snowflake never returns the PASSWORD property value.

  3. In the navigation menu, select Data Products » Marketplace and search for “Snowflake Connector for ServiceNow®V2”. Make sure that the found listing is GZSTZTP0KL1 (this value can be checked in the URL).

  4. In the page for the Snowflake Connector for ServiceNow®V2, select Get.

    The Installation dialog displays and you can begin the installation process.

    Under the Options accordion configure the following:

    • Application name - change the default name of the connector.

      Note

      The new connector’s name must be different to the prior name. If you were using the default value for connector name which is SNOWFLAKE_CONNECTOR_FOR_SERVICENOW, you will must name the new connector something different. Remember to adjust the value in your custom scripts or data pipelines.

    • Warehouse used for installation - select the warehouse that you want to use for installing the connector.

      Note

      This is not the same warehouse that is used by the connector to synchronize data from ServiceNow®. In a later step, you will provide a separate warehouse for this purpose.

    1. Select Get.

The Installing App dialog appears and when complete the title change to Successfully Installed.

To continue configuration, select Configure.

When complete the dialog closes, and the Snowflake Connector for ServiceNow®V2 page displays the UI for configuring and managing the connector.

Note

You can close the installation dialog box prematurely and come back later when the installation process has completed. Additionally, an email will be send to inform you about the installation status.

  1. Connector configuration

    1. Configure

      In the Configure dialog, specify the following:

      • for Warehouse, Destination Database and Destination Schema use Select existing option and provide exactly the same values as were used in the previous version of the connector.

      • for Role choose existing role that was used as a data owner role in connector V1. This role will be granted DATA_READER application role.

        Note

        If you relied on additional roles in your data pipelines you can:

        • switch your data pipelines to rely on this single role that will serve as DATA_READER instead (recommended),

        • manually grant DATA_READER application role to these additional roles. See how to grant application roles.

      Select Configure.

    2. Authentication

      The following procedure describes how to set up a connection to ServiceNow. You can select either basic authentication (username and password) or OAuth.

      1. Select the previously chosen method: Basic authentication or OAuth2.

      2. Basic authentication flow

        1. For ServiceNow Instance, ServiceNow Username and ServiceNow Password fields provide exactly the same values that were used previously.

        2. Select Connect.

      3. OAuth flow

        1. For ServiceNow Instance field, enter the same value that was used previously.

          This is the first part of the hostname of your ServiceNow® instance. For example, if the URL to your ServiceNow® instance is:

          https://myinstance.service-now.com
          
          Copy

          The name of your instance would be myinstance.

        2. Reuse the application registry:

          1. Login to your ServiceNow® instance, then select Homepage.

          2. Search for System OAuth, then select Application Registry.

          3. Select the previously used registry and copy values of Client ID and Client Secret into the corresponding fields in Snowflake.

          4. Select Connect.

          5. A dialog appears asking you to login to your ServiceNow® instance with User name and Password. Provide the credentials of the user you want the connector to authenticate with - it needs to have the privileges listed in Preparing your ServiceNow® instance.

          6. Once logged in, confirm that you want to allow the connector to connect to your ServiceNow® account. Select Allow.

            Note

            If you were redirected directly to this dialog without needing to provide credentials, then you are already logged in to your ServiceNow® instance. Ensure you are logged in as the same user the connector should use and that user has the necessary privileges.

            Note: The current logged-in user is shown in upper right corner of the dialog.

    1. Validate source

      • In the Journal Table field, enter the same value that was previously used (if was configured, otherwise leave this field empty).

At this point you should have a successfully configured a new version of the connector which is ingesting data into the previously specified destination schema. The connector uses exported configurations such as what tables were enabled and continues ingesting data according to their previously configured schedules.

To validate that import was successful examine:

  • if previously enabled tables are still enabled and connector ingests data for them (available in the CONFIGURED_TABLES view),

  • if they have correct ingestion schedules or page sizes (for example no default values) (available in the CONFIGURED_TABLES view),

  • if connector’s default ingestion schedule or page size are persisted (available in the CONNECTOR_CONFIGURATION view).

For more information about available views, see About Monitoring the Connector.

Uninstall V1 connector

Warning

After completing this uninstall, you won’t be able to export the connector’s data again. Ensure that the new version works correctly before uninstalling the prior instance.

  1. (Optional) During the installation of the v1 connector a number of objects were created that need to be dropped manually. These objects are (in parentheses there are names of the objects in case the installation was via Snowsight):

    • security integration (SERVICENOW_GZSTZTP0KHD_INTEGRATION),

    • API integration (SERVICENOW_GZSTZTP0KHD_API),

    • secret (CONNECTORS_UI.SERVICENOW_GZSTZTP0KHD.SECRET).

    To find API integration and secret manually you can run the following query from the context of connector V1 database:

    SELECT value:apiIntegrationName, value:secret FROM GLOBAL_CONFIG WHERE key = 'connection_config';
    
    Copy

    You can drop these objects safely after completing the Uninstall V1 connector step.

  2. In the navigation menu, select Data » Databases and find the shared database that was used for V1 of the connector installation.

    Important

    Don’t confuse the connector’s installation database with the destination database. If the destination database and schema were reused in the V2 installation, they must not be dropped.

  3. In the top-right corner. select the three-dot menu and press Drop. After confirmation, the deprecated version of the connector will be removed.

At this point only one version of the connector should be installed.

Troubleshooting

If you face any issues while performing the migration, please submit a case to Snowflake Support. For currently known issues with the migration please see this community article.