Snowflake Connector for ServiceNow®V2 Migration Guide¶
The Snowflake connector for ServiceNow® is subject to the Connector Terms.
This topic describes how to migrate from the Snowflake Connector for ServiceNow® V1 to the V2 version such that previously ingested data is preserved and the V2 connector continues replicating incremental data changes from where older version finished.
Why upgrade?¶
The Snowflake Connector for ServiceNow®V2 is based on the latest Native Apps Framework, which is Generally Available on AWS and Azure. Due to underlying changes introduced in the V2 version of the connector you must reinstallation using a new listing based on this new version.
To preserve connector’s state and configuration, please follow this guide.
Before you begin¶
Review carefully the items below to ensure you are ready to safely reinstall the connector using the new version.
Deprecation notice¶
The Snowflake Connector for ServiceNow®V2 is generally available, as a result then Snowflake Connector for ServiceNow®, referred to here as V1, is now deprecated. If you wish to preserve your connector’s state and configuration, you must migrate on or before June 23rd, 2024. After this date the prior version of the connector will become unusable and data export will be no longer possible.
Important
Once the deprecated version of the connector is uninstalled from the account, it cannot be reinstalled.
Release notes¶
Please see release notes to see what has changed in the new version.
Underlying changes¶
The following changes have been made between the prior version and the current version of the connector:
Re-architected from Native Apps V1 framework to Native Apps V2.
Connector now updates automatically, and consequently the RUN_UPGRADE procedure has been removed.
Ingested data, including flattened views, is no longer owned by the data owner role, but rather by the application. From now on by default you will have access to
SELECT
privilege via DATA_READER application role. If additional privileges are needed by your data pipelines, they need to be granted explicitly to the data owner role before performing migration.Signatures of many views and procedures shared by the connector’s database have changed. Please review them before migration. If you were using them in custom scripts or data pipelines ensure that they are adjusted.
All of the public procedures return now data in the structured JSON format.
Dropped serverless support. Virtual warehouse must always be provided.
Performance of API calls to ServiceNow® has been improved:
Connector now supports ingesting large tables. Previously the limitation was 16 MB for an entire response. Now 16 MB per row.
Connector retry mechanism is more accurate which allows it to establish the correct page size for ingesting given table quicker.
Review the known limitations section of the new connector version. In particular changes:
connector does not support
MANAGED ACCESS
destination schemas.it’s not possible to execute certain procedures via user-created tasks.
The current version of the connector also includes several new features:
Completely new UI experience and installation flow.
Better manageability due to new application roles mechanism giving the user access to
ADMIN
,VIEWER
, andDATA_READER
roles.Connector’s state and configuration is now regularly saved in the
__EXPORT_CONNECTOR_STATE
table.Connector supports vanity URLs for ServiceNow® instance when installing from worksheets.
How to upgrade to Snowflake Connector for ServiceNow®V2¶
This section explains how to safely migrate your data and configuration to the Snowflake Connector for ServiceNow®V2.
Upgrade your current connector instance¶
We require your connector user version 4.21.0, or later. To check your app version, open Snowsight, create an SQL worksheet and, from the context of the connector database, run:
CALL <connector_v1_app>.PUBLIC.GET_VERSION();
If your version is 4.21.0 or higher, you can skip this step. Otherwise, run:
CALL <connector_v1_app>.PUBLIC.RUN_UPGRADE();
Attention
It’s recommended that you upgrade to the latest available 4.x.x version. This ensures that the most recent changes apply to your connector’s state.
Export your data¶
The procedure creates a new __CONNECTOR_STATE_EXPORT
table in the destination schema with an exported state. To perform an
export the following conditions must be met:
the connector must be stopped.
there is no ongoing table reload.
the
__CONNECTOR_STATE_EXPORT
table must not exist in the destination schema. If it exists, before performing the export, the table must be dropped or renamed.
Note
The __CONNECTOR_STATE_EXPORT
table contains all information necessary to restore the connector state during reinstallation,
but the following information is missing and should be noted:
The destination database and schema, warehouse, Data Reader role, ServiceNow URL, secret object, API Integration, and name of journal table (if configured) aren’t exported. This information must be provided when reinstalling the connector.
For each ingested table only the newest ingestion state is exported. As a result after the connector state import, historical data and statistics won’t be available.
Determine which table reloads are running by executing the command:
SELECT * FROM ENABLED_TABLES WHERE RELOADING = TRUE;
This command identifies all the tables in the reloading state. You can choose whether to wait until the reloads are completed or cancel them by running:
CALL CANCEL_RELOAD('<table_name>');
Note
If desired, you will need to reload the table again after reinstalling the connector in the new version. Data concerning table reloads is not exported.
Stop the connector by running:
CALL STOP_CONNECTOR();
Connector state and configuration can be exported only while the connector is stopped. Don’t resume the connector later, as this ensures that no two connectors are running simultaneously, especially if the destination database and schema are going to be reused in the new installation. Stopping the connector will protect you from access failures and additional cost as a result of running two connectors.
Export the connector’s data. The data can be exported with the following procedure:
CALL EXPORT_CONNECTOR_STATE();
Once configuration is exported it’s no longer updated. After each change in configuration or before connector reinstallation we recommend dropping the
__CONNECTOR_STATE_EXPORT
table and re exporting the state.
At this point __CONNECTOR_STATE_EXPORT
table containing the connector’s state and configuration should be visible in the destination schema.
Install Snowflake Connector for ServiceNow®V2¶
This section focuses on providing the same configuration values used in the previous connector install.
For more information see Installing and configuring the connector with Snowsight in the Snowflake Connector for ServiceNow®V2 documentation.
Ensure that the currently installed connector V1 instance is stopped.
(Optionally) Save currently used configuration values required in the later steps (and are not exported). To retrieve
connector configuration values (Journal Table, Data Owner Role, Warehouse, Destination Database, Destination Schema, ServiceNow® instance), run:
SELECT * FROM GLOBAL_CONFIG WHERE KEY IN ('journal_table_name', 'data_owner_role', 'warehouse_name', 'destination_db_and_schema', 'connection_config');
ServiceNow® username that was used to configure the connector, run:
SELECT value:secret FROM GLOBAL_CONFIG WHERE KEY = 'connection_config'; DESCRIBE SECRET <secret_value>;
Note
Snowflake never returns the
PASSWORD
property value.
In the navigation menu, select Data Products » Marketplace and search for “Snowflake Connector for ServiceNow®V2”. Make sure that the found listing is GZSTZTP0KL1 (this value can be checked in the URL).
In the page for the Snowflake Connector for ServiceNow®V2, select Get.
The
Installation
dialog displays and you can begin the installation process.Under the Options accordion configure the following:
Application name - change the default name of the connector.
Note
The new connector’s name must be different to the prior name. If you were using the default value for connector name which is
SNOWFLAKE_CONNECTOR_FOR_SERVICENOW
, you will must name the new connector something different. Remember to adjust the value in your custom scripts or data pipelines.Warehouse used for installation - select the warehouse that you want to use for installing the connector.
Note
This is not the same warehouse that is used by the connector to synchronize data from ServiceNow®. In a later step, you will provide a separate warehouse for this purpose.
Select Get.
The Installing App
dialog appears and when complete the title change to Successfully Installed
.
To continue configuration, select Configure.
When complete the dialog closes, and the Snowflake Connector for ServiceNow®V2 page displays the UI for configuring and managing the connector.
Note
You can close the installation dialog box prematurely and come back later when the installation process has completed. Additionally, an email will be send to inform you about the installation status.
Connector configuration
Configure
In the
Configure
dialog, specify the following:for Warehouse, Destination Database and Destination Schema use Select existing option and provide exactly the same values as were used in the previous version of the connector.
for Role choose existing role that was used as a data owner role in connector V1. This role will be granted DATA_READER application role.
Note
If you relied on additional roles in your data pipelines you can:
switch your data pipelines to rely on this single role that will serve as DATA_READER instead (recommended),
manually grant DATA_READER application role to these additional roles. See how to grant application roles.
Select Configure.
Authentication
The following procedure describes how to set up a connection to ServiceNow. You can select either basic authentication (username and password) or OAuth.
Select the previously chosen method: Basic authentication or OAuth2.
Basic authentication flow
For ServiceNow Instance, ServiceNow Username and ServiceNow Password fields provide exactly the same values that were used previously.
Select Connect.
OAuth flow
For ServiceNow Instance field, enter the same value that was used previously.
This is the first part of the hostname of your ServiceNow® instance. For example, if the URL to your ServiceNow® instance is:
https://myinstance.service-now.com
The name of your instance would be
myinstance
.Reuse the application registry:
Login to your ServiceNow® instance, then select Homepage.
Search for System OAuth, then select Application Registry.
Select the previously used registry and copy values of Client ID and Client Secret into the corresponding fields in Snowflake.
Select Connect.
A dialog appears asking you to login to your ServiceNow® instance with User name and Password. Provide the credentials of the user you want the connector to authenticate with - it needs to have the privileges listed in Preparing your ServiceNow® instance.
Once logged in, confirm that you want to allow the connector to connect to your ServiceNow® account. Select Allow.
Note
If you were redirected directly to this dialog without needing to provide credentials, then you are already logged in to your ServiceNow® instance. Ensure you are logged in as the same user the connector should use and that user has the necessary privileges.
Note: The current logged-in user is shown in upper right corner of the dialog.
Validate source
In the Journal Table field, enter the same value that was previously used (if was configured, otherwise leave this field empty).
At this point you should have a successfully configured a new version of the connector which is ingesting data into the previously specified destination schema. The connector uses exported configurations such as what tables were enabled and continues ingesting data according to their previously configured schedules.
To validate that import was successful examine:
if previously enabled tables are still enabled and connector ingests data for them (available in the
CONFIGURED_TABLES
view),if they have correct ingestion schedules or page sizes (for example no default values) (available in the
CONFIGURED_TABLES
view),if connector’s default ingestion schedule or page size are persisted (available in the
CONNECTOR_CONFIGURATION
view).
For more information about available views, see About Monitoring the Connector.
Uninstall V1 connector¶
Warning
After completing this uninstall, you won’t be able to export the connector’s data again. Ensure that the new version works correctly before uninstalling the prior instance.
(Optional) During the installation of the v1 connector a number of objects were created that need to be dropped manually. These objects are (in parentheses there are names of the objects in case the installation was via Snowsight):
security integration (
SERVICENOW_GZSTZTP0KHD_INTEGRATION
),API integration (
SERVICENOW_GZSTZTP0KHD_API
),secret (
CONNECTORS_UI.SERVICENOW_GZSTZTP0KHD.SECRET
).
To find API integration and secret manually you can run the following query from the context of connector V1 database:
SELECT value:apiIntegrationName, value:secret FROM GLOBAL_CONFIG WHERE key = 'connection_config';
You can drop these objects safely after completing the Uninstall V1 connector step.
In the navigation menu, select Data » Databases and find the shared database that was used for V1 of the connector installation.
Important
Don’t confuse the connector’s installation database with the destination database. If the destination database and schema were reused in the V2 installation, they must not be dropped.
In the top-right corner. select the three-dot menu and press Drop. After confirmation, the deprecated version of the connector will be removed.
At this point only one version of the connector should be installed.
Troubleshooting¶
If you face any issues while performing the migration, please submit a case to Snowflake Support. For currently known issues with the migration please see this community article.