Cost Governance of the Snowflake Connector for ServiceNow®

The Snowflake connector for ServiceNow® is subject to the Connector Terms.

This topic provides best practices for cost governance and finding the optimal warehouse size for the Snowflake Connector for ServiceNow®.

Measuring Cost of the Connector

If the connector has a separate account only for data ingestion and storage, and the account shows no other activity (such as executing queries by users using the ingested data), you can read the overall cost on the account level. To learn more, refer to Exploring Overall Cost.

If the account is not dedicated only to the connector or you need to investigate the costs further, you should analyze the charged costs for the three components separately:

For an introduction to these three components of cost, refer to Understanding Overall Cost.

General Recommendations

To obtain cost generated by the connector, we recommend that you create a separate account solely for using the connector. Using a specific account you track the exact data transfer generated by the connector.

If you cannot use a separate account for the connector, consider the following:

  • Create a separate database for storing ingested data to track storage cost easier.

  • Allocate a warehouse only for the connector to get the exact compute cost.

  • Use object tags on databases and a warehouse to build custom cost reports.

  • If you use a serverless configuration, you can query the SERVERLESS_TASK_HISTORY view, filter on the name of the connector, and check the CREDITS_USED column to get the cost.

Compute Cost

We recommend that you create a dedicated warehouse only for the connector. This configuration allows you to create resource monitors on the warehouse. You can use the monitors to send email alerts and suspend the warehouse, stopping the connector when the set credit quota is exceeded. The connector automatically resumes after the credit quota is renewed. Note that setting credit quota too low in configurations where large volumes of data are ingested may cause the connector to not ingest all data. A major benefit is that the warehouse size can be adjusted to the data volume.

For information on how to check credits consumed by the warehouse, refer to Exploring Compute Cost. You can also assign object tags to the warehouse and use the tags to create cost reports.

If the warehouse used by the connector is used by other workflows, you can split the cost by roles. To split usage by roles, use the query for splitting warehouse usage and add the following WHERE clause on the QUERY_HISTORY view:

WAREHOUSE_NAME = '<connector warehouse name>' AND
ROLE_NAME = '<role created for the connector to ingest data>'
Copy

Note that the role is the name created when the connector was installed, for example SNOWFLAKE_CONNECTOR_FOR_GOOGLE_ANALYTICS_RAW_DATA.

The query gives only an approximation of the cost.

Note

Only one native app may use the warehouse, otherwise costs of different applications are inseparable because each native app uses the same role name (APP_PRIMARY).

For connectors configured to use serverless tasks, you can query the SERVERLESS_TASK_HISTORY view. The view exposes CREDITS_USED and DATABASE_NAME columns, the latter of which you can use for filtering on the name of the connector.

Storage Cost

The Snowflake Connector for ServiceNow® stores data in two places:

  • The connector database, which is created from the public share and which holds the connector internal state.

  • The user-specified schema where the ingested data is stored.

Data storage is also used by the Snowflake Fail-safe feature. The amount of data stored in Fail-safe depends on the table updates done by the connector.

If you want to check storage usage using Snowsight, we recommend that you use a separate database for storing ingested data. This way you can filter the graphs for storage usage by object, which shows usage by individual database. You can also view storage use by querying the DATABASE_STORAGE_USAGE_HISTORY view and filtering by databases used by the connector.

If the database contains other schemas not related to the connector, you can query storage usage of a specific schema that is dedicated to the data ingested from the connector. You can get the information from the TABLE_STORAGE_METRICS view after filtering by database and schema names and aggregating columns with storage usage.

In addition, the amount of Fail-safe data increases if the table rows ingested from ServiceNow® are updated frequently or the whole table is reloaded. Typically, seven to ten days after the connector is set up, the amount of Fail-safe data stabilizes (assuming that no reloads are performed and that the flow of ingested data is at a steady rate).

Data Transfer Cost

Snowflake charges only for egress traffic generated by the connector, based on the size of the requests from the connector to ServiceNow®. The responses from ServiceNow® do not generate cost on Snowflake side.

Information on data transfer usage is available only in the aggregated form for all external functions on the account level. To access the number of transferred bytes, use the DATA_TRANSFER_HISTORY view and filter by the EXTERNAL_ACCESS transfer type.

Determining optimal warehouse size for the connector instance

To find the optimal warehouse size for the connector, you should consider the factors that affect the performance of the connector, such as the size of ServiceNow® instance, the number of enabled tables, and the schedule for synchronizing each table. For example, if only a few tables are enabled the connector might not benefit from increased parallelization.

We recommend that you define a set of measurable expectations, such as time intervals in which all tables should be synchronized, and pick the smallest warehouse size that meets these expectations. For large amounts of ingested data with tens of synchronized tables, the default recommendation is Large warehouse. Alternatively, when you just want to try out the connector and enable a single table for ingestion, an X-Small warehouse should be sufficient. To determine if you can downsize the warehouse, see Monitoring warehouse load.

Starting and stopping the connector automatically

To save on cost, you can run the connector only during a specified timeframe (for example, outside business hours), by calling the STOP_CONNECTOR and RESUME_CONNECTOR procedures.

You can automate starting and stopping the connector with serverless tasks. For example, to run the connector outside of UTC business hours, you might use the following query:

CREATE TASK start_connector_after_business_hours
   SCHEDULE USING CRON 0 17 * * MON-FRI Europe/London
   AS CALL <my_connector>.PUBLIC.RESUME_CONNECTOR();

CREATE TASK stop_connector_before_business_hours
   SCHEDULE USING CRON 0 9 * * MON-FRI Europe/London
   AS CALL <my_connector_>.PUBLIC.STOP_CONNECTOR();
Copy
Language: English