Manage Cross-Cloud Auto-Fulfillment costs¶
As a provider, you can enable Cross-Cloud Auto-Fulfillment for a listing to automatically make your data product available in other Snowflake regions.
When you configure Cross-Cloud Auto-Fulfillment for your listing, you don’t have to manage replicating the data. However, you still incur costs associated with transferring and storing your data product in other Snowflake regions to support consumers of your listing.
Unlike traditional manual database replication, Cross-Cloud Auto-Fulfillment does not require a separate account in each region that you support. Instead, Snowflake creates one secure share area for an organization to manage auto-fulfillment to a region and associates billing costs with that area. Because of that, the costs associated with Cross-Cloud Auto-Fulfillment are attributed differently when compared to manual database replication costs.
In addition, egress cost optimization can reduce the costs of auto-fulfillment. For an introduction to egress const optimization, see Optimizing Auto-fulfillment data transfer costs using Egress Cost Optimizer.
About Cross-Cloud Auto-Fulfillment¶
Cross-Cloud Auto-Fulfillment lets you offer a data product in any supported Snowflake region, based on the availability and access options you select for your listing, without having to manually replicate data.
For more details, see How Cross-Cloud Auto-fulfillment works.
Understand Cross-Cloud Auto-Fulfillment costs¶
Cross-Cloud Auto-Fulfillment incurs usage costs in the same way that regular usage of Snowflake does:
- Compute resources
Auto-fulfillment operations use compute resources to copy data and manage the status of the data in the secure share areas in other regions.
Snowflake Marketplace calculates compute costs for listing auto-fulfillment to VPS regions by using VPS rates. For details on VPS rates, see Snowflake Service Consumption Table.
- Storage resources
Databases transferred to secure share areas in other regions incur storage costs.
- Data transfer resources (egress)
The initial database auto-fulfillment and subsequent synchronization operations transfer data between regions. Cloud providers charge for data transferred from one region to another within their own network or a region in another cloud.
The data transfer rate is determined by the location of the source account (i.e. the account that stores the primary database) and the destination region and cloud. For data transfer pricing, see the pricing guide (on the Snowflake website).
For more information about data transfer billing, refer to Understanding Data Transfer Cost.
Egress costs can often be reduced by enabling Egress Cost Optimization. For more information see Optimizing Auto-fulfillment data transfer costs using Egress Cost Optimizer.
When you use Cross-Cloud Auto-Fulfillment, these usage costs are attributed to one Snowflake-managed secure share area for each region in which you have active consumers of your listings. See View actual costs for details about attributing costs. For details about the components of cost in Snowflake, see Understanding Overall Cost.
Estimate and control costs¶
When you configure auto-fulfillment for your listing, the following factors can affect the cost of fulfilling your listing to other regions:
- Compute Resource Factors
Queries run by Snowflake to fulfill your listing contributes to compute resources. The refresh frequency that you set affects how frequently these queries run.
- Storage Resource Factors
The size of the database, the rate at which data is appended and updated, and the rate of change in the database affect how much data is auto-fulfilled and stored initially and continuously.
- Data Transfer Resource Factors
The cloud region that the listing is auto-fulfilled to, and the cloud provider of that region affect the cost of data transfer. The more regions that consumers request your listing in, the higher the cost to fulfill those listings, due to the data transfer cost. For data transfer pricing, see the pricing guide (on the Snowflake website).
If you want to minimize costs associated with auto-fulfillment, review the usage of your listings and learn more about preparing your data for auto-fulfillment:
- Monitor Compute Resources
Identify the queries run by Snowflake and review the refresh frequency interval for your listings.
Refer to the LISTING_AUTO_FULFILLMENT_REFRESH_DAILY View to identify the listings and databases contributing to compute cost.
To identify the queries run by Snowflake to support auto-fulfillment, review the Query History and filter on Client generated statements. Refer to the Query History Page.
Review the refresh frequency interval that you set for the listing. Refer to Monitoring and managing auto-fulfillment settings.
- Monitor Storage Resources
Determine what data to put in your listing and how to structure your data to minimize the amount that needs to be auto-fulfilled. Refer to Prepare data for a listing. Cross-Cloud Auto-Fulfillment does not support secure views that reference data stored in other databases.
Refer to the LISTING_AUTO_FULFILLMENT_DATABASE_STORAGE_DAILY View to identify listings and databases contributing to storage cost.
- Monitor Data Transfer Resources
Identify the regions in which secure share areas have been created. Run the SHOW REPLICATION ACCOUNTS command.
To view estimated costs for all secure share areas associated with the provider accounts in your organization, use the LISTING_AUTO_FULFILLMENT_USAGE_HISTORY View in the ORGANIZATION_USAGE schema of the SNOWFLAKE database.
To view actual costs for accounts in your organization, use other views in the ORGANIZATION_USAGE schema of the SNOWFLAKE database.
Benefits and costs of egress cost optimization¶
Egress cost optimization can be used to reduce and control listing auto-fulfillment costs.
- Initial costs:
The first time data is auto-fulfilled using the egress cost optimizer, the data is cached in Snowflake-managed S3-compatible storage with zero-egress costs, and you are charged for the initial egress of all the data in each listing to this storage location. Thereafter, egress is charged only for data updates.
- Incremental data loading vs full data reloading:
If you regularly replace tables, or truncate and reload tables, be aware that this fresh data will be treated as a new table. Using these processes causes those tables to be re-cached, which incurs a higher cost than modifying the data by using less resource-intensive methods.
- Greater savings with many regions or clouds:
Sharing data across more regions increases your savings on total egress costs. The more regions where data is shared, the greater the savings with the egress cost optimizer.
- Database level, not listing level:
Where an auto-fulfillment schedule is set on the account level, rather than on the listing level, the egress cost optimizer will be enabled on all the listings that follow the account schedule. After the cost optimizer is enabled on a database, all subsequent auto-fulfillment involving that database will use it.
For more information about pricing for egress between source and target regions or clouds, see the Snowflake pricing guide.
Limitations¶
Incremental data ingestion is required for the cloud cache to be fully used by the egress cost optimizer.
The cloud cache is only used by the egress cost optimizer for refreshes made by auto-fulfillment.
View actual costs¶
You can use the ORGANIZATION_USAGE view or the Snowsight Usage dashboard to view costs associated with Cross-Cloud Auto-Fulfillment and attribute costs associated with fulfilling listings to specific regions. Use the accounts prefixed with SNOWFLAKE_MANAGED$ and AUTO_FULFILLMENT_AREA$ to attribute cost to specific regions.
You must be an account administrator (use the ACCOUNTADMIN role) or use the ORGANIZATION_USAGE_VIEWER database role to view usage data for Snowflake.
To view actual costs in Snowsight, do the following:
Sign in to Snowsight.
Select Admin » Cost Management, and select the Consumption tab.
Select a warehouse to use to view the usage data.
Using the accounts filter, select the accounts titled SNOWFLAKE_MANAGED$PUBLIC_<region_name> or AUTO_FULFILLMENT_AREA$-<region_name> to filter on the secure share areas used by auto-fulfillment.
For example, select SNOWFLAKE_MANAGED$PUBLIC_AWS_EU_WEST_2 to view the costs associated with using auto-fulfilling data to the AWS region eu_west_2.
Use the filters to view all usage types, or focus on compute, storage, or data transfer costs.
The costs that you see reflect all listings shared to a particular region by any account in your organization. To identify which listings are being consumed in which regions and contributing to the costs in a specific region, see Monitor listing use.
To view estimated costs using SQL, you can query the LISTING_AUTO_FULFILLMENT_USAGE_HISTORY View in the ORGANIZATION_USAGE schema. To view actual costs, refer to the other views in the ORGANIZATION_USAGE schema. For more details on viewing costs, see Exploring Overall Cost.