Release notes

Overview of the most recent releases of Snowflake Open Catalog.

October 18, 2024

Snowflake Open Catalog - General Availability

With this release, we are pleased to announce the general availability of Snowflake Open Catalog, which was previously named Polaris Catalog and was available as a preview feature. With general availability, we’ve made the following updates:

  • A service admin can now create additional users for the Open Catalog account. These users can manage the account through the Open Catalog web interface. For details, see Manage users.

  • A catalog admin can now secure individual namespaces or tables within a catalog. You can also continue to secure a catalog at the catalog level. For details, see Secure catalogs.

  • When viewing the schema for a table in Snowflake Open Catalog, you can now view the nested schema for a column. For details, see View the schema for a table.

  • We’ve added billing support for Open Catalog, but you can use Open Catalog for free until April 30, 2025. For more information, see Billing.

August 8, 2024

With this release, we are pleased to announce the availability of the following new enhancements in this update to Polaris Catalog.

Snowflake now supports queries on tables with nested namespaces

Previously, we listed a limitation where Snowflake couldn’t read tables registered in Polaris Catalog that are located under a nested namespace. Snowflake now supports querying tables located under a nested namespace. For example, if you create a nested namespace namespace1.namespace1a.namespace1ab, Snowflake can read tables grouped under the namespace1ab namespace. For more information, see Create a namespace.

July 30, 2024

With this release, we are pleased to announce the initial public preview release of the Polaris Catalog hosted on Snowflake with the following features.

Apache Iceberg™ Rest API

Polaris Catalog provides an Apache Iceberg Rest Catalog API, which enables support for any query engine that supports the Apache Iceberg™ Rest catalog specification.

Authentication

Users can create service connections that provide a Client ID and Client Secret service credentials. These credentials are used for authentication by using OAuth 2.0.

Polaris Catalog User Interface

Polaris Catalog is provided with a web application to simplify the management of the catalog. Within the UI, users can manage catalogs, service principals, and the privileges for service principals.

Role Based Security Model

A role-based access control (RBAC) security model is included, so customers can manage the level of access each user or user group is allowed on the catalog. For more information, see Access control.

Credential Vending

Access to the storage objects where the data resides is managed by Polaris Catalog. When a user requests access to a table, whether for read or write, a temporary scoped storage credential is generated and passed back to the calling engine, which provides the appropriate access permissions to the folder in which the data resides within the storage.

Snowflake Warehouse Catalog Integration for Polaris Catalog

A new catalog integration for Polaris Catalog is available within Snowflake. This catalog integration allows users to create unmanaged Apache Iceberg™ tables that point to tables that reside in Polaris Catalog for querying.

Considerations and limitations

The following considerations and limitations apply to Polaris Catalog, and are subject to change:

Signup

  • Only Snowflake customers can sign up for Polaris Catalog.

Catalogs

  • Polaris Catalog currently supports Apache Iceberg™ tables that use either:

    • Polaris Catalog as the Iceberg catalog

    • Snowflake as the Iceberg catalog. External Iceberg catalogs other than Snowflake aren’t currently supported. If you want to add Iceberg tables from other external catalogs, you must migrate them.

  • You can’t import existing Iceberg tables from vendors such as Glue or Tabular into an internal catalog in Polaris Catalog, but they can be added to external catalogs.

  • For internal catalogs, you can’t rename a table across namespaces. For example, you can’t rename a table from /mytables/ns1/table1 to /mytables/ns2/table1.

  • When creating an internal or external catalog, you can’t specify a default base location or allowed location that overlaps with the directory hierarchy for a different catalog. For example, if the default base location for catalog1 is s3://mytables/db1/schema1/table1, you can’t specify the default base location for your new catalog as s3://mytables/db1/.

Table access

  • Snowflake can query but can’t write to tables managed by Polaris Catalog.

  • Snowflake Iceberg tables, which are available in external catalogs, are read only in Polaris Catalog.

Access control

  • The scoped access policy for a table is limited to the <table_base>/metadata/ and <table_base>/data/ directories.

Iceberg

  • When calling the registerTable API, you can’t register a table in a location that is outside of the parent namespace directory. For example, if the folder hierarchy for a catalog is s3://teambucket/iceberg/namespace1/namespace1a/, you can’t create mytbl3 with a base location s3://teambucket/iceberg/namespace1. You can create it, for example, with a base location s3://teambucket/iceberg/namespace1/mytbl3.

  • If you call the dropTable API and request to purge the table’s data and metadata by setting the purgeRequested parameter to true, Polaris Catalog makes a best effort to delete the following items:

    • All data and metadata files associated with the table

    • The storage directory for the table

    However, some of these items might not be deleted. If so, navigate to your external cloud storage to identify and delete the orphaned files or storage directory yourself.

Language: English