iceberg: Iceberg does not support drop namespace cascade

The semantic of drop namespace cascade is different between Iceberg and Spark, then run Spark SQL DROP DATABASE xxx CASCADE cause failure if the database is not empty.

Iceberg interface SupportsNamespaces

  /**
   * Drop a namespace. If the namespace exists and was dropped, this will return true.
   *
   * @param namespace a namespace. {@link Namespace}
   * @return true if the namespace was dropped, false otherwise.
   * @throws NamespaceNotEmptyException If the namespace does not empty
   */
  boolean dropNamespace(Namespace namespace) throws NamespaceNotEmptyException;

Spark interface SupportsNamespaces extends CatalogPlugin

  /**
   * Drop a namespace from the catalog, recursively dropping all objects within the namespace.
   * <p>
   * If the catalog implementation does not support this operation, it may throw
   * {@link UnsupportedOperationException}.
   *
   * @param namespace a multi-part namespace
   * @return true if the namespace was dropped
   * @throws NoSuchNamespaceException If the namespace does not exist (optional)
   * @throws UnsupportedOperationException If drop is not a supported operation
   */
  boolean dropNamespace(String[] namespace) throws NoSuchNamespaceException;

The CASCADE logic is handled by DropNamespaceExec

    if (nsCatalog.namespaceExists(ns)) {
      // The default behavior of `SupportsNamespace.dropNamespace()` is cascading,
      // so make sure the namespace to drop is empty.
      if (!cascade) {
        if (catalog.asTableCatalog.listTables(ns).nonEmpty
          || nsCatalog.listNamespaces(ns).nonEmpty) {
          throw QueryExecutionErrors.cannotDropNonemptyNamespaceError(namespace)
        }
      }

      if (!nsCatalog.dropNamespace(ns)) {
        throw QueryExecutionErrors.cannotDropNonemptyNamespaceError(namespace)
      }

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 25 (25 by maintainers)

Most upvoted comments

HadoopCatalog

May I ask too, how big of an issue is this for you in practice? What’s the workflow that you use that requires Cascade support?

I encounter this issue when change the TPCDS test suite from Delta table format to Iceberg. The drop database cascade sql works with delta but failed in iceberg.

https://github.com/apache/incubator-kyuubi/blob/2baabf46cacf745e5ceaf78192f9f2359dc9e477/kyuubi-server/src/test/scala/org/apache/kyuubi/operation/tpcds/OutputSchemaTPCDSSuite.scala#L70