Skip to content

HMS 4.x metadata transformer silently rewrites schema locations #757

Description

@sweb

Affected Stackable version

26.7

Affected Apache Hive version

4.0.0

Current and expected behavior

Hive 4 introduced metastore.metadata.transformer.class and defaults it to org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.

Against a metastore used by Trino, the transformer causes two distinct failures:

  1. Schema locations are silently rewritten on read. A schema created with an explicit location under hive.metastore.warehouse.dir is stored correctly in DBS, but every read through the Thrift API returns <warehouse>/<dbname>.db instead. Trino's Iceberg connector derives new table locations from that value, so tables land in the wrong prefix.
  2. CREATE TABLE through the Trino Hive connector is converted from MANAGED_TABLE to EXTERNAL_TABLE. Trino then refuses to write to its own table with Cannot write to non-managed Hive table, so INSERT INTO is broken for every table created through the Hive connector.

Possible solution

For our use cases, disabling the transformer may be the right choice here.

Workaround

Disable the transformer through the metastore's config overrides. Note the value is a single
space, not an empty string:

spec:
  metastore:
    configOverrides:
      hive-site.xml:
        metastore.metadata.transformer.class: " "

Additional context

No response

Environment

k8s: v1.35.1 - also seen on OpenShift environments

Would you like to work on fixing this bug?

None

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions