You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "sd4324530 (via GitHub)" <gi...@apache.org> on 2026/09/02 00:54:37 UTC
[PR] [docs] Fix misleading DROP COLUMN doc for hive catalog [paimon]
sd4324530 opened a new pull request, #9538:
URL: https://github.com/apache/paimon/pull/9538
### Purpose
The current docs for `ALTER TABLE ... DROP (col)` against a `hive` catalog
tell users they can work around the
`The following columns have types incompatible with the existing columns in
their respective positions` error by either:
1. disabling `hive.metastore.disallow.incompatible.col.type.changes` on the hive server (correct), or
2. setting `hadoop.hive.metastore.disallow.incompatible.col.type.changes=false` on the Paimon catalog / via Flink SQL `SET` / via `spark-sql --conf spark.hadoop.hive.metastore.disallow.incompatible.col.type.changes=false` (incorrect for a remote HMS).
Option 2 does not work against a real (remote) Hive Metastore. The compatibility check lives in the Hive Metastore **server**, in `HiveAlterHandler#alterTable` (hive-metastore):
```java
if (HiveConf.getBoolVar(hiveConf,
HiveConf.ConfVars.METASTORE_DISALLOW_INCOMPATIBLE_COL_TYPE_CHANGES, false)
&& !oldt.getTableType().equals(TableType.VIRTUAL_VIEW.toString())) {
MetaStoreUtils.throwExceptionIfIncompatibleColTypeChange(...);
}
```
The hiveConf here is the server-side HiveConf (loaded from hive-site.xml), and the alter is invoked over Thrift via client.alter_table_with_environmentContext(...) from HiveAlterTableUtils#alterTableWithEnv. The Thrift RPC does not carry the client's HiveConf to the server, so any client-side setting (including SET hadoop.hive.metastore.disallow.incompatible.col.type.changes=false in Flink SQL, the equivalent WITH (...) catalog option, or spark-sql --conf spark.hadoop.hive.metastore.disallow.incompatible.col.type.changes=false) has no effect. The check runs with the server's own value of hive.metastore.disallow.incompatible.col.type.changes, whose default in HiveConf is true.
That's why users following the documented workaround still hit the error. Paimon's own unit test for this scenario passes only because it uses an embedded HMSHandler in the same JVM, where the client HiveConf and the server HiveConf happen to be the same object.
This PR drops option 2 and rewrites the note in
docs/docs/flink/sql-alter.md and docs/docs/spark/sql-alter.md
(plus the related tip in docs/docs/flink/sql-ddl.md) so the only recommended path is to sethive.metastore.disallow.incompatible.col.type.changes=false on the Hive Metastore server (hive-site.xml, then restart HMS).
Refs: HIVE-17832.
Tests
Manual: re-rendered the three updated doc pages; the new :::warning block reads as the only supported way to opt out of the compatibility check, and the misleading "or set ... in your paimon catalog" bullet is removed.
No code / no test changes; this is a docs-only fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [docs] Fix misleading DROP COLUMN doc for hive catalog [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #9538:
URL: https://github.com/apache/paimon/pull/9538
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [docs] Fix misleading DROP COLUMN doc for hive catalog [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #9538:
URL: https://github.com/apache/paimon/pull/9538#issuecomment-5506622051
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [docs] Fix misleading DROP COLUMN doc for hive catalog [paimon]
Posted by "sd4324530 (via GitHub)" <gi...@apache.org>.
sd4324530 commented on PR #9538:
URL: https://github.com/apache/paimon/pull/9538#issuecomment-5502997522
@JingsongLi
Please take a look when you have time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org