You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "shivjha30 (via GitHub)" <gi...@apache.org> on 2024/05/08 04:48:16 UTC
[I] "Iceberg.engine.hive.enabled" Conf is not honouring for HIVE CATALOG [iceberg]
shivjha30 opened a new issue, #10286:
URL: https://github.com/apache/iceberg/issues/10286
### Apache Iceberg version
1.4.3
### Query engine
Hive
### Please describe the bug đ
In the iceberg documentation it could be seen that
"To enable Hive support globally for an application, set iceberg.engine.hive.enabled=true in its Hadoop configuration."
The value of iceberg.engine.hive.enabled must be true in order to enable hive support. However, it could still use Iceberg's hive support even if it is set as "iceberg.engine.hive.enabled" to false. If i understand correctly, the value "iceberg.engine.hive.enabled" is irrelevant.
Current scenario:
if it is a hive catalog we are setting engine.hive.enabled as true.
In
[iceberg/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java](https://github.com/apache/iceberg/blob/01bc864b8eb8c4ca4240af85f00f4e67c78c466e/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java#L486)
Line 486 in [01bc864](https://github.com/apache/iceberg/commit/01bc864b8eb8c4ca4240af85f00f4e67c78c466e)
Even when we set ConfigProperties.ENGINE_HIVE_ENABLED to FALSE in Hadoop configuration, we are passing the ConfigProperties.ENGINE_HIVE_ENABLED as TRUE in the HiveIcebergMetaHook#commitCreateTable if it's of HiveCatalog type.
We are initially evaluating the 'engine.hive.enabled' value in HiveTableOperations#hiveEngineEnabled. In cases where it's a Hive catalog, this value is consistently 'True', rendering the 'iceberg.engine.hive.enabled' as unnecessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] "Iceberg.engine.hive.enabled" Conf is not honouring for HIVE CATALOG [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #10286:
URL: https://github.com/apache/iceberg/issues/10286#issuecomment-2458451614
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] "Iceberg.engine.hive.enabled" Conf is not honouring for HIVE CATALOG [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #10286:
URL: https://github.com/apache/iceberg/issues/10286#issuecomment-2487026481
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] "Iceberg.engine.hive.enabled" Conf is not honouring for HIVE CATALOG [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on issue #10286:
URL: https://github.com/apache/iceberg/issues/10286#issuecomment-2099890479
This is intentional.
If you are using [HiveIcebergMetaHook](https://github.com/apache/iceberg/blob/ed0959257cba02f378f7097d81cecaaaef9fa43f/mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java) then you are using Hive to create the table. The expectation in this case, that you want to be able read the table with Hive.
See the discussion here: https://github.com/apache/iceberg/pull/1495#discussion_r1582544488
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] "Iceberg.engine.hive.enabled" Conf is not honouring for HIVE CATALOG [iceberg]
Posted by "shivjha30 (via GitHub)" <gi...@apache.org>.
shivjha30 commented on issue #10286:
URL: https://github.com/apache/iceberg/issues/10286#issuecomment-2100717247
@pvary then, for a cluster level if we set icebeg.engine.hive.enabled as false and we donât pass the table properties(hive.engine.enabled as false), even then we are creating the table with Iceberg, ideally it shouldnât be.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] "Iceberg.engine.hive.enabled" Conf is not honouring for HIVE CATALOG [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #10286: "Iceberg.engine.hive.enabled" Conf is not honouring for HIVE CATALOG
URL: https://github.com/apache/iceberg/issues/10286
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org