You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/01/21 10:54:22 UTC
[GitHub] [iceberg] massdosage commented on a change in pull request #3944: add section to include instructions for hive on tez
massdosage commented on a change in pull request #3944:
URL: https://github.com/apache/iceberg/pull/3944#discussion_r789554198
##########
File path: site/docs/hive.md
##########
@@ -72,6 +72,15 @@ catalog.createTable(tableId, schema, spec, tableProperties);
The table level configuration overrides the global Hadoop configuration.
+#### Hive on Tez configuration
+
+To use Tez engine on Hive, Tez needs to be upgraded to >= `0.10.1` to contain the fix [Tez-4248](https://issues.apache.org/jira/browse/TEZ-4248).
Review comment:
```suggestion
To use the Tez engine on Hive, Tez needs to be upgraded to >= `0.10.1` which contains a necessary fix [Tez-4248](https://issues.apache.org/jira/browse/TEZ-4248).
```
##########
File path: site/docs/hive.md
##########
@@ -72,6 +72,15 @@ catalog.createTable(tableId, schema, spec, tableProperties);
The table level configuration overrides the global Hadoop configuration.
+#### Hive on Tez configuration
+
+To use Tez engine on Hive, Tez needs to be upgraded to >= `0.10.1` to contain the fix [Tez-4248](https://issues.apache.org/jira/browse/TEZ-4248).
+
+!!! Warning
+For Hive `2.3.x`, need to manual build from Tez `branch-0.9` for compatibility issue in Tez `0.10.1`.
+
+And also set the hive config `tez.mrreader.config.update.properties=hive.io.file.readcolumn.names,hive.io.file.readcolumn.ids`.
Review comment:
```suggestion
You will also need to set the following property in the Hive configuration: `tez.mrreader.config.update.properties=hive.io.file.readcolumn.names,hive.io.file.readcolumn.ids`.
```
##########
File path: site/docs/hive.md
##########
@@ -72,6 +72,15 @@ catalog.createTable(tableId, schema, spec, tableProperties);
The table level configuration overrides the global Hadoop configuration.
+#### Hive on Tez configuration
+
+To use Tez engine on Hive, Tez needs to be upgraded to >= `0.10.1` to contain the fix [Tez-4248](https://issues.apache.org/jira/browse/TEZ-4248).
+
+!!! Warning
+For Hive `2.3.x`, need to manual build from Tez `branch-0.9` for compatibility issue in Tez `0.10.1`.
Review comment:
```suggestion
For Hive `2.3.x`, you will need to manually build Tez from the `branch-0.9` branch due to a backwards incompatibility issue with Tez `0.10.1`.
```
##########
File path: site/docs/hive.md
##########
@@ -72,6 +72,15 @@ catalog.createTable(tableId, schema, spec, tableProperties);
The table level configuration overrides the global Hadoop configuration.
+#### Hive on Tez configuration
+
+To use Tez engine on Hive, Tez needs to be upgraded to >= `0.10.1` to contain the fix [Tez-4248](https://issues.apache.org/jira/browse/TEZ-4248).
+
+!!! Warning
+For Hive `2.3.x`, need to manual build from Tez `branch-0.9` for compatibility issue in Tez `0.10.1`.
Review comment:
So if I understand correctly, Hive 2.x only works with Tez 0.9.x and not 0.10.x? And therefore one needs to compile and build a specific version of Tez oneself (from that branch?) and then override the version that comes with Hive 2.3.x?
##########
File path: site/docs/hive.md
##########
@@ -72,6 +72,15 @@ catalog.createTable(tableId, schema, spec, tableProperties);
The table level configuration overrides the global Hadoop configuration.
+#### Hive on Tez configuration
+
+To use Tez engine on Hive, Tez needs to be upgraded to >= `0.10.1` to contain the fix [Tez-4248](https://issues.apache.org/jira/browse/TEZ-4248).
Review comment:
Is the above for Hive >=3? If so, probably worth mentioning it. Does this also mean you need to override the Tez jar files that come with the standard Hive installation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org