[GitHub] [iceberg] marton-bod edited a comment on issue #2360: Internal Tests Cannot us ORC or Avro Datasource Writers

[GitBox <gi...@apache.org>]

marton-bod edited a comment on issue #2360:
URL: https://github.com/apache/iceberg/issues/2360#issuecomment-805043527


   Yes, we faced the same issue that Hive exec uses a different `orc-core` version than `iceberg-orc`, therefore we were not able to pull both of them in and run unit tests that can write to Hive ORC tables and Iceberg ORC tables at the same time. In a real cluster, this problem goes away, because we shade the `orc-core` dep used by iceberg-orc and include that into the runtime jar. As @pvary mentioned our initial idea to resolve this was to create a new `bundled-orc` module in Iceberg to avoid any classpath conflicts with Hive. A similar approach could resolve the problems for Spark too.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org