You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/05/10 19:34:01 UTC
[GitHub] [iceberg] RussellSpitzer opened a new pull request #2579: Example of how we can test Spark3 Artifacts against Both Spark Versions
RussellSpitzer opened a new pull request #2579:
URL: https://github.com/apache/iceberg/pull/2579
I removed the previous changes to build.gradle for this demonstration, basically
what I've done is create a second source set which uses the exact same directories
as the normal test configuration. The difference is that the test runtime declares
a dependecy on the newer Spark version while still using classes generated by the
original iceberg-spark3:compile task.
This task will compile the test clases using Spark 3.1 but use the library classes
compiled against Spark 3.0. We can use a similar approach for Spark3-Extensions
and Spark3-Runtime Jar.
One thing we may want to fix here is that I just lazily update the Spark Dependecy
by overriding the value in the base configuration. We may want to exclude and then
reintroduce it, but that seemed a bit more difficult in my testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer closed pull request #2579: Example of how we can test Spark3 Artifacts against Both Spark Versions
Posted by GitBox <gi...@apache.org>.
RussellSpitzer closed pull request #2579:
URL: https://github.com/apache/iceberg/pull/2579
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org