You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "ajantha-bhat (via GitHub)" <gi...@apache.org> on 2024/10/30 14:23:26 UTC
[I] Runtime jars are not including module's license and notice [iceberg]
ajantha-bhat opened a new issue, #11431:
URL: https://github.com/apache/iceberg/issues/11431
### Apache Iceberg version
1.6.1 (latest release)
### Query engine
None
### Please describe the bug 🐞
None of the shadow jars, including `LICENSE` and `NOTICE` from module directory. It only include from root directory.
This is wrong.
Example,` iceberg-hive-runtime` has its own license and notice. But the runtime jar compiled includes only the root license and notice.
I tried printing `projectDir` , it is the module directory itself.
I tired adding `duplicatesStrategy = DuplicatesStrategy.INCLUDE`, that didn't help either.
Expected that runtime jars should pack the `LICENSE` and `NOTICE` from its module directory.
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Runtime jars are not including module's license and notice [iceberg]
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on issue #11431:
URL: https://github.com/apache/iceberg/issues/11431#issuecomment-2449626642
Thanks. Seems to be the problem with the archive utitlity.app in the mac. Which was overwriting the identical files without the prompt.
I can confirm it is not the issue. Sorry for the noise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Runtime jars are not including module's license and notice [iceberg]
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on issue #11431:
URL: https://github.com/apache/iceberg/issues/11431#issuecomment-2449305766
Are you sure? I checked again, the NOTICE file in the module directory I shared is 500 lines and the one in jar is just 25 lines.
You missed something?
https://github.com/apache/iceberg/issues/11431#issuecomment-2449185232
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Runtime jars are not including module's license and notice [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on issue #11431:
URL: https://github.com/apache/iceberg/issues/11431#issuecomment-2449176257
Do you mean `sources` jar like https://repository.apache.org/content/repositories/orgapacheiceberg-1175/org/apache/iceberg/iceberg-spark-runtime-3.5_2.13/1.7.0/iceberg-spark-runtime-3.5_2.13-1.7.0-sources.jar? I see they are included in the binary jar.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Runtime jars are not including module's license and notice [iceberg]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #11431:
URL: https://github.com/apache/iceberg/issues/11431#issuecomment-2449806988
No problem at all, these things are important for the release, so better to double check it than having non-ASF compliant releases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Runtime jars are not including module's license and notice [iceberg]
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on issue #11431:
URL: https://github.com/apache/iceberg/issues/11431#issuecomment-2449185232
I meant the actual runtime jar
https://repository.apache.org/content/repositories/orgapacheiceberg-1175/org/apache/iceberg/iceberg-spark-runtime-3.5_2.12/1.7.0/iceberg-spark-runtime-3.5_2.12-1.7.0.jar
For example, this jar has to use the license file from its module directory
https://github.com/apache/iceberg/blob/main/spark/v3.5/spark-runtime/NOTICE
But if we unzip the jar and see, the notice file is from root directory. It doesn't include the notice for the libraries it packs as a runtime jar.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Runtime jars are not including module's license and notice [iceberg]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #11431:
URL: https://github.com/apache/iceberg/issues/11431#issuecomment-2449423197
I believe they are both present. I just build the runtime jars locally:
```
./gradlew build -x test -x integrationTest
ls -lah spark/v3.5/spark-runtime/build/libs/iceberg-spark-runtime-3.5_2.12-1.7.0-SNAPSHOT*
-rw-r--r-- 1 fokko.driesprong staff 6.0K Oct 30 12:55 spark/v3.5/spark-runtime/build/libs/iceberg-spark-runtime-3.5_2.12-1.7.0-SNAPSHOT-javadoc.jar
-rw-r--r-- 1 fokko.driesprong staff 6.0K Oct 30 12:55 spark/v3.5/spark-runtime/build/libs/iceberg-spark-runtime-3.5_2.12-1.7.0-SNAPSHOT-sources.jar
-rw-r--r-- 1 fokko.driesprong staff 6.0K Oct 30 12:55 spark/v3.5/spark-runtime/build/libs/iceberg-spark-runtime-3.5_2.12-1.7.0-SNAPSHOT-tests.jar
-rw-r--r-- 1 fokko.driesprong staff 43M Oct 30 12:55 spark/v3.5/spark-runtime/build/libs/iceberg-spark-runtime-3.5_2.12-1.7.0-SNAPSHOT.jar
```
When unzipping, it shows that there are two `LICENSE` and `NOTICE` files:
```
unzip iceberg-spark-runtime-3.5_2.12-1.7.0-SNAPSHOT.jar
...
replace LICENSE? [y]es, [n]o, [A]ll, [N]one, [r]ename: r
new name: LICENSE2
inflating: LICENSE2
replace NOTICE? [y]es, [n]o, [A]ll, [N]one, [r]ename: r
new name: NOTICE2
```
Looks like `LICENSE` is [the root one](https://github.com/apache/iceberg/blob/main/LICENSE):
```
tail LICENSE
This product includes code from Apache Flink.
* Parameterized test at class level logic in ParameterizedTestExtension.java
* Parameter provider annotation for parameterized tests in Parameters.java
* Parameter field annotation for parameterized tests in Parameter.java
Copyright: 1999-2022 The Apache Software Foundation.
Home page: https://flink.apache.org/
License: https://www.apache.org/licenses/LICENSE-2.0%
```
And `LICENSE2` looks like the [one in the subdir](https://github.com/apache/iceberg/blob/main/spark/v3.5/spark-runtime/LICENSE):
```
tail LICENSE2
--------------------------------------------------------------------------------
This product includes code from Apache HttpComponents Client.
* retry and error handling logic in ExponentialHttpRequestRetryStrategy.java
Copyright: 1999-2022 The Apache Software Foundation.
Home page: https://hc.apache.org/
License: https://www.apache.org/licenses/LICENSE-2.0
```
One of them might have been dropped because of the identical name, but it looks like it is in there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Runtime jars are not including module's license and notice [iceberg]
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on issue #11431:
URL: https://github.com/apache/iceberg/issues/11431#issuecomment-2448893435
I don't have solution yet (not a Gradle expert).
I am figuring out. But let me know if you find something obvious.
Can refer any of the runtime shadow jar task like spark, hive, flink etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Runtime jars are not including module's license and notice [iceberg]
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat closed issue #11431: Runtime jars are not including module's license and notice
URL: https://github.com/apache/iceberg/issues/11431
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Runtime jars are not including module's license and notice [iceberg]
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on issue #11431:
URL: https://github.com/apache/iceberg/issues/11431#issuecomment-2449189810
So, basically the code in this shadow jar block is not working
https://github.com/apache/iceberg/blob/91e04c9c88b63dc01d6c8e69dfdc8cd27ee811cc/spark/v3.5/build.gradle#L270-L273
I am trying some solutions, no clue yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Runtime jars are not including module's license and notice [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on issue #11431:
URL: https://github.com/apache/iceberg/issues/11431#issuecomment-2449212156
I downloaded the above runtime jar, compared it with that from the module directory but found no difference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Runtime jars are not including module's license and notice [iceberg]
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on issue #11431:
URL: https://github.com/apache/iceberg/issues/11431#issuecomment-2448887845
Same thing observed for spark runtime and flink runtime also.
Also, checked old release like 1.5.0 and it also has the same problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org