You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@parquet.apache.org by "Fokko (via GitHub)" <gi...@apache.org> on 2026/07/27 20:42:10 UTC
[PR] GH-3690: Update third-party NOTICE and LICENSE [parquet-java]
Fokko opened a new pull request, #3694:
URL: https://github.com/apache/parquet-java/pull/3694
### Rationale for this change
Updates the LICENSE and NOTICE with the 3rd party packages that are bundled in the binaries. `luben/zstd-jni` with the BSD-2 License is important.
### What changes are included in this PR?
### Are these changes tested?
### Are there any user-facing changes?
<!-- Please uncomment the line below and replace ${GITHUB_ISSUE_ID} with the actual Github issue id. -->
<!-- Closes #${GITHUB_ISSUE_ID} -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] GH-3690: Update third-party NOTICE and LICENSE [parquet-java]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on PR #3694:
URL: https://github.com/apache/parquet-java/pull/3694#issuecomment-5106355058
taking a look now, everything looks good. im double checking CLI since it has the largest set of 3rd party dependencies (22!)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] GH-3690: Update third-party NOTICE and LICENSE [parquet-java]
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on code in PR #3694:
URL: https://github.com/apache/parquet-java/pull/3694#discussion_r3662207677
##########
parquet-cli/src/main/resources/META-INF/NOTICE:
##########
@@ -43,3 +43,82 @@ notice:
| See the License for the specific language governing permissions and
| limitations under the License.
+--------------------------------------------------------------------------------
+
+This product bundles Apache Avro, which includes the following in its NOTICE
Review Comment:
Why do we need to copy the license of a bundled dependency?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] GH-3690: Update third-party NOTICE and LICENSE [parquet-java]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on code in PR #3694:
URL: https://github.com/apache/parquet-java/pull/3694#discussion_r3663191794
##########
parquet-cli/src/main/resources/META-INF/NOTICE:
##########
@@ -43,3 +43,82 @@ notice:
| See the License for the specific language governing permissions and
| limitations under the License.
+--------------------------------------------------------------------------------
+
+This product bundles Apache Avro, which includes the following in its NOTICE
Review Comment:
This is an interpretation of the legalese around what needs to be included in the LICENSE/NOTICE. Please note that a lot of projects do this differently.
My interpretation of it. For the ASF it is **required** for all the 3rd party source that is bundled with the source itself (so, borrowing a class from Avro etc) should be noted in the `LICENSE`. Since the ASF mostly cares around the source distribution, it is a bit of a gray area on how to handle the binary distribution where we include 3rd party binary because of bundling (in the `bundle` or `runtime` jar) or shading (mostly `FastUtil`). Some licenses explicitly call out that the license should be included when when binary distributed (eg, the [BSD-2 Clause](https://opensource.org/license/BSD-2-Clause): _Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution_.
So, including these makes sure that we adhere to all the requirements. The big downside is that the LICENSE and NOTICE grow pretty big, which should be avoided according to the [`{LICENSE,NOTICE}` assembling how-to](https://infra.apache.org/licensing-howto.html). Since Parquet only has two packages (`parquet-hadoop-bundle` and `parquet-cli-runtime`) that ship a reasonable amound of 3rd party packages, I don't think it is an issue of having a somewhat bigger LICENSE and NOTICE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] GH-3690: Update third-party NOTICE and LICENSE [parquet-java]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #3694:
URL: https://github.com/apache/parquet-java/pull/3694#discussion_r3668692513
##########
parquet-cli/src/main/resources/META-INF/LICENSE:
##########
Review Comment:
verified that all 22 dependencies are present (according to https://github.com/apache/parquet-java/issues/3690#issuecomment-5085686385)
com.beust:jcommander:1.82
com.fasterxml.jackson.core:jackson-annotations:2.22
com.fasterxml.jackson.core:jackson-core:2.22.1
com.fasterxml.jackson.core:jackson-databind:2.22.1
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.22.1
**com.github.luben:zstd-jni:1.5.7-11**
**commons-codec:commons-codec:1.11**
commons-pool:commons-pool:1.6
**io.airlift:aircompressor:2.0.3**
it.unimi.dsi:fastutil:8.5.18
**javax.annotation:javax.annotation-api:1.3.2**
net.openhft:zero-allocation-hashing:0.27ea1
net.sf.opencsv:opencsv:2.3
org.apache.avro:avro:1.11.5
org.apache.commons:commons-compress:1.26.2
org.apache.commons:commons-lang3:3.20.0
org.apache.commons:commons-text:1.15.0
org.apache.thrift:libthrift:0.23.0
**org.locationtech.jts:jts-core:1.20.0**
org.slf4j:slf4j-api:1.7.33
**org.tukaani:xz:1.12**
**org.xerial.snappy:snappy-java:1.1.10.8**
the 7 highlighted ones needs attention
| Dependency | Required `LICENSE` change | Reason |
|---|---|---|
| `javax.annotation:javax.annotation-api:1.3.2` | Select and include CDDL-1.0; provide the source location and identify it as ASF Category B, binary-only. | It is dual-licensed under CDDL-1.0 or GPLv2 with Classpath Exception. CDDL-1.0 is the preferable option for the shaded binary. |
| `org.locationtech.jts:jts-core:1.20.0` | Select and include EDL-1.0. | It is dual-licensed under EPL-2.0 or EDL-1.0. EDL-1.0 is the ASF Category-A option. |
| `org.xerial.snappy:snappy-java:1.1.10.8` | Include Google Snappy BSD-3, Bitshuffle MIT, and Intel PureJavaCrc32C BSD terms and copyrights. | The Apache-2.0 dependency bundles native and copied code under these additional licenses. |
| `io.airlift:aircompressor:2.0.3` | Include Google Snappy BSD-3 terms and copyright. | Its upstream `notice.md` identifies bundled Google Snappy code under BSD-3. |
| `commons-codec:commons-codec:1.11` | Include the Intel PureJavaCrc32C BSD terms and copyright. | Its `PureJavaCrc32C` implementation contains Intel-owned BSD-licensed portions. One shared Intel section can also cover snappy-java. |
| `org.tukaani:xz:1.12` | Replace the public-domain statement with the 0BSD terms and author attribution. | Version 1.12 source files explicitly use `SPDX-License-Identifier: 0BSD`. |
| `com.github.luben:zstd-jni:1.5.7-11` | Include the wrapper BSD-2, native Zstandard BSD-3, `libdivsufsort-lite` MIT, and legacy Zstandard BSD-2 coverage. | The JAR physically embeds the JNI wrapper and native code governed by these different licenses. Shared MIT and BSD-2 terms can avoid duplication. |
##########
parquet-column/src/main/resources/META-INF/NOTICE:
##########
Review Comment:
nit: Maven’s default generated NOTICE already uses `Apache Parquet <module’s name>`. Since these NOTICE file doesnt contain any other entries, we dont necessary need to add them.
This applies to both
- parquet-column/src/main/resources/META-INF/NOTICE
- parquet-hadoop/src/main/resources/META-INF/NOTICE
Furthermore the opposite problem occurs in [parquet-protobuf/src/main/resources/META-INF/NOTICE](https://github.com/apache/parquet-java/blob/master/parquet-protobuf/src/main/resources/META-INF/NOTICE). We should not use "Apache Parquet Java" but use the module name instead.
##########
parquet-cli/src/main/resources/META-INF/NOTICE:
##########
@@ -43,3 +43,82 @@ notice:
| See the License for the specific language governing permissions and
| limitations under the License.
+--------------------------------------------------------------------------------
+
+This product bundles Apache Avro, which includes the following in its NOTICE
+file:
+
+ Apache Avro
+ Copyright 2009-2025 The Apache Software Foundation
+
+ This product includes software developed at
+ The Apache Software Foundation (http://www.apache.org/).
+
+--------------------------------------------------------------------------------
+
+This product bundles Apache Thrift, which includes the following in its NOTICE
+file:
+
+ Apache Thrift
+ Copyright (C) 2006 - 2019, The Apache Software Foundation
+
+ This product includes software developed at
+ The Apache Software Foundation (http://www.apache.org/).
+
+--------------------------------------------------------------------------------
+
+This product bundles Jackson, which includes the following in its NOTICE file:
+
+ # Jackson JSON processor
+
+ Jackson is a high-performance, Free/Open Source JSON processing library.
+ It was originally written by Tatu Saloranta (tatu.saloranta@iki.fi), and has
+ been in development since 2007.
+ It is currently developed by a community of developers.
+
+ Copyright 2007-, Tatu Saloranta (tatu.saloranta@iki.fi)
+
+ jackson-core bundles a shaded copy of FastDoubleParser
+ <https://github.com/wrandelshofer/FastDoubleParser> and a copy of the
+ Schubfach number-writing code <https://github.com/c4f7fcce9cb06515/Schubfach>,
+ both available under the MIT license.
+
+--------------------------------------------------------------------------------
+
+This product bundles several Apache Commons libraries, which include the
+following in their NOTICE files:
+
+ Apache Commons Codec
+ Copyright 2002-2017 The Apache Software Foundation
+
+ This product includes software developed at
+ The Apache Software Foundation (http://www.apache.org/).
+
+ src/test/org/apache/commons/codec/language/DoubleMetaphoneTest.java
+ contains test data from http://aspell.net/test/orig/batch0.tab.
+ Copyright (C) 2002 Kevin Atkinson (kevina@gnu.org)
+
+ The content of package org.apache.commons.codec.language.bm has been
+ translated from the original php source code available at
+ http://stevemorse.org/phoneticinfo.htm with permission from the original
+ authors. Original source copyright: Copyright (c) 2008 Alexander Beider &
+ Stephen P. Morse.
+
+ Apache Commons Compress
+ Copyright 2002-2024 The Apache Software Foundation
+
+ This product includes software developed at
+ The Apache Software Foundation (https://www.apache.org/).
+
+ Apache Commons Lang
+ Copyright 2001-2025 The Apache Software Foundation
+
+ Apache Commons Text
+ Copyright 2014-2025 The Apache Software Foundation
+
+ Apache Commons Pool
+ Copyright 2001-2012 The Apache Software Foundation
+
+ This product includes software developed by
+ The Apache Software Foundation (http://www.apache.org/).
+
Review Comment:
Paired with codex, these 13 (out of 22) dependencies should propagate NOTICE.
we captured all except 2 (highlighted)
| Dependency | Required NOTICE content |
|---|---|
| `org.apache.thrift:libthrift:0.23.0` | Apache Thrift copyright notice |
| `org.apache.avro:avro:1.11.5` | Apache Avro copyright notice |
| `com.fasterxml.jackson.core:jackson-annotations:2.22` | Shared Jackson notice |
| `com.fasterxml.jackson.core:jackson-core:2.22.1` | Shared Jackson notice, including FastDoubleParser and Schubfach |
| `com.fasterxml.jackson.core:jackson-databind:2.22.1` | Shared Jackson notice |
| `com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.22.1` | Covered by the shared Jackson notice; no separate text |
| `commons-codec:commons-codec:1.11` | Commons Codec copyright and Beider–Morse attribution; exclude the test-only `DoubleMetaphoneTest` notice |
| `org.apache.commons:commons-compress:1.26.2` | Commons Compress copyright notice |
| `org.apache.commons:commons-lang3:3.20.0` | Commons Lang copyright notice |
| `commons-pool:commons-pool:1.6` | Commons Pool copyright notice |
| `org.apache.commons:commons-text:1.15.0` | Commons Text copyright notice |
| **`io.airlift:aircompressor:2.0.3`** | Dain Sundstrom and Google Snappy copyrights |
| **`org.xerial.snappy:snappy-java:1.1.10.8`** | Google Snappy, Apache `PureJavaCrc32C`, static `libstdc++`/GCC Runtime Library Exception, and contributor notices |
##########
parquet-cli/src/main/resources/META-INF/NOTICE:
##########
@@ -43,3 +43,82 @@ notice:
| See the License for the specific language governing permissions and
| limitations under the License.
+--------------------------------------------------------------------------------
+
+This product bundles Apache Avro, which includes the following in its NOTICE
Review Comment:
> I don't think it is an issue of having a somewhat bigger LICENSE and NOTICE.
Big is relative 😆 I've seen wayyy longer LICENSE and NOTICE files
BTW these are the specific 3rd party dependencies bundled:
https://github.com/apache/parquet-java/issues/3690#issuecomment-5085686385
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] GH-3690: Update third-party NOTICE and LICENSE [parquet-java]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko merged PR #3694:
URL: https://github.com/apache/parquet-java/pull/3694
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org