You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "OlgaGorlova (via GitHub)" <gi...@apache.org> on 2023/04/24 10:05:52 UTC

[GitHub] [beam] OlgaGorlova opened a new issue, #26403: Update vendored calcite to eliminate vulnerability from shaded log4j:1.2.17 and protobuf-java:3.19.2

OlgaGorlova opened a new issue, #26403:
URL: https://github.com/apache/beam/issues/26403

   ### What would you like to happen?
   
   Hi team,
   
   The beam-vendor-calcite-1_28_0 contains a bunch of shaded dependencies with major security vulnerabilities. For example, 
   [log4j:1.2.17](https://mvnrepository.com/artifact/log4j/log4j/1.2.17) and [protobuf-java:3.19.2](https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java/3.19.2). 
   
   Are there any plans to upgrade the vendored calcite? 
   
   We are using [beam-sdks-java-extensions-sql:2.46.0](https://mvnrepository.com/artifact/org.apache.beam/beam-sdks-java-extensions-sql/2.46.0)  and [beam-sdks-java-extensions-sql—zetasql:2.46.0](https://mvnrepository.com/artifact/org.apache.beam/beam-sdks-java-extensions-sql-zetasql/2.46.0) dependencies in our code. We use [SqlTransform](https://github.com/apache/beam/blob/db8db0b6ed0fe1e4891f207f0f7f811798e54db1/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTransform.java) class which internally references classes from beam-vendor-calcite-1_28_0. 
   
   Looks like beam-vendor-calcite is created as fat-jar, and the log4j:1.2.17 is shaded into this beam-vendor-calcite-1_28_0:0.2. So, we cannot exclude log4j:1.2.17 from this vendor jar and we cannot even override it with the latest versions either.
   We cannot use any other beam-vendor-calcite version as beam-vendor-calcite-1_28_0:0.2 is the latest version.
   We cannot exclude beam-vendor-calcite because internally SqlTransform class references classes from beam-vendor-calcite-1_28_0 such as [org.apache.beam.vendor.calcite.v1_28_0.com.google.common.collect.ImmutableList](https://github.com/apache/beam/blob/db8db0b6ed0fe1e4891f207f0f7f811798e54db1/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTransform.java#L48).
   
   And, if we include this vendor jar, it gets flagged vulnerable by our security scan due to shaded log4j:1.2.17.
   Hence, it would be really helpful if the log4j:1.2.17 was excluded or replaced with its latest version in beam-vendor-calcite-1_28_0.
   
   Please note that there are more vulnerable dependencies shaded in beam-vendor-calcite-1_28_0, but log4j:1.2.17 is currently the most critical one:
   
   - [com.google.protobuf:protobuf-java:3.19.2](https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java/3.19.2)
   - [com.google.protobuf:protobuf-java-util:3.19.2](https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java-util/3.19.2)
   - [net.minidev:json-smart:2.3](https://mvnrepository.com/artifact/net.minidev/json-smart/2.3)
   - [org.yaml:snakeyaml:1.24](https://mvnrepository.com/artifact/org.yaml/snakeyaml/1.24)
   - [commons-io:commons-io:2.4](https://mvnrepository.com/artifact/commons-io/commons-io/2.4)
   - [com.fasterxml.jackson.core:jackson-databind:2.10.0](https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind/2.10.0)
   - [com.google.code.gson:gson:2.8.6](https://mvnrepository.com/artifact/com.google.code.gson/gson/2.8.6)
   
   ### Issue Priority
   
   Priority: 3 (nice-to-have improvement)
   
   ### Issue Components
   
   - [ ] Component: Python SDK
   - [X] Component: Java SDK
   - [ ] Component: Go SDK
   - [ ] Component: Typescript SDK
   - [ ] Component: IO connector
   - [ ] Component: Beam examples
   - [ ] Component: Beam playground
   - [ ] Component: Beam katas
   - [ ] Component: Website
   - [ ] Component: Spark Runner
   - [ ] Component: Flink Runner
   - [ ] Component: Samza Runner
   - [ ] Component: Twister2 Runner
   - [ ] Component: Hazelcast Jet Runner
   - [ ] Component: Google Cloud Dataflow Runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] OlgaGorlova commented on issue #26403: Update vendored calcite to eliminate vulnerability from shaded log4j:1.2.17 and protobuf-java:3.19.2

Posted by "OlgaGorlova (via GitHub)" <gi...@apache.org>.
OlgaGorlova commented on issue #26403:
URL: https://github.com/apache/beam/issues/26403#issuecomment-1534207144

   Hi @Abacn, thank you for looking into this.
   You're right, I also don't think it will resolve the issue with security scans. 
   However, our security scan is finding the reference to log4j under beam-vendor-calcite-1_28_0-0.2.jar/META-INF/maven/log4j/* .
   Looks like the package is built as a fat jar. Is there a reason for that?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] Abacn commented on issue #26403: Update vendored calcite to eliminate vulnerability from shaded log4j:1.2.17 and protobuf-java:3.19.2

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #26403:
URL: https://github.com/apache/beam/issues/26403#issuecomment-1535073536

   That is the maven metadata file (`pom.xml`) for log4j. The fat jar does not include log4j. beam-vendor-calcite-1_28_0-0.2.jar is our vendored dependency. Beam vendors a few dependencies which shaded all necessary transitive dependencies in a fat jar. This avoids conflict introduced by user dependencies.
   
   I setup a project using SqlTransform, and running `mvn dependency:tree` there is no log4j dependency:
   
   ```
   [INFO] com.github.yathu:dependencyTriageTest:jar:1.0-SNAPSHOT
   [INFO] +- org.apache.beam:beam-sdks-java-core:jar:2.46.0:compile
   [INFO] |  +- org.apache.beam:beam-model-pipeline:jar:2.46.0:compile
   [INFO] |  |  \- org.conscrypt:conscrypt-openjdk-uber:jar:2.5.2:compile
   [INFO] |  +- org.apache.beam:beam-model-job-management:jar:2.46.0:compile
   [INFO] |  +- org.apache.beam:beam-vendor-grpc-1_48_1:jar:0.1:compile
   [INFO] |  +- org.apache.beam:beam-vendor-guava-26_0-jre:jar:0.1:compile
   [INFO] |  +- net.bytebuddy:byte-buddy:jar:1.12.14:compile
   [INFO] |  +- com.google.code.findbugs:jsr305:jar:3.0.2:compile
   [INFO] |  +- com.google.errorprone:error_prone_annotations:jar:2.10.0:compile
   [INFO] |  +- com.fasterxml.jackson.core:jackson-core:jar:2.14.1:compile
   [INFO] |  +- com.fasterxml.jackson.core:jackson-annotations:jar:2.14.1:compile
   [INFO] |  +- com.fasterxml.jackson.core:jackson-databind:jar:2.14.1:compile
   [INFO] |  +- org.apache.avro:avro:jar:1.8.2:compile
   [INFO] |  |  +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
   [INFO] |  |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
   [INFO] |  |  +- com.thoughtworks.paranamer:paranamer:jar:2.7:compile
   [INFO] |  |  +- org.apache.commons:commons-compress:jar:1.8.1:compile
   [INFO] |  |  \- org.tukaani:xz:jar:1.5:compile
   [INFO] |  +- org.xerial.snappy:snappy-java:jar:1.1.8.4:compile
   [INFO] |  \- joda-time:joda-time:jar:2.10.10:compile
   [INFO] +- org.apache.beam:beam-sdks-java-extensions-sql:jar:2.46.0:compile
   [INFO] |  +- org.checkerframework:checker-qual:jar:3.27.0:compile
   [INFO] |  +- org.apache.beam:beam-sdks-java-extensions-avro:jar:2.46.0:compile
   [INFO] |  +- org.apache.beam:beam-sdks-java-extensions-join-library:jar:2.46.0:compile
   [INFO] |  +- org.apache.beam:beam-sdks-java-extensions-sql-udf:jar:2.46.0:compile
   [INFO] |  +- commons-codec:commons-codec:jar:1.15:compile
   [INFO] |  +- org.apache.commons:commons-csv:jar:1.8:compile
   [INFO] |  +- org.apache.beam:beam-vendor-calcite-1_28_0:jar:0.2:compile
   [INFO] |  +- com.alibaba:fastjson:jar:1.2.69:compile
   [INFO] |  +- org.codehaus.janino:janino:jar:3.0.11:compile
   [INFO] |  +- org.codehaus.janino:commons-compiler:jar:3.0.11:compile
   [INFO] |  +- org.mongodb:mongo-java-driver:jar:3.12.11:compile
   [INFO] |  +- org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:jar:2.46.0:compile
   [INFO] |  |  +- com.google.cloud.bigdataoss:gcsio:jar:2.2.6:compile
   [INFO] |  |  |  +- com.google.api.grpc:grpc-google-cloud-storage-v2:jar:2.2.2-alpha:compile
   [INFO] |  |  |  |  \- com.google.api.grpc:proto-google-cloud-storage-v2:jar:2.2.2-alpha:compile
   [INFO] |  |  |  +- io.opencensus:opencensus-impl:jar:0.31.0:compile
   [INFO] |  |  |  |  +- io.opencensus:opencensus-impl-core:jar:0.31.0:compile
   [INFO] |  |  |  |  \- com.lmax:disruptor:jar:3.4.2:compile
   [INFO] |  |  |  +- io.opencensus:opencensus-exporter-stats-stackdriver:jar:0.31.0:compile
   [INFO] |  |  |  |  +- io.opencensus:opencensus-contrib-exemplar-util:jar:0.31.0:compile
   [INFO] |  |  |  |  +- io.opencensus:opencensus-contrib-resource-util:jar:0.31.0:compile
   [INFO] |  |  |  |  \- io.opencensus:opencensus-exporter-metrics-util:jar:0.31.0:compile
   [INFO] |  |  |  \- io.opencensus:opencensus-contrib-grpc-metrics:jar:0.31.0:compile
   [INFO] |  |  +- com.google.apis:google-api-services-cloudresourcemanager:jar:v1-rev20220828-2.0.0:compile
   [INFO] |  |  +- org.apache.httpcomponents:httpcore:jar:4.4.14:compile
   [INFO] |  |  \- org.apache.httpcomponents:httpclient:jar:4.5.13:compile
   [INFO] |  +- org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.46.0:compile
   [INFO] |  |  +- org.apache.beam:beam-sdks-java-expansion-service:jar:2.46.0:compile
   [INFO] |  |  |  \- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.14.1:compile
   [INFO] |  |  |     \- org.yaml:snakeyaml:jar:1.33:compile
   [INFO] |  |  +- org.apache.beam:beam-sdks-java-extensions-protobuf:jar:2.46.0:compile
   [INFO] |  |  +- org.apache.beam:beam-sdks-java-extensions-arrow:jar:2.46.0:compile
   [INFO] |  |  +- com.google.api:gax:jar:2.23.0:compile
   [INFO] |  |  +- com.google.api:gax-grpc:jar:2.23.0:compile
   [INFO] |  |  +- com.google.api:gax-grpc:jar:testlib:2.23.0:compile
   [INFO] |  |  +- com.google.api:gax-httpjson:jar:0.108.0:compile
   [INFO] |  |  +- com.google.api:api-common:jar:2.6.0:compile
   [INFO] |  |  +- com.google.apis:google-api-services-healthcare:jar:v1-rev20220818-2.0.0:compile
   [INFO] |  |  +- com.google.apis:google-api-services-pubsub:jar:v1-rev20220904-2.0.0:compile
   [INFO] |  |  +- com.google.cloud:google-cloud-bigquerystorage:jar:2.31.0:compile
   [INFO] |  |  |  +- com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:jar:0.155.0:compile
   [INFO] |  |  |  +- com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:jar:0.155.0:compile
   [INFO] |  |  |  +- org.json:json:jar:20220924:compile
   [INFO] |  |  |  +- com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:jar:0.155.0:compile
   [INFO] |  |  |  +- com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:jar:0.155.0:compile
   [INFO] |  |  |  \- com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:jar:2.31.0:compile
   [INFO] |  |  +- com.google.cloud.bigtable:bigtable-client-core:jar:1.26.3:compile
   [INFO] |  |  |  +- com.google.api.grpc:grpc-google-common-protos:jar:2.3.2:compile
   [INFO] |  |  |  +- com.google.api.grpc:grpc-google-cloud-bigtable-v2:jar:1.27.1:compile
   [INFO] |  |  |  +- com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:jar:1.27.1:compile
   [INFO] |  |  |  +- com.google.cloud:google-cloud-core-http:jar:1.95.4:compile
   [INFO] |  |  |  |  \- com.google.http-client:google-http-client-appengine:jar:1.39.2:compile
   [INFO] |  |  |  +- com.google.cloud.bigtable:bigtable-metrics-api:jar:1.26.3:compile
   [INFO] |  |  |  +- io.opencensus:opencensus-contrib-grpc-util:jar:0.28.0:compile
   [INFO] |  |  |  \- io.dropwizard.metrics:metrics-core:jar:3.1.2:compile
   [INFO] |  |  +- com.google.cloud:google-cloud-bigtable:jar:2.19.0:compile
   [INFO] |  |  |  +- com.google.cloud:google-cloud-bigtable-stats:jar:2.19.0:compile
   [INFO] |  |  |  +- io.grpc:grpc-rls:jar:1.52.1:runtime
   [INFO] |  |  |  +- com.google.cloud:google-cloud-monitoring:jar:3.11.0:compile
   [INFO] |  |  |  \- com.google.api.grpc:proto-google-cloud-monitoring-v3:jar:3.11.0:runtime
   [INFO] |  |  +- com.google.cloud:google-cloud-core:jar:2.10.0:compile
   [INFO] |  |  +- com.google.cloud:google-cloud-core-grpc:jar:2.10.0:compile
   [INFO] |  |  +- com.google.cloud.datastore:datastore-v1-proto-client:jar:2.9.0:compile
   [INFO] |  |  |  \- com.google.http-client:google-http-client-protobuf:jar:1.41.8:compile
   [INFO] |  |  +- com.google.cloud:google-cloud-firestore:jar:3.7.10:compile
   [INFO] |  |  |  \- com.google.cloud:proto-google-cloud-firestore-bundle-v1:jar:3.7.10:compile
   [INFO] |  |  +- com.google.cloud:google-cloud-pubsub:jar:1.123.2:compile
   [INFO] |  |  +- com.google.cloud:google-cloud-spanner:jar:6.36.0:compile
   [INFO] |  |  |  +- com.google.cloud:grpc-gcp:jar:1.3.2:compile
   [INFO] |  |  |  +- com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:jar:6.36.0:compile
   [INFO] |  |  |  +- com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:jar:6.36.0:compile
   [INFO] |  |  |  +- com.google.api.grpc:grpc-google-cloud-spanner-v1:jar:6.36.0:compile
   [INFO] |  |  |  \- com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:jar:6.36.0:compile
   [INFO] |  |  +- com.google.code.gson:gson:jar:2.9.1:compile
   [INFO] |  |  +- com.google.oauth-client:google-oauth-client:jar:1.34.1:compile
   [INFO] |  |  +- io.grpc:grpc-alts:jar:1.52.1:compile
   [INFO] |  |  +- io.grpc:grpc-api:jar:1.52.1:compile
   [INFO] |  |  +- io.grpc:grpc-auth:jar:1.52.1:compile
   [INFO] |  |  +- io.grpc:grpc-core:jar:1.52.1:compile
   [INFO] |  |  +- io.grpc:grpc-census:jar:1.52.1:compile
   [INFO] |  |  +- io.grpc:grpc-context:jar:1.52.1:compile
   [INFO] |  |  +- io.grpc:grpc-grpclb:jar:1.52.1:compile
   [INFO] |  |  +- io.grpc:grpc-netty:jar:1.52.1:compile
   [INFO] |  |  |  +- io.netty:netty-codec-http2:jar:4.1.79.Final:compile
   [INFO] |  |  |  |  \- io.netty:netty-codec-http:jar:4.1.79.Final:compile
   [INFO] |  |  |  +- io.netty:netty-handler-proxy:jar:4.1.79.Final:runtime
   [INFO] |  |  |  |  \- io.netty:netty-codec-socks:jar:4.1.79.Final:runtime
   [INFO] |  |  |  \- io.netty:netty-transport-native-unix-common:jar:4.1.79.Final:runtime
   [INFO] |  |  +- io.grpc:grpc-netty-shaded:jar:1.52.1:compile
   [INFO] |  |  +- io.grpc:grpc-protobuf:jar:1.52.1:compile
   [INFO] |  |  +- io.grpc:grpc-stub:jar:1.52.1:compile
   [INFO] |  |  +- io.grpc:grpc-xds:jar:1.52.1:compile
   [INFO] |  |  +- com.google.api.grpc:grpc-google-cloud-pubsub-v1:jar:1.105.2:compile
   [INFO] |  |  +- com.google.api.grpc:grpc-google-cloud-pubsublite-v1:jar:1.10.0:compile
   [INFO] |  |  +- com.google.guava:guava:jar:31.1-jre:compile
   [INFO] |  |  +- com.fasterxml.jackson.datatype:jackson-datatype-joda:jar:2.14.1:compile
   [INFO] |  |  +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.14.1:compile
   [INFO] |  |  +- io.netty:netty-handler:jar:4.1.77.Final:compile
   [INFO] |  |  |  +- io.netty:netty-common:jar:4.1.77.Final:compile
   [INFO] |  |  |  +- io.netty:netty-resolver:jar:4.1.77.Final:compile
   [INFO] |  |  |  +- io.netty:netty-buffer:jar:4.1.77.Final:compile
   [INFO] |  |  |  +- io.netty:netty-transport:jar:4.1.77.Final:compile
   [INFO] |  |  |  \- io.netty:netty-codec:jar:4.1.77.Final:compile
   [INFO] |  |  +- io.netty:netty-tcnative-boringssl-static:jar:2.0.52.Final:compile
   [INFO] |  |  |  +- io.netty:netty-tcnative-classes:jar:2.0.52.Final:compile
   [INFO] |  |  |  +- io.netty:netty-tcnative-boringssl-static:jar:linux-x86_64:2.0.52.Final:compile
   [INFO] |  |  |  +- io.netty:netty-tcnative-boringssl-static:jar:linux-aarch_64:2.0.52.Final:compile
   [INFO] |  |  |  +- io.netty:netty-tcnative-boringssl-static:jar:osx-x86_64:2.0.52.Final:compile
   [INFO] |  |  |  +- io.netty:netty-tcnative-boringssl-static:jar:osx-aarch_64:2.0.52.Final:compile
   [INFO] |  |  |  \- io.netty:netty-tcnative-boringssl-static:jar:windows-x86_64:2.0.52.Final:compile
   [INFO] |  |  +- com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:jar:2.31.0:compile
   [INFO] |  |  +- com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:jar:2.19.0:compile
   [INFO] |  |  +- com.google.api.grpc:proto-google-cloud-datastore-v1:jar:0.104.4:compile
   [INFO] |  |  +- com.google.api.grpc:proto-google-cloud-firestore-v1:jar:3.7.10:compile
   [INFO] |  |  +- com.google.api.grpc:proto-google-cloud-pubsub-v1:jar:1.105.2:compile
   [INFO] |  |  +- com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:jar:6.36.0:compile
   [INFO] |  |  +- com.google.api.grpc:proto-google-cloud-spanner-v1:jar:6.36.0:compile
   [INFO] |  |  +- com.google.api.grpc:proto-google-common-protos:jar:2.14.0:compile
   [INFO] |  |  +- org.apache.arrow:arrow-memory-core:jar:5.0.0:compile
   [INFO] |  |  +- org.apache.arrow:arrow-vector:jar:5.0.0:compile
   [INFO] |  |  |  +- org.apache.arrow:arrow-format:jar:5.0.0:compile
   [INFO] |  |  |  \- com.google.flatbuffers:flatbuffers-java:jar:1.12.0:compile
   [INFO] |  |  +- com.google.http-client:google-http-client-gson:jar:1.41.2:compile
   [INFO] |  |  \- org.threeten:threetenbp:jar:1.4.4:compile
   [INFO] |  +- com.google.api.grpc:proto-google-cloud-bigtable-v2:jar:2.19.0:compile
   [INFO] |  |  +- com.google.auto.value:auto-value-annotations:jar:1.10.1:compile
   [INFO] |  |  +- com.google.auto.value:auto-value:jar:1.10.1:compile
   [INFO] |  |  +- javax.annotation:javax.annotation-api:jar:1.3.2:compile
   [INFO] |  |  +- com.google.guava:failureaccess:jar:1.0.1:compile
   [INFO] |  |  +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile
   [INFO] |  |  \- com.google.j2objc:j2objc-annotations:jar:1.3:compile
   [INFO] |  +- com.google.apis:google-api-services-bigquery:jar:v2-rev20220924-2.0.0:compile
   [INFO] |  +- com.google.api.grpc:proto-google-cloud-pubsublite-v1:jar:1.10.0:compile
   [INFO] |  +- com.google.cloud:google-cloud-pubsublite:jar:1.10.0:compile
   [INFO] |  |  +- io.opencensus:opencensus-api:jar:0.31.1:compile
   [INFO] |  |  +- io.grpc:grpc-protobuf-lite:jar:1.52.1:compile
   [INFO] |  |  +- com.google.api.grpc:proto-google-iam-v1:jar:1.9.0:compile
   [INFO] |  |  +- io.grpc:grpc-googleapis:jar:1.52.1:runtime
   [INFO] |  |  +- io.opencensus:opencensus-proto:jar:0.2.0:compile
   [INFO] |  |  +- io.grpc:grpc-services:jar:1.52.1:compile
   [INFO] |  |  +- com.google.re2j:re2j:jar:1.6:compile
   [INFO] |  |  +- com.google.android:annotations:jar:4.1.1.4:runtime
   [INFO] |  |  +- org.codehaus.mojo:animal-sniffer-annotations:jar:1.22:runtime
   [INFO] |  |  +- io.perfmark:perfmark-api:jar:0.26.0:runtime
   [INFO] |  |  +- commons-logging:commons-logging:jar:1.2:compile
   [INFO] |  |  +- io.opencensus:opencensus-contrib-http-util:jar:0.31.1:compile
   [INFO] |  |  +- com.google.flogger:google-extensions:jar:0.7.4:compile
   [INFO] |  |  +- com.google.flogger:flogger:jar:0.7.4:compile
   [INFO] |  |  \- org.checkerframework:checker-compat-qual:jar:2.5.3:compile
   [INFO] |  +- org.apache.beam:beam-sdks-java-io-mongodb:jar:2.46.0:compile
   [INFO] |  +- com.google.protobuf:protobuf-java:jar:3.21.12:compile
   [INFO] |  \- com.google.protobuf:protobuf-java-util:jar:3.21.12:compile
   [INFO] +- org.slf4j:slf4j-api:jar:1.7.30:compile
   [INFO] +- org.slf4j:slf4j-jdk14:jar:1.7.30:runtime
   [INFO] +- org.hamcrest:hamcrest-core:jar:2.1:compile
   [INFO] |  \- org.hamcrest:hamcrest:jar:2.1:compile
   [INFO] +- org.hamcrest:hamcrest-library:jar:2.1:compile
   [INFO] +- junit:junit:jar:4.13-beta-3:compile
   [INFO] +- org.apache.beam:beam-runners-direct-java:jar:2.46.0:runtime
   [INFO] +- org.mockito:mockito-core:jar:3.7.7:test
   [INFO] |  +- net.bytebuddy:byte-buddy-agent:jar:1.10.19:test
   [INFO] |  \- org.objenesis:objenesis:jar:3.1:runtime
   [INFO] +- org.apache.beam:beam-runners-portability-java:jar:2.46.0:runtime
   [INFO] |  +- org.apache.beam:beam-runners-java-fn-execution:jar:2.46.0:compile
   [INFO] |  +- org.apache.beam:beam-runners-java-job-service:jar:2.46.0:runtime
   [INFO] |  \- org.apache.beam:beam-sdks-java-harness:jar:2.46.0:runtime
   [INFO] +- org.apache.beam:beam-runners-flink-1.14:jar:2.46.0:runtime
   [INFO] |  +- org.apache.beam:beam-runners-core-java:jar:2.46.0:compile
   [INFO] |  +- org.apache.beam:beam-runners-core-construction-java:jar:2.46.0:compile
   [INFO] |  |  \- io.github.classgraph:classgraph:jar:4.8.104:compile
   [INFO] |  +- args4j:args4j:jar:2.33:runtime
   [INFO] |  +- org.apache.flink:flink-clients_2.12:jar:1.14.3:runtime
   [INFO] |  |  +- commons-cli:commons-cli:jar:1.3.1:runtime
   [INFO] |  |  \- org.apache.flink:flink-shaded-force-shading:jar:14.0:runtime
   [INFO] |  +- org.apache.flink:flink-streaming-java_2.12:jar:1.14.3:runtime
   [INFO] |  |  +- org.apache.flink:flink-file-sink-common:jar:1.14.3:runtime
   [INFO] |  |  +- org.apache.flink:flink-scala_2.12:jar:1.14.3:runtime
   [INFO] |  |  |  +- org.scala-lang:scala-reflect:jar:2.12.7:runtime
   [INFO] |  |  |  +- org.scala-lang:scala-library:jar:2.12.7:runtime
   [INFO] |  |  |  +- org.scala-lang:scala-compiler:jar:2.12.7:runtime
   [INFO] |  |  |  |  \- org.scala-lang.modules:scala-xml_2.12:jar:1.0.6:runtime
   [INFO] |  |  |  \- com.twitter:chill_2.12:jar:0.7.6:runtime
   [INFO] |  |  |     \- com.twitter:chill-java:jar:0.7.6:runtime
   [INFO] |  |  +- org.apache.flink:flink-shaded-guava:jar:30.1.1-jre-14.0:runtime
   [INFO] |  |  \- org.apache.commons:commons-math3:jar:3.5:runtime
   [INFO] |  +- org.apache.flink:flink-core:jar:1.14.3:runtime
   [INFO] |  |  +- org.apache.flink:flink-annotations:jar:1.14.3:runtime
   [INFO] |  |  +- org.apache.flink:flink-shaded-asm-7:jar:7.1-14.0:runtime
   [INFO] |  |  +- org.apache.commons:commons-lang3:jar:3.3.2:runtime
   [INFO] |  |  +- com.esotericsoftware.kryo:kryo:jar:2.24.0:runtime
   [INFO] |  |  |  \- com.esotericsoftware.minlog:minlog:jar:1.2:runtime
   [INFO] |  |  \- commons-collections:commons-collections:jar:3.2.2:runtime
   [INFO] |  +- org.apache.flink:flink-metrics-core:jar:1.14.3:runtime
   [INFO] |  +- org.apache.flink:flink-java:jar:1.14.3:runtime
   [INFO] |  +- org.apache.flink:flink-runtime:jar:1.14.3:runtime
   [INFO] |  |  +- org.apache.flink:flink-rpc-core:jar:1.14.3:runtime
   [INFO] |  |  +- org.apache.flink:flink-rpc-akka-loader:jar:1.14.3:runtime
   [INFO] |  |  +- org.apache.flink:flink-queryable-state-client-java:jar:1.14.3:runtime
   [INFO] |  |  +- org.apache.flink:flink-hadoop-fs:jar:1.14.3:runtime
   [INFO] |  |  +- commons-io:commons-io:jar:2.8.0:runtime
   [INFO] |  |  +- org.apache.flink:flink-shaded-netty:jar:4.1.65.Final-14.0:runtime
   [INFO] |  |  +- org.apache.flink:flink-shaded-jackson:jar:2.12.4-14.0:runtime
   [INFO] |  |  +- org.apache.flink:flink-shaded-zookeeper-3:jar:3.4.14-14.0:runtime
   [INFO] |  |  +- org.javassist:javassist:jar:3.24.0-GA:runtime
   [INFO] |  |  \- org.lz4:lz4-java:jar:1.8.0:runtime
   [INFO] |  +- org.apache.flink:flink-optimizer:jar:1.14.3:runtime
   [INFO] |  +- org.apache.beam:beam-model-fn-execution:jar:2.46.0:compile
   [INFO] |  \- org.apache.beam:beam-sdks-java-fn-execution:jar:2.46.0:compile
   [INFO] \- org.apache.beam:beam-runners-google-cloud-dataflow-java:jar:2.46.0:runtime
   [INFO]    +- org.apache.beam:beam-sdks-java-io-kafka:jar:2.46.0:runtime
   [INFO]    |  +- org.springframework:spring-expression:jar:5.3.25:runtime
   [INFO]    |  |  \- org.springframework:spring-core:jar:5.3.25:runtime
   [INFO]    |  |     \- org.springframework:spring-jcl:jar:5.3.25:runtime
   [INFO]    |  +- io.confluent:kafka-avro-serializer:jar:5.3.2:runtime
   [INFO]    |  |  +- io.confluent:common-config:jar:5.3.2:runtime
   [INFO]    |  |  \- io.confluent:common-utils:jar:5.3.2:runtime
   [INFO]    |  |     \- com.101tec:zkclient:jar:0.10:runtime
   [INFO]    |  \- io.confluent:kafka-schema-registry-client:jar:5.3.2:runtime
   [INFO]    +- com.google.cloud.bigdataoss:util:jar:2.2.6:compile
   [INFO]    |  +- com.google.api-client:google-api-client-jackson2:jar:1.32.2:compile
   [INFO]    |  \- com.google.apis:google-api-services-iamcredentials:jar:v1-rev20210326-1.32.1:compile
   [INFO]    +- com.google.flogger:flogger-system-backend:jar:0.7.3:compile
   [INFO]    +- com.google.api-client:google-api-client:jar:2.0.0:compile
   [INFO]    |  \- com.google.http-client:google-http-client-apache-v2:jar:1.42.1:compile
   [INFO]    +- com.google.apis:google-api-services-clouddebugger:jar:v2-rev20220318-2.0.0:runtime
   [INFO]    +- com.google.apis:google-api-services-dataflow:jar:v1b3-rev20220920-2.0.0:runtime
   [INFO]    +- com.google.apis:google-api-services-storage:jar:v1-rev20220705-2.0.0:compile
   [INFO]    +- com.google.auth:google-auth-library-credentials:jar:1.15.0:compile
   [INFO]    +- com.google.auth:google-auth-library-oauth2-http:jar:1.15.0:compile
   [INFO]    +- com.google.http-client:google-http-client:jar:1.42.3:compile
   [INFO]    \- com.google.http-client:google-http-client-jackson2:jar:1.42.3:compile
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] Abacn commented on issue #26403: Update vendored calcite to eliminate vulnerability from shaded log4j:1.2.17 and protobuf-java:3.19.2

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #26403:
URL: https://github.com/apache/beam/issues/26403#issuecomment-1522231293

   vendor calcite 1.28 should not have shaded 1og4j. It is excluded: 
   
   https://github.com/apache/beam/blob/1a51ba5edebef9622c016331071764830adffc94/vendor/calcite-1_28_0/build.gradle#L94
   
    org.apache.beam.vendor.calcite.v1_28_0.com.google.common.collect.ImmutableList is a misuse. It should be
   
   org.apache.beam.vendor.guava.v26_xxx.com.google.common.collect.ImmutableList
   
   Let me do an audit of `com.google.common` that is not under vendor guava space.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Update vendored calcite to eliminate vulnerability from shaded log4j:1.2.17 and protobuf-java:3.19.2 [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn closed issue #26403: Update vendored calcite to eliminate vulnerability from shaded log4j:1.2.17 and protobuf-java:3.19.2
URL: https://github.com/apache/beam/issues/26403


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] oleksandr-mashchenko-db commented on issue #26403: Update vendored calcite to eliminate vulnerability from shaded log4j:1.2.17 and protobuf-java:3.19.2

Posted by "oleksandr-mashchenko-db (via GitHub)" <gi...@apache.org>.
oleksandr-mashchenko-db commented on issue #26403:
URL: https://github.com/apache/beam/issues/26403#issuecomment-1525145571

   Thank you for the feedback, @Abacn. 
   Did you have a chance to perform audit of com.google.common ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Update vendored calcite to eliminate vulnerability from shaded log4j:1.2.17 and protobuf-java:3.19.2 [beam]

Posted by "DebajitKumarPhukan (via GitHub)" <gi...@apache.org>.
DebajitKumarPhukan commented on issue #26403:
URL: https://github.com/apache/beam/issues/26403#issuecomment-1779755921

   @Abacn FYI, below "critical" vulnerabilities still exist and are getting blocked by JFrog XRay scans.
   
   [org.yaml:snakeyaml:1.24](https://mvnrepository.com/artifact/org.yaml/snakeyaml/1.24)
   [com.fasterxml.jackson.core:jackson-databind:2.10.0](https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind/2.10.0)
   
   **Apache Beam SQL Extension Version**
   ```
   <dependency>
       <groupId>org.apache.beam</groupId>
       <artifactId>beam-sdks-java-extensions-sql</artifactId>
       <version>2.51.0</version>
   </dependency>
   ```
   
   **Which Internally Uses Below Version**
   ```
   <dependency>
       <groupId>org.apache.beam</groupId>
       <artifactId>beam-vendor-calcite-1_28_0</artifactId>
       <version>0.2</version>
   </dependency>
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Update vendored calcite to eliminate vulnerability from shaded log4j:1.2.17 and protobuf-java:3.19.2 [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #26403:
URL: https://github.com/apache/beam/issues/26403#issuecomment-1781100820

   > @Abacn FYI, below "critical" vulnerabilities still exist along with the `log4j`  vulnerability and are getting blocked by JFrog XRay scans.
   > 
   > 
   > 
   > [org.yaml:snakeyaml:1.24](https://mvnrepository.com/artifact/org.yaml/snakeyaml/1.24)
   > 
   > [com.fasterxml.jackson.core:jackson-databind:2.10.0](https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind/2.10.0)
   > 
   > 
   > 
   > **Apache Beam SQL Extension Version**
   > 
   > ```
   > 
   > <dependency>
   > 
   >     <groupId>org.apache.beam</groupId>
   > 
   >     <artifactId>beam-sdks-java-extensions-sql</artifactId>
   > 
   >     <version>2.51.0</version>
   > 
   > </dependency>
   > 
   > ```
   > 
   > 
   > 
   > **Which Internally Uses Below Version**
   > 
   > ```
   > 
   > <dependency>
   > 
   >     <groupId>org.apache.beam</groupId>
   > 
   >     <artifactId>beam-vendor-calcite-1_28_0</artifactId>
   > 
   >     <version>0.2</version>
   > 
   > </dependency>
   > 
   > ```
   
   Thanks for reminding, both jackson and snakeYAML vulnerabilities are indeed well-known and should be fixed by upgrades. The obstacle is that upgrading calcite for beam SQL is known to be nontrivial due to breaking changes (happened last time vendor upgrade) so it is expected to take more effort than the other two vendor dependency beam currently does. We currently do not have bandwidth and encourage contributors could test with new calcite versions.
   
   Generally, vendor calcite and its dependency are shaded and only used by beam SQL artifact and vulnerabilities usually not exploitable (disclaimer - not professional opinion about security). Other part of beam and user code do not depend on the dependencies of vulnerabilities. e.g. Beam Java core depends on more recent version of jackson databind. 
   
   Nevertheless it should be possible to upgrade these transitive dependencies and keep calcite version which requires much less effort. Could initiate that when I got time.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] Abacn commented on issue #26403: Update vendored calcite to eliminate vulnerability from shaded log4j:1.2.17 and protobuf-java:3.19.2

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #26403:
URL: https://github.com/apache/beam/issues/26403#issuecomment-1533798942

   > Did you have a chance to perform audit of com.google.common ?
   
   Yes, done in #26463. The change will be available in Beam v2.48.0. Though I don't think this resolves the main issue of this Issue. Dot the log4j dependency, users should be able to overwrite the dependency to a newer version because it is not shaded in vendor-calcite-1.28.0. Will confirm.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org