[VOTE][New Jar] spark-core,spark-sql,spark-hive (for data quality)

[zixi0825 <64...@qq.com>]

Hi,DolphinScheduler Community

 

The data quality module uses spark as the execution engine. Therefore, we need to introduce some new jars (spark-core, spark-sql, spark-hive), which complies with the Apache-2.0 License. I checked their related dependencies to make sure it complies with the license of the Apache project.

 

new jar :

org.apache.spark:spark-core_2.11 version: 2.4.0 license: Apache 2.0

org.apache.spark:spark-sql_2.11   version: 2.4.0 license: Apache 2.0

org.apache.spark:spark-hive_2.11 version: 2.4.0 license: Apache 2.0






 

org.apache.spark:spark-core_2.11 dependent jar

 

Apache 2.0

------------------------------------------------------

com.clearspring.analytics:stream

com.fasterxml.jackson.module:jackson-module-scala_2.11

com.ning:compress-lzf

com.twitter:chill_2.11

com.twitter:chill-java

io.dropwizard.metrics:metrics-core

io.dropwizard.metrics:metrics-graphite

io.dropwizard.metrics:metrics-json

io.dropwizard.metrics:metrics-jvm

org.apache.avro:avro-mapred

org.apache.commons:commons-crypto

org.apache.commons:commons-lang3

org.apache.ivy:ivy

org.apache.spark:spark-kvstore_2.11

org.apache.spark:spark-launcher_2.11

org.apache.spark:spark-network-common_2.11

org.apache.spark:spark-network-shuffle_2.11

org.apache.spark:spark-tags_2.11

org.apache.spark:spark-unsafe_2.11

org.apache.xbean:xbean-asm6-shaded

org.json4s:json4s-jackson_2.11

org.lz4:lz4-java

org.roaringbitmap:RoaringBitmap

oro:oro

 

MIT

------------------------------------------------------

net.razorvine:pyrolite

org.slf4j:jcl-over-slf4j

org.slf4j:slf4j-log4j12

 

Common Development and Distribution License (CDDL) 1.1

------------------------------------------------------

org.glassfish.jersey.containers:jersey-container-servlet

org.glassfish.jersey.containers:jersey-container-servlet-core

org.glassfish.jersey.core:jersey-client

org.glassfish.jersey.core:jersey-common

org.glassfish.jersey.core:jersey-server

 

BSD 2-Clause

------------------------------------------------------

com.github.luben:zstd-jni

net.sf.py4j:py4j

 

BSD 3-Clause

------------------------------------------------------

org.scala-lang:scala-library






 

org.apache.spark:spark-sql_2.11 dependent jar

 

Apache 2.0

------------------------------------------------------

com.univocity:univocity-parsers

org.apache.arrow:arrow-vector

org.apache.orc:orc-core

org.apache.orc:orc-mapreduce

org.apache.parquet:parquet-column

org.apache.parquet:parquet-hadoop

org.apache.spark:spark-sketch_2.11

org.apache.spark:spark-catalyst_2.11

org.apache.spark:spark-tags_2.11

org.apache.xbean:xbean-asm6-shaded






 

org.apache.spark:spark-hive_2.11 dependent jar

 

Apache 2.0

------------------------------------------------------

com.twitter:parquet-hadoop-bundle

org.apache.calcite:calcite-avatica 

org.apache.calcite:calcite-core

org.apache.hive:hive-exec

 

BSD 2-Clause

------------------------------------------------------

org.jodd:jodd-core






 

Github address:

https://github.com/apache/spark




License:

https://github.com/apache/spark/blob/master/LICENSE




Notice:

https://github.com/apache/spark/blob/master/NOTICE




Maven repository:

https://mvnrepository.com/artifact/org.apache.spark/spark-core

https://mvnrepository.com/artifact/org.apache.spark/spark-sql

https://mvnrepository.com/artifact/org.apache.spark/spark-hive

 

The vote will be open for at least 72 hours or until necessary number of votes are reached.

Please vote accordingly:

 

[ ] +1 approve

[ ] +0 no opinion

[ ] -1 disapprove with the reason