You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@dolphinscheduler.apache.org by zixi0825 <64...@qq.com> on 2021/02/18 11:54:22 UTC
[VOTE][New Jar] spark-core,spark-sql,spark-hive(data quality)
Hi,DolphinScheduler Community
The data quality module uses spark as the execution engine. Therefore, we need to introduce some new jars (spark-core, spark-sql, spark-hive), which complies with the Apache-2.0 License. I checked their related dependencies to make sure it complies with the license of the Apache project.
new jar :
org.apache.spark:spark-core_2.11 version: 2.4.0 license: Apache 2.0
org.apache.spark:spark-sql_2.11 version: 2.4.0 license: Apache 2.0
org.apache.spark:spark-hive_2.11 version: 2.4.0 license: Apache 2.0
org.apache.spark:spark-core_2.11 dependent jar
Apache 2.0
------------------------------------------------------
com.clearspring.analytics:stream
com.fasterxml.jackson.core:jackson-databind
com.fasterxml.jackson.module:jackson-module-scala_2.11
com.google.code.findbugs:jsr305
com.ning:compress-lzf
com.twitter:chill_2.11
com.twitter:chill-java
commons-net:commons-net
io.dropwizard.metrics:metrics-core
io.dropwizard.metrics:metrics-graphite
io.dropwizard.metrics:metrics-json
io.dropwizard.metrics:metrics-jvm
io.netty:netty-all
io.netty:netty
log4j:log4j
org.apache.avro:avro
org.apache.avro:avro-mapred
org.apache.commons:commons-math3
org.apache.commons:commons-crypto
org.apache.commons:commons-lang3
org.apache.curator:curator-recipes
org.apache.hadoop:hadoop-client
org.apache.ivy:ivy
org.apache.spark:spark-kvstore_2.11
org.apache.spark:spark-launcher_2.11
org.apache.spark:spark-network-common_2.11
org.apache.spark:spark-network-shuffle_2.11
org.apache.spark:spark-tags_2.11
org.apache.spark:spark-unsafe_2.11
org.apache.xbean:xbean-asm6-shaded
org.apache.zookeeper:zookeeper
org.json4s:json4s-jackson_2.11
org.lz4:lz4-java
org.roaringbitmap:RoaringBitmap
org.xerial.snappy:snappy-java
oro:oro
MIT
------------------------------------------------------
net.razorvine:pyrolite
org.slf4j:jcl-over-slf4j
org.slf4j:jul-to-slf4j
org.slf4j:slf4j-api
org.slf4j:slf4j-log4j12
Common Development and Distribution License (CDDL) 1.1
------------------------------------------------------
org.glassfish.jersey.containers:jersey-container-servlet
org.glassfish.jersey.containers:jersey-container-servlet-core
org.glassfish.jersey.core:jersey-client
org.glassfish.jersey.core:jersey-common
org.glassfish.jersey.core:jersey-server
Common Development and Distribution License (CDDL) 1.0
------------------------------------------------------
javax.activation:activation
javax.servlet:javax.servlet-api
BSD 2-Clause
------------------------------------------------------
com.github.luben:zstd-jni
net.sf.py4j:py4j
BSD 3-Clause
------------------------------------------------------
org.scala-lang:scala-library
org.apache.spark:spark-sql_2.11 dependent jar
Apache 2.0
------------------------------------------------------
com.fasterxml.jackson.core:jackson-databind
com.univocity:univocity-parsers
org.apache.arrow:arrow-vector
org.apache.orc:orc-core
org.apache.orc:orc-mapreduce
org.apache.parquet:parquet-column
org.apache.parquet:parquet-hadoop
org.apache.spark:spark-sketch_2.11
org.apache.spark:spark-catalyst_2.11
org.apache.spark:spark-core_2.11
org.apache.spark:spark-tags_2.11
org.apache.xbean:xbean-asm6-shaded
org.eclipse.jetty:jetty-servlet
org.apache.spark:spark-hive_2.11 dependent jar
Apache 2.0
------------------------------------------------------
com.google.code.findbugs:jsr305
com.twitter:parquet-hadoop-bundle
commons-codec:commons-codec
commons-httpclient:commons-httpclient
joda-time:joda-time
org.apache.avro:avro
org.apache.avro:avro-mapred
org.apache.calcite:calcite-avatica version -1.2.0-incubating
org.apache.calcite:calcite-core version -1.2.0-incubating
org.apache.derby:derby
org.apache.httpcomponents:httpclient
org.apache.spark:spark-sql_2.11
org.apache.spark:spark-core_2.11
org.apache.thrift:libfb303
org.apache.thrift:libthrift
org.codehaus.jackson:jackson-mapper-asl
org.datanucleus:datanucleus-core
org.apache.hive:hive-exec
org.apache.hive:hive-metastore
BSD 2-Clause
------------------------------------------------------
org.jodd:jodd-core
Github address:
https://github.com/apache/spark
License:
https://github.com/apache/spark/blob/master/LICENSE
Notice:
https://github.com/apache/spark/blob/master/NOTICE
Maven repository:
https://mvnrepository.com/artifact/org.apache.spark/spark-core
https://mvnrepository.com/artifact/org.apache.spark/spark-sql
https://mvnrepository.com/artifact/org.apache.spark/spark-hive
The vote will be open for at least 72 hours or until necessary number of votes are reached.
Please vote accordingly:
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason
Re:[VOTE][New Jar] spark-core,spark-sql,spark-hive(data quality)
Posted by zixi0825 <64...@qq.com>.
The email needs to be updated and has been closed.
------------------ Original ------------------
From: "zixi0825" <649790970@qq.com>;
Date: Thu, Feb 18, 2021 07:54 PM
To: "dev"<dev@dolphinscheduler.apache.org>;
Subject: [VOTE][New Jar] spark-core,spark-sql,spark-hive(data quality)
Hi,DolphinScheduler Community
The data quality module uses spark as the execution engine. Therefore, we need to introduce some new jars (spark-core, spark-sql, spark-hive), which complies with the Apache-2.0 License. I checked their related dependencies to make sure it complies with the license of the Apache project.
new jar :
org.apache.spark:spark-core_2.11 version: 2.4.0 license: Apache 2.0
org.apache.spark:spark-sql_2.11 version: 2.4.0 license: Apache 2.0
org.apache.spark:spark-hive_2.11 version: 2.4.0 license: Apache 2.0
org.apache.spark:spark-core_2.11 dependent jar
Apache 2.0
------------------------------------------------------
com.clearspring.analytics:stream
com.fasterxml.jackson.core:jackson-databind
com.fasterxml.jackson.module:jackson-module-scala_2.11
com.google.code.findbugs:jsr305
com.ning:compress-lzf
com.twitter:chill_2.11
com.twitter:chill-java
commons-net:commons-net
io.dropwizard.metrics:metrics-core
io.dropwizard.metrics:metrics-graphite
io.dropwizard.metrics:metrics-json
io.dropwizard.metrics:metrics-jvm
io.netty:netty-all
io.netty:netty
log4j:log4j
org.apache.avro:avro
org.apache.avro:avro-mapred
org.apache.commons:commons-math3
org.apache.commons:commons-crypto
org.apache.commons:commons-lang3
org.apache.curator:curator-recipes
org.apache.hadoop:hadoop-client
org.apache.ivy:ivy
org.apache.spark:spark-kvstore_2.11
org.apache.spark:spark-launcher_2.11
org.apache.spark:spark-network-common_2.11
org.apache.spark:spark-network-shuffle_2.11
org.apache.spark:spark-tags_2.11
org.apache.spark:spark-unsafe_2.11
org.apache.xbean:xbean-asm6-shaded
org.apache.zookeeper:zookeeper
org.json4s:json4s-jackson_2.11
org.lz4:lz4-java
org.roaringbitmap:RoaringBitmap
org.xerial.snappy:snappy-java
oro:oro
MIT
------------------------------------------------------
net.razorvine:pyrolite
org.slf4j:jcl-over-slf4j
org.slf4j:jul-to-slf4j
org.slf4j:slf4j-api
org.slf4j:slf4j-log4j12
Common Development and Distribution License (CDDL) 1.1
------------------------------------------------------
org.glassfish.jersey.containers:jersey-container-servlet
org.glassfish.jersey.containers:jersey-container-servlet-core
org.glassfish.jersey.core:jersey-client
org.glassfish.jersey.core:jersey-common
org.glassfish.jersey.core:jersey-server
Common Development and Distribution License (CDDL) 1.0
------------------------------------------------------
javax.activation:activation
javax.servlet:javax.servlet-api
BSD 2-Clause
------------------------------------------------------
com.github.luben:zstd-jni
net.sf.py4j:py4j
BSD 3-Clause
------------------------------------------------------
org.scala-lang:scala-library
org.apache.spark:spark-sql_2.11 dependent jar
Apache 2.0
------------------------------------------------------
com.fasterxml.jackson.core:jackson-databind
com.univocity:univocity-parsers
org.apache.arrow:arrow-vector
org.apache.orc:orc-core
org.apache.orc:orc-mapreduce
org.apache.parquet:parquet-column
org.apache.parquet:parquet-hadoop
org.apache.spark:spark-sketch_2.11
org.apache.spark:spark-catalyst_2.11
org.apache.spark:spark-core_2.11
org.apache.spark:spark-tags_2.11
org.apache.xbean:xbean-asm6-shaded
org.eclipse.jetty:jetty-servlet
org.apache.spark:spark-hive_2.11 dependent jar
Apache 2.0
------------------------------------------------------
com.google.code.findbugs:jsr305
com.twitter:parquet-hadoop-bundle
commons-codec:commons-codec
commons-httpclient:commons-httpclient
joda-time:joda-time
org.apache.avro:avro
org.apache.avro:avro-mapred
org.apache.calcite:calcite-avatica version -1.2.0-incubating
org.apache.calcite:calcite-core version -1.2.0-incubating
org.apache.derby:derby
org.apache.httpcomponents:httpclient
org.apache.spark:spark-sql_2.11
org.apache.spark:spark-core_2.11
org.apache.thrift:libfb303
org.apache.thrift:libthrift
org.codehaus.jackson:jackson-mapper-asl
org.datanucleus:datanucleus-core
org.apache.hive:hive-exec
org.apache.hive:hive-metastore
BSD 2-Clause
------------------------------------------------------
org.jodd:jodd-core
Github address:
https://github.com/apache/spark
License:
https://github.com/apache/spark/blob/master/LICENSE
Notice:
https://github.com/apache/spark/blob/master/NOTICE
Maven repository:
https://mvnrepository.com/artifact/org.apache.spark/spark-core
https://mvnrepository.com/artifact/org.apache.spark/spark-sql
https://mvnrepository.com/artifact/org.apache.spark/spark-hive
The vote will be open for at least 72 hours or until necessary number of votes are reached.
Please vote accordingly:
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason