You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@wayang.apache.org by be...@apache.org on 2021/09/28 16:30:15 UTC
svn commit: r50153 - in /dev/wayang: ./ 0.6.0/ 0.6.0/rc1/
Author: bertty
Date: Tue Sep 28 16:30:15 2021
New Revision: 50153
Log:
Staging of rc1 of Apache Wayang 0.6.0
Added:
dev/wayang/
dev/wayang/0.6.0/
dev/wayang/0.6.0/rc1/
dev/wayang/0.6.0/rc1/README.md
dev/wayang/0.6.0/rc1/RELEASE_NOTES
dev/wayang/0.6.0/rc1/wayang-0.6.0-source-release.zip (with props)
dev/wayang/0.6.0/rc1/wayang-0.6.0-source-release.zip.asc
dev/wayang/0.6.0/rc1/wayang-0.6.0-source-release.zip.sha512
dev/wayang/0.6.0/rc1/wayang-0.6.0.pom
dev/wayang/0.6.0/rc1/wayang-0.6.0.pom.asc
Added: dev/wayang/0.6.0/rc1/README.md
==============================================================================
--- dev/wayang/0.6.0/rc1/README.md (added)
+++ dev/wayang/0.6.0/rc1/README.md Tue Sep 28 16:30:15 2021
@@ -0,0 +1,367 @@
+# Apache Wayang <img align="right" width="128px" src="https://wayang.apache.org/assets/img/logo/logo_400x160.png" alt="Wayang logo">
+
+[](https://travis-ci.org/wayang-ecosystem/wayang)
+[](https://gitter.im/wayang-ecosystem/Lobby)
+[](https://maven-badges.herokuapp.com/maven-central/org.apache.wayang/wayang)
+
+#### Turning a shadows into a show
+
+
+Apache Wayang in contrast to classical data processing systems that provide one dedicated execution engine, Apache Wayang rather is a *meta processing framework*: You can specify your data processing app via one of Wayang's API and then Wayang will pick an optimal configuration of classical processing frameworks, such as Java Streams or Apache Spark, to run your app on. Finally, Wayang will also perform the execution, thereby hiding the different specific platform APIs and coordinate inter-platform communication.
+
+This approach aims at freeing data engineers and software developers from the burden of knowing the zoo of different data processing systems, their APIs, strengths and weakness; the intricacies of coordinating and integrating different processing platforms; and the inflexibility when tying to a fix set of processing platforms. As of now, Wayang has built in support for the following processing platforms:
+- Java 8 Streams
+- [Apache Spark](https://spark.apache.org/)
+- [GraphChi](https://github.com/GraphChi/graphchi-java)
+- [Postgres](http://www.postgresql.org)
+- [SQLite](https://www.sqlite.org/)
+
+## How to use Wayang
+
+**Requirements.**
+Apache Wayang is built with Java 8 and Scala 2.11. However, to execute Wayang it is sufficient to have Java 8 installed. If you want to build Wayang yourself, you will also need to have [Apache Maven](http://maven.apache.org) installed. Please also consider that processing platforms employed by Wayang might have further requirements.
+
+**Get Wayang.**
+Wayang is available via Maven Central. To use it with Maven, for instance, include the following into you POM file:
+```xml
+<dependency>â¨
+ <groupId>org.apache.wayang</groupId>
+ <artifactId>wayang-***</artifactId>
+ <version>0.3.0</version>â¨
+</dependency>
+```
+Note the `***`: Wayang ships with multiple modules that can be included in your app, depending on how you want to use it:
+* `wayang-core`: provides core data structures and the optimizer (required)
+* `wayang-basic`: provides common operators and data types for your apps (recommended)
+* `wayang-api`: provides an easy-to-use Scala and Java API to assemble Wayang plans (recommended)
+* `wayang-java`, `wayang-spark`, `wayang-graphchi`, `wayang-sqlite3`, `wayang-postgres`: adapters for the various supported processing platforms
+* `wayang-profiler`: provides functionality to learn operator and UDF cost functions from historical execution data
+
+For the sake of version flexibility, you still have to include your Hadoop (`hadoop-hdfs` and `hadoop-common`) and Spark (`spark-core` and `spark-graphx`) version of choice.
+
+In addition, you can obtain the most recent snapshot version of Wayang via Sonatype's snapshot repository. Just included
+```xml
+<repositories>
+ <repository>
+ <id>sonatype-snapshots</id>
+ <name>Sonatype Snapshot Repository</name>
+ <url>https://oss.sonatype.org/content/repositories/snapshots</url>
+ </repository>
+<repositories>
+```
+
+If you need to rebuild Wayang, e.g., to use a different Scala version, you can simply do so via Maven:
+
+1. Adapt the version variables (e.g., `spark.version`) in the main `pom.xml` file.
+2. Build Wayang with the adapted versions.
+ ```shell
+ $ mvn clean install
+ ```
+ Note the `standalone` profile to fix Hadoop and Spark versions, so that Wayang apps do not explicitly need to declare the corresponding dependencies.
+ Also, note the `distro` profile, which assembles a binary Wayang distribution.
+ To activate these profiles, you need to specify them when running maven, i.e.,
+ ```shell
+ mvn clean install -P<profile name>
+ ```
+
+**Configure Wayang.** In order for Wayang to work properly, it is necessary to tell Wayang about the capacities of your processing platforms and how to reach them. While there is a default configuration that allows to test Wayang right away, we recommend to create a properties file to adapt the configuration where necessary. To have Wayang use that configuration transparently, just run you app via
+```shell
+$ java -Dwayang.configuration=url://to/my/wayang.properties ...
+```
+
+You can find the most relevant settings in the following:
+* General settings
+ * `wayang.core.log.enabled (= true)`: whether to log execution statistics to allow learning better cardinality and cost estimators for the optimizer
+ * `wayang.core.log.executions (= ~/.wayang/executions.json)` where to log execution times of operator groups
+ * `wayang.core.log.cardinalities (= ~/.wayang/cardinalities.json)` where to log cardinality measurements
+ * `wayang.core.optimizer.instrumentation (= org.apache.wayang.core.profiling.OutboundInstrumentationStrategy)`: where to measure cardinalities in Wayang plans; other options are `org.apache.wayang.core.profiling.NoInstrumentationStrategy` and `org.apache.wayang.core.profiling.FullInstrumentationStrategy`
+ * `wayang.core.optimizer.reoptimize (= false)`: whether to progressively optimize Wayang plans
+ * `wayang.basic.tempdir (= file:///tmp)`: where to store temporary files, in particular for inter-platform communication
+* Java Streams
+ * `wayang.java.cpu.mhz (= 2700)`: clock frequency of processor the JVM runs on in MHz
+ * `wayang.java.hdfs.ms-per-mb (= 2.7)`: average throughput from HDFS to JVM in ms/MB
+* Apache Spark
+ * `spark.master (= local)`: Spark master
+ * various other Spark settings are supported, e.g., `spark.executor.memory`, `spark.serializer`, ...
+ * `wayang.spark.cpu.mhz (= 2700)`: clock frequency of processor the Spark workers run on in MHz
+ * `wayang.spark.hdfs.ms-per-mb (= 2.7)`: average throughput from HDFS to the Spark workers in ms/MB
+ * `wayang.spark.network.ms-per-mb (= 8.6)`: average network throughput of the Spark workers in ms/MB
+ * `wayang.spark.init.ms (= 4500)`: time it takes Spark to initialize in ms
+* GraphChi
+ * `wayang.graphchi.cpu.mhz (= 2700)`: clock frequency of processor GraphChi runs on in MHz
+ * `wayang.graphchi.cpu.cores (= 2)`: number of cores GraphChi runs on
+ * `wayang.graphchi.hdfs.ms-per-mb (= 2.7)`: average throughput from HDFS to GraphChi in ms/MB
+* SQLite
+ * `wayang.sqlite3.jdbc.url`: JDBC URL to use SQLite
+ * `wayang.sqlite3.jdbc.user`: optional user name
+ * `wayang.sqlite3.jdbc.password`: optional password
+ * `wayang.sqlite3.cpu.mhz (= 2700)`: clock frequency of processor SQLite runs on in MHz
+ * `wayang.sqlite3.cpu.cores (= 2)`: number of cores SQLite runs on
+* PostgreSQL
+ * `wayang.postgres.jdbc.url`: JDBC URL to use PostgreSQL
+ * `wayang.postgres.jdbc.user`: optional user name
+ * `wayang.postgres.jdbc.password`: optional password
+ * `wayang.postgres.cpu.mhz (= 2700)`: clock frequency of processor PostgreSQL runs on in MHz
+ * `wayang.postgres.cpu.cores (= 2)`: number of cores PostgreSQL runs on
+
+**Code with Wayang.** The recommended way to specify your apps with Wayang is via its Scala or Java API from the `wayang-api` module. You can find examples below.
+
+**Learn cost functions.**
+Wayang provides a utility to learn cost functions from historical execution data.
+Specifically, Wayang can learn configurations for load profile estimators (that estimate CPU load, disk load etc.) for both operators and UDFs, as long as the configuration provides a template for those estimators.
+As an example, the `JavaMapOperator` draws its load profile estimator configuration via the configuration key `wayang.java.map.load`.
+Now, it is possible to specify a load profile estimator template in the configuration under the key `<original key>.template`, e.g.:
+```xml
+wayang.java.map.load.template = {\
+ "in":1, "out":1,\
+ "cpu":"?*in0"\
+}
+```
+This template specifies a load profile estimator that expects (at least) one input cardinality and one output cardinality.
+Further, it models a CPU load that is proportional to the input cardinality.
+However, more complex functions are possible.
+In particular, you can use
+* the variables `in0`, `in1`, ... and `out0`, `out1`, ... to incorporate the input and output cardinalities, respectively;
+* operator properties, such as `numIterations` for the `PageRankOperator` implementations;
+* the operators `+`, `-`, `*`, `/`, `%`, `^`, and parantheses;
+* the functions `min(x0, x1, ...))`, `max(x0, x1, ...)`, `abs(x)`, `log(x, base)`, `ln(x)`, `ld(x)`;
+* and the constants `e` and `pi`.
+
+While Wayang specifies templates for all execution operators, you will need to specify that your UDFs are modelled by some configuration-based cost function (see the k-means example below) and create the according initial specification and template yourself.
+Once, you gathered execution data, you can run
+```shell
+java ... org.apache.wayang.profiler.ga.GeneticOptimizerApp [configuration URL [execution log]]
+```
+This app will try to find appropriate values for the question marks (`?`) in the load profile estimator templates to fit the gathered execution data and ready-made configuration entries for the load profile estimators.
+You can then copy them into your configuration.
+
+### WordCount
+
+The "Hello World!" of data processing systems is the wordcount.
+
+#### Java API
+```java
+import org.apache.wayang.api.JavaPlanBuilder;
+import org.apache.wayang.basic.data.Tuple2;
+import org.apache.wayang.core.api.Configuration;
+import org.apache.wayang.core.api.WayangContext;
+import org.apache.wayang.core.optimizer.cardinality.DefaultCardinalityEstimator;
+import org.apache.wayang.java.Java;
+import org.apache.wayang.spark.Spark;
+import java.util.Collection;
+import java.util.Arrays;
+
+public class WordcountJava {
+
+ public static void main(String[] args){
+
+ // Settings
+ String inputUrl = "file:/tmp.txt";
+
+ // Get a plan builder.
+ WayangContext wayangContext = new WayangContext(new Configuration())
+ .withPlugin(Java.basicPlugin())
+ .withPlugin(Spark.basicPlugin());
+ JavaPlanBuilder planBuilder = new JavaPlanBuilder(wayangContext)
+ .withJobName(String.format("WordCount (%s)", inputUrl))
+ .withUdfJarOf(WordcountJava.class);
+
+ // Start building the WayangPlan.
+ Collection<Tuple2<String, Integer>> wordcounts = planBuilder
+ // Read the text file.
+ .readTextFile(inputUrl).withName("Load file")
+
+ // Split each line by non-word characters.
+ .flatMap(line -> Arrays.asList(line.split("\\W+")))
+ .withSelectivity(10, 100, 0.9)
+ .withName("Split words")
+
+ // Filter empty tokens.
+ .filter(token -> !token.isEmpty())
+ .withSelectivity(0.99, 0.99, 0.99)
+ .withName("Filter empty words")
+
+ // Attach counter to each word.
+ .map(word -> new Tuple2<>(word.toLowerCase(), 1)).withName("To lower case, add counter")
+
+ // Sum up counters for every word.
+ .reduceByKey(
+ Tuple2::getField0,
+ (t1, t2) -> new Tuple2<>(t1.getField0(), t1.getField1() + t2.getField1())
+ )
+ .withCardinalityEstimator(new DefaultCardinalityEstimator(0.9, 1, false, in -> Math.round(0.01 * in[0])))
+ .withName("Add counters")
+
+ // Execute the plan and collect the results.
+ .collect();
+
+ System.out.println(wordcounts);
+ }
+}
+```
+
+#### Scala API
+
+```scala
+import org.apache.wayang.api._
+import org.apache.wayang.core.api.{Configuration, WayangContext}
+import org.apache.wayang.java.Java
+import org.apache.wayang.spark.Spark
+
+object WordcountScala {
+ def main(args: Array[String]) {
+
+ // Settings
+ val inputUrl = "file:/tmp.txt"
+
+ // Get a plan builder.
+ val wayangContext = new WayangContext(new Configuration)
+ .withPlugin(Java.basicPlugin)
+ .withPlugin(Spark.basicPlugin)
+ val planBuilder = new PlanBuilder(wayangContext)
+ .withJobName(s"WordCount ($inputUrl)")
+ .withUdfJarsOf(this.getClass)
+
+ val wordcounts = planBuilder
+ // Read the text file.
+ .readTextFile(inputUrl).withName("Load file")
+
+ // Split each line by non-word characters.
+ .flatMap(_.split("\\W+"), selectivity = 10).withName("Split words")
+
+ // Filter empty tokens.
+ .filter(_.nonEmpty, selectivity = 0.99).withName("Filter empty words")
+
+ // Attach counter to each word.
+ .map(word => (word.toLowerCase, 1)).withName("To lower case, add counter")
+
+ // Sum up counters for every word.
+ .reduceByKey(_._1, (c1, c2) => (c1._1, c1._2 + c2._2)).withName("Add counters")
+ .withCardinalityEstimator((in: Long) => math.round(in * 0.01))
+
+ // Execute the plan and collect the results.
+ .collect()
+
+ println(wordcounts)
+ }
+}
+```
+
+### k-means
+
+Wayang is also capable of iterative processing, which is, e.g., very important for machine learning algorithms, such as k-means.
+
+#### Scala API
+
+```scala
+import org.apache.wayang.api._
+import org.apache.wayang.core.api.{Configuration, WayangContext}
+import org.apache.wayang.core.function.FunctionDescriptor.ExtendedSerializableFunction
+import org.apache.wayang.core.function.ExecutionContext
+import org.apache.wayang.core.optimizer.costs.LoadProfileEstimators
+import org.apache.wayang.java.Java
+import org.apache.wayang.spark.Spark
+
+import scala.util.Random
+import scala.collection.JavaConversions._
+
+object kmeans {
+ def main(args: Array[String]) {
+
+ // Settings
+ val inputUrl = "file:/kmeans.txt"
+ val k = 5
+ val iterations = 100
+ val configuration = new Configuration
+
+ // Get a plan builder.
+ val wayangContext = new WayangContext(new Configuration)
+ .withPlugin(Java.basicPlugin)
+ .withPlugin(Spark.basicPlugin)
+ val planBuilder = new PlanBuilder(wayangContext)
+ .withJobName(s"k-means ($inputUrl, k=$k, $iterations iterations)")
+ .withUdfJarsOf(this.getClass)
+
+ case class Point(x: Double, y: Double)
+ case class TaggedPoint(x: Double, y: Double, cluster: Int)
+ case class TaggedPointCounter(x: Double, y: Double, cluster: Int, count: Long) {
+ def add_points(that: TaggedPointCounter) = TaggedPointCounter(this.x + that.x, this.y + that.y, this.cluster, this.count + that.count)
+ def average = TaggedPointCounter(x / count, y / count, cluster, 0)
+ }
+
+ // Read and parse the input file(s).
+ val points = planBuilder
+ .readTextFile(inputUrl).withName("Read file")
+ .map { line =>
+ val fields = line.split(",")
+ Point(fields(0).toDouble, fields(1).toDouble)
+ }.withName("Create points")
+
+
+ // Create initial centroids.
+ val random = new Random
+ val initialCentroids = planBuilder
+ .loadCollection(for (i <- 1 to k) yield TaggedPointCounter(random.nextGaussian(), random.nextGaussian(), i, 0)).withName("Load random centroids")
+
+ // Declare UDF to select centroid for each data point.
+ class SelectNearestCentroid extends ExtendedSerializableFunction[Point, TaggedPointCounter] {
+
+ /** Keeps the broadcasted centroids. */
+ var centroids: Iterable[TaggedPointCounter] = _
+
+ override def open(executionCtx: ExecutionContext) = {
+ centroids = executionCtx.getBroadcast[TaggedPointCounter]("centroids")
+ }
+
+ override def apply(point: Point): TaggedPointCounter = {
+ var minDistance = Double.PositiveInfinity
+ var nearestCentroidId = -1
+ for (centroid <- centroids) {
+ val distance = Math.pow(Math.pow(point.x - centroid.x, 2) + Math.pow(point.y - centroid.y, 2), 0.5)
+ if (distance < minDistance) {
+ minDistance = distance
+ nearestCentroidId = centroid.cluster
+ }
+ }
+ new TaggedPointCounter(point.x, point.y, nearestCentroidId, 1)
+ }
+ }
+
+ // Do the k-means loop.
+ val finalCentroids = initialCentroids.repeat(iterations, { currentCentroids =>
+ points
+ .mapJava(new SelectNearestCentroid,
+ udfLoad = LoadProfileEstimators.createFromSpecification(
+ "my.udf.costfunction.key", configuration
+ ))
+ .withBroadcast(currentCentroids, "centroids").withName("Find nearest centroid")
+ .reduceByKey(_.cluster, _.add_points(_)).withName("Add up points")
+ .withCardinalityEstimator(k)
+ .map(_.average).withName("Average points")
+ }).withName("Loop")
+
+ // Collect the results.
+ .collect()
+
+ println(finalCentroids)
+ }
+}
+```
+
+## License
+
+All files in this repository are licensed under the Apache Software License 2.0
+
+Copyright 2020, 2021 Apache (incubating) Wayang Team
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
Added: dev/wayang/0.6.0/rc1/RELEASE_NOTES
==============================================================================
--- dev/wayang/0.6.0/rc1/RELEASE_NOTES (added)
+++ dev/wayang/0.6.0/rc1/RELEASE_NOTES Tue Sep 28 16:30:15 2021
@@ -0,0 +1,47 @@
+==============================================================
+Apache Wayang (incubating) 0.6.0
+==============================================================
+
+This is the first official release of Apache Wayang.
+
+On this release is possible to found the the implementation of
+the following platforms:
+- Apache Flink v1.7.1
+- Apache Giraph v1.2.0-hadoop2
+- GraphChi v0.2.2(This is only available wit scala 11.x)
+- Java Streams (version depends on the java version)
+- JDBC-Template
+- Postgres v9.4.1208(It's a implementation of JDBC-Template)
+- Apache Spark v3.1.2 (scala 12.x) and v2.4.8 (scala 11.x)
+- SQLite3 v3.8.11.2(It's a implementation of JDBC-Template)
+
+NOTE: depending on the scala version the list of platforms available
+could be modified.
+
+
+New Features
+------------
+
+- NONE -
+
+Incompatible changes
+--------------------
+
+- NONE -
+
+Miscellaneous changes
+---------------------
+
+WAYANG-3 Unification of the logs
+WAYANG-[32, 33, 35, 38] License issues removed
+
+Known Issues
+------------
+
+WAYANG-23 Iteration Memory Issue
+
+Bug Fixes
+---------
+
+WAYANG-12 Issues on the compilation
+WAYANG-40 Different version of platform depending on the scala version
Added: dev/wayang/0.6.0/rc1/wayang-0.6.0-source-release.zip
==============================================================================
Binary file - no diff available.
Propchange: dev/wayang/0.6.0/rc1/wayang-0.6.0-source-release.zip
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: dev/wayang/0.6.0/rc1/wayang-0.6.0-source-release.zip.asc
==============================================================================
--- dev/wayang/0.6.0/rc1/wayang-0.6.0-source-release.zip.asc (added)
+++ dev/wayang/0.6.0/rc1/wayang-0.6.0-source-release.zip.asc Tue Sep 28 16:30:15 2021
@@ -0,0 +1,16 @@
+-----BEGIN PGP SIGNATURE-----
+
+iQIzBAABCgAdFiEEqhSFtTMF8dlAn4a7T+7DRY0yMSIFAmFTNigACgkQT+7DRY0y
+MSKLjQ//S9GnEjbMVqSMHSLys/ACyVbkBQ2AvMI4pjWiHAONPC/bnFsZyhXHo3bK
+dT+8AKao+Nc1NEG88JSs0jghRQIsny5MNn8S9F5Mhbp5zB4RG5dpGl+e8vsyNeRn
+D22kXHEFEcEPtDivd2W5u33K7tvidRKeVEe1axcHULI0XiG6TORfzBiN27D6E9bZ
+bHd8kg7P8kIUsjHAm1lx4NfFY4/cqb3Gni2MueDRY8jnPIOqSc+0vA0arLhD0AgU
+SoNoMs9AFUWY95aEnTaBb25fpOQaEThF2iY5WwKewkh3bX7/lujB3YMXPtdbiEGD
+2aw8zB+V0VZ7ZqU2fTvpW0wPHcZVgvzEEw2FohB1+7ShraBj+1tIuTyX2/fOQX3S
+QMGsE2JAxy75mqyr/oFCTI9tj8YLnmddH2l74PfHMzwDH78PYJRnK3F8a4mWzX/A
+ZZvUJxW76+CX0hStilmEvDurWwOfCNv6SSp3VoGoQdtB7ZryLFtD1N1+Ng3uyg1J
+Kpr8w05ygxGV/4POZivkIcc+LUx6chT/pU0NNY7PAaEghjlSXRYqPok03ElCJ13f
+CqXXoPAQ/hZPj+N5tl2RN3xhSNXrAaC3WWz17Pnr3Ye2Opc+Eczs09SVA1zd5rBp
+I7PNZQC5MN1GBXig6xWeg6dvTu7np365h+7UJOuAsy30VQbgMyg=
+=ZGfz
+-----END PGP SIGNATURE-----
Added: dev/wayang/0.6.0/rc1/wayang-0.6.0-source-release.zip.sha512
==============================================================================
--- dev/wayang/0.6.0/rc1/wayang-0.6.0-source-release.zip.sha512 (added)
+++ dev/wayang/0.6.0/rc1/wayang-0.6.0-source-release.zip.sha512 Tue Sep 28 16:30:15 2021
@@ -0,0 +1 @@
+a27d9f3444a87468f93705cf5c65bcab895e9bd12e0e472a35fb73ed3a918fe67b8b93ed5d7e33e9de2b14c3141efab84c9cfcd1eac813f5e50a51f46198afcd
\ No newline at end of file
Added: dev/wayang/0.6.0/rc1/wayang-0.6.0.pom
==============================================================================
--- dev/wayang/0.6.0/rc1/wayang-0.6.0.pom (added)
+++ dev/wayang/0.6.0/rc1/wayang-0.6.0.pom Tue Sep 28 16:30:15 2021
@@ -0,0 +1,1321 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache</groupId>
+ <artifactId>apache</artifactId>
+ <version>23</version>
+ </parent>
+
+ <groupId>org.apache.wayang</groupId>
+ <artifactId>wayang</artifactId>
+ <version>0.6.0</version>
+ <packaging>pom</packaging>
+
+ <name>Apache Wayang</name>
+ <description>
+ Apache Wayang is a tool to build platform-agnostic data processing apps and have them both optimized for and
+ executed on multiple different execution platforms, such as Java Streams and Apache Spark.
+ </description>
+
+ <url>https://wayang.apache.org</url>
+ <inceptionYear>2020</inceptionYear>
+
+ <organization>
+ <name>The Apache Software Foundation</name>
+ <url>https://www.apache.org/</url>
+ </organization>
+
+ <licenses>
+ <license>
+ <name>Apache License, Version 2.0</name>
+ <url>https://www.apache.org/licenses/LICENSE-2.0</url>
+ <distribution>repo</distribution>
+ <comments>
+ Apache Wayang is an effort undergoing incubation at The Apache Software
+ Foundation (ASF), sponsored by the Apache Incubator PMC. Incubation is required
+ of all newly accepted projects until a further review indicates that the
+ infrastructure, communications, and decision making process have stabilized in
+ a manner consistent with other successful ASF projects. While incubation status
+ is not necessarily a reflection of the completeness or stability of the code,
+ it does indicate that the project has yet to be fully endorsed by the ASF.
+ </comments>
+ </license>
+ </licenses>
+
+ <scm>
+ <connection>scm:git:https://gitbox.apache.org/repos/asf/incubator-wayang.git</connection>
+ <developerConnection>scm:git:https://gitbox.apache.org/repos/asf/incubator-wayang.git</developerConnection>
+ <url>https://github.com/apache/incubator-wayang</url>
+ <tag>wayang-0.6.0</tag>
+ </scm>
+
+ <issueManagement>
+ <system>JIRA</system>
+ <url>https://issues.apache.org/jira/browse/WAYANG</url>
+ </issueManagement>
+
+ <mailingLists>
+ <mailingList>
+ <name>Apache Wayang Developer List</name>
+ <subscribe>mailto:dev-subscribe@wayang.apache.org</subscribe>
+ <unsubscribe>mailto:dev-unsubscribe@wayang.apache.org</unsubscribe>
+ <post>mailto:dev@wayang.apache.org</post>
+ <archive>http://mail-archives.apache.org/mod_mbox/wayang-dev/</archive>
+ </mailingList>
+ <mailingList>
+ <name>Apache Wayang Commits List</name>
+ <subscribe>mailto:commits-subscribe@wayang.apache.org</subscribe>
+ <unsubscribe>mailto:commits-unsubscribe@wayang.apache.org</unsubscribe>
+ <post>mailto:commits@wayang.apache.org</post>
+ <archive>http://mail-archives.apache.org/mod_mbox/wayang-commits/</archive>
+ </mailingList>
+ </mailingLists>
+
+ <properties>
+ <encoding>UTF-8</encoding>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ <project.root>${basedir}</project.root>
+ <dist.id>${project.artifactId}</dist.id>
+ <jacoco.destfile>${project.build.directory}/jacoco.exec</jacoco.destfile>
+ <maven.kotlin.skip>false</maven.kotlin.skip>
+
+ <assertj.version>3.17.2</assertj.version>
+ <commons-io.version>2.5</commons-io.version>
+ <guava.version>19.0</guava.version>
+ <hamcrest.version>1.3</hamcrest.version>
+ <jackson.version>2.11.2</jackson.version>
+ <jacoco.version>0.8.5</jacoco.version>
+ <jodatime.version>2.10.6</jodatime.version>
+ <jsonpath.version>2.4.0</jsonpath.version>
+ <junit5.version>5.6.1</junit5.version>
+ <mockito.version>3.5.10</mockito.version>
+ <mockk.version>1.10.0</mockk.version>
+ <external.platforms.scope>provided</external.platforms.scope>
+ <hadoop.version>2.7.7</hadoop.version>
+ <!-- To be overridden by individual modules -->
+ <java-module-name>org.apache.wayang.default</java-module-name>
+ <code.coverage.project.folder>${basedir}/</code.coverage.project.folder>
+ <code.coverage.overall.data.folder>${basedir}/target/aggregate.exec</code.coverage.overall.data.folder>
+ </properties>
+
+ <!-- Make Snapshots of Apache projects available -->
+ <repositories>
+ <repository>
+ <id>apache-snapshots</id>
+ <url>https://repository.apache.org/content/repositories/snapshots</url>
+ <releases>
+ <enabled>false</enabled>
+ </releases>
+ <snapshots>
+ <enabled>true</enabled>
+ </snapshots>
+ </repository>
+ </repositories>
+
+ <!-- Make Snapshots of Apache plugins available -->
+ <pluginRepositories>
+ <pluginRepository>
+ <id>apache-snapshots</id>
+ <url>https://repository.apache.org/content/repositories/snapshots</url>
+ <releases>
+ <enabled>false</enabled>
+ </releases>
+ <snapshots>
+ <enabled>true</enabled>
+ </snapshots>
+ </pluginRepository>
+ </pluginRepositories>
+
+
+
+ <profiles>
+ <profile>
+ <!-- Include the external platforms in the build. -->
+ <id>standalone</id>
+ <properties>
+ <external.platforms.scope>compile</external.platforms.scope>
+ </properties>
+ </profile>
+ <profile>
+ <id>distro</id>
+ <modules>
+ <module>wayang-distro</module>
+ </modules>
+ </profile>
+ <profile>
+ <!-- Profile to be activated when building the docs artifacts. -->
+ <id>build-docs</id>
+ <properties>
+ <shared.resources>${project.build.directory}/shared-resources</shared.resources>
+ <maven.main.skip>false</maven.main.skip>
+ <maven.test.skip>false</maven.test.skip>
+ <maven.install.skip>false</maven.install.skip>
+ <maven.deploy.skip>false</maven.deploy.skip>
+ <skipTests>false</skipTests>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.wayang</groupId>
+ <artifactId>wayang-resources</artifactId>
+ <version>0.6.0</version>
+ <type>zip</type>
+ <optional>true</optional>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <plugins>
+ <!-- Configures JavaDoc generation. -->
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-javadoc-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>aggregate-javadoc</id>
+ <goals>
+ <goal>aggregate-no-fork</goal>
+ </goals>
+ <phase>package</phase>
+ </execution>
+ </executions>
+ </plugin>
+
+ <plugin>
+ <groupId>org.jfrog.buildinfo</groupId>
+ <artifactId>artifactory-maven-plugin</artifactId>
+ </plugin>
+
+ </plugins>
+
+ </build>
+ </profile>
+ <profile>
+ <id>web-documentation</id>
+ <modules>
+ <module>wayang-docs</module>
+ </modules>
+ </profile>
+
+ <profile>
+ <!-- Perform steps to deploy Wayang. -->
+ <id>deployment</id>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-javadoc-plugin</artifactId>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-gpg-plugin</artifactId>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+
+ <profile>
+ <!-- Perform steps to deploy Wayang. -->
+ <id>deployment-dryrun</id>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-javadoc-plugin</artifactId>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+
+
+ <profile>
+
+ <!-- Profile to be run on the CI server, JARs JavaDocs -->
+ <id>ci</id>
+ <build>
+ <plugins>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-javadoc-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>package-javadoc</id>
+ <goals>
+ <goal>jar</goal>
+ </goals>
+ <phase>package</phase>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-checkstyle-plugin</artifactId>
+ <configuration>
+ <configLocation>google_checks.xml</configLocation>
+ <includes>**/*</includes>
+ <excludes>
+ **/.flattened-pom.xml,.git/**/*,target/**/*,**/target/**/*,.idea/**/*,**/spring.schemas,**/*.svg,mvnw,mvnw.cmd,**/*.graphml,work/**/*
+ </excludes>
+ <sourceDirectories>./</sourceDirectories>
+ </configuration>
+ <executions>
+ <execution>
+ <phase>
+ validate
+ </phase>
+ <goals>
+ <goal>check</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+
+ <profile>
+ <id>java8</id>
+ <activation>
+ <jdk>1.8</jdk>
+ </activation>
+ <properties>
+ <java.version>1.8</java.version>
+ <source.level>1.8</source.level>
+ <maven.compiler.source>1.8</maven.compiler.source>
+ <maven.compiler.target>1.8</maven.compiler.target>
+ </properties>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-enforcer-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>enforce-maven-version</id>
+ <goals>
+ <goal>enforce</goal>
+ </goals>
+ <configuration>
+ <rules>
+ <requireJavaVersion>
+ <version>1.8</version>
+ </requireJavaVersion>
+ </rules>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+
+ <!-- Used to build Java 11+ -->
+ <profile>
+ <id>java11</id>
+ <activation>
+ <jdk>11</jdk>
+ </activation>
+ <properties>
+ <java.version>11</java.version>
+ <source.level>11</source.level>
+ <maven.compiler.source>11</maven.compiler.source>
+ <maven.compiler.target>11</maven.compiler.target>
+ <maven.compiler.release>11</maven.compiler.release>
+ <jaxb.version>2.2.11</jaxb.version>
+ <java-activation.version>1.1.1</java-activation.version>
+ <javax-annotation-api>1.3.2</javax-annotation-api>
+ </properties>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-enforcer-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>enforce-maven-version</id>
+ <goals>
+ <goal>enforce</goal>
+ </goals>
+ <configuration>
+ <rules>
+ <requireJavaVersion>
+ <version>11</version>
+ </requireJavaVersion>
+ </rules>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-javadoc-plugin</artifactId>
+ <configuration>
+ <source>8</source>
+ <detectJavaApiLink>false</detectJavaApiLink>
+ </configuration>
+ </plugin>
+ </plugins>
+ </build>
+
+ </profile>
+
+ <!--
+ Profile to be run before a release is executed, currently does the following:
+
+ - Prevents thrid-party snapshot dependencies in projects
+ -->
+ <!-- TODO: This is actually automatically done by the maven-release-plugin:prepare goal Therefore if could be removed -->
+ <profile>
+ <id>pre-release</id>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-enforcer-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>enforce-no-third-party-snapshots</id>
+ <goals>
+ <goal>enforce</goal>
+ </goals>
+ <configuration>
+ <rules>
+ <requireReleaseDeps>
+ <failWhenParentIsSnapshot>false</failWhenParentIsSnapshot>
+ <excludes>
+ <exclude>org.apache.wayang:*</exclude>
+ </excludes>
+ </requireReleaseDeps>
+ </rules>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+
+
+ <profile>
+ <id>scala</id>
+ <activation>
+ <file>
+ <exists>src/main/scala</exists>
+ </file>
+ </activation>
+ <build>
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>net.alchim31.maven</groupId>
+ <artifactId>scala-maven-plugin</artifactId>
+ <version>4.4.0</version>
+ <executions>
+ <execution>
+ <id>compile-scala</id>
+ <goals>
+ <goal>compile</goal>
+ <goal>testCompile</goal>
+ </goals>
+ <configuration>
+ <scalaVersion>${scala.version}</scalaVersion>
+ <sourceDir>${project.build.sourceDirectory}/../scala</sourceDir>
+ <testSourceDir>${project.build.testSourceDirectory}/../scala</testSourceDir>
+ </configuration>
+ </execution>
+ <execution>
+ <id>add-scala-src</id>
+ <phase>prepare-package</phase>
+ <goals>
+ <goal>add-source</goal>
+ </goals>
+ </execution>
+ <!-- Getting strange CNFE errors:
+ java.lang.NoClassDefFoundError: javax/tools/ToolProvider
+ -->
+ <!--execution>
+ <id>generate-scaladoc</id>
+ <goals>
+ <goal>doc-jar</goal>
+ </goals>
+ </execution-->
+ </executions>
+ </plugin>
+ <plugin>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <finalName>${project.artifactId}_${scala.mayor.version}-${project.version}</finalName>
+ </configuration>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-resources-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>copy-pom</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>copy-resources</goal>
+ </goals>
+ <configuration>
+ <outputDirectory>${project.build.directory}/${scala.mayor.version}</outputDirectory>
+ <resources>
+ <resource>
+ <directory>${basedir}</directory>
+ <includes>
+ <include>pom.xml</include>
+ </includes>
+ <filtering>true</filtering>
+ </resource>
+ </resources>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+
+
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-install-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>default-install</id>
+ <configuration>
+ <skip>true</skip>
+ </configuration>
+ </execution>
+ <execution>
+ <id>install-scala-version</id>
+ <phase>install</phase>
+ <goals>
+ <goal>install-file</goal>
+ </goals>
+ <configuration>
+ <groupId>${project.groupId}</groupId>
+ <artifactId>${project.artifactId}_${scala.mayor.version}</artifactId>
+ <version>${project.version}</version>
+ <packaging>${project.packaging}</packaging>
+ <file>${project.build.directory}/${project.artifactId}_${scala.mayor.version}-${project.version}.jar</file>
+ <pomFile>${project.build.directory}/${scala.mayor.version}/replacer/pom.xml</pomFile>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>net.alchim31.maven</groupId>
+ <artifactId>scala-maven-plugin</artifactId>
+ </plugin>
+ <!-- <plugin>-->
+ <!-- <artifactId>maven-jar-plugin</artifactId>-->
+ <!-- </plugin>-->
+ <plugin>
+ <groupId>com.google.code.maven-replacer-plugin</groupId>
+ <artifactId>replacer</artifactId>
+ <version>1.5.3</version>
+ <executions>
+ <execution>
+ <id>replace-artifactid</id>
+ <phase>prepare-package</phase>
+ <goals>
+ <goal>replace</goal>
+ </goals>
+ <configuration>
+ <includes>
+ <include>${basedir}/target/${scala.mayor.version}/pom.xml</include>
+ </includes>
+ <replacements>
+ <replacement>
+ <token><artifactId>${project.artifactId}</artifactId></token>
+ <value><artifactId>${project.artifactId}_${scala.mayor.version}</artifactId></value>
+ </replacement>
+ </replacements>
+
+ <!-- Replace to -->
+ <outputFile>${basedir}/target/${scala.mayor.version}/replacer/pom.xml</outputFile>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+
+ <profile>
+ <id>scala-11</id>
+ <properties>
+ <scala.version>2.11.12</scala.version>
+ <scala.mayor.version>2.11</scala.mayor.version>
+ <spark.version>2.4.8</spark.version>
+ </properties>
+ </profile>
+
+ <profile>
+ <id>scala-12</id>
+ <properties>
+ <scala.version>2.12.12</scala.version>
+ <scala.mayor.version>2.12</scala.mayor.version>
+ <spark.version>3.1.2</spark.version>
+ </properties>
+ </profile>
+
+ <profile>
+ <id>scala-13</id>
+ <properties>
+ <scala.version>2.13.3</scala.version>
+ <scala.mayor.version>2.13</scala.mayor.version>
+ <spark.version>3.1.2</spark.version>
+ </properties>
+ </profile>
+
+
+ <profile>
+ <id>antlr</id>
+ <activation>
+ <file>
+ <exists>src/main/antlr4</exists>
+ </file>
+ </activation>
+ <properties>
+ <antlr.version>4.9.1</antlr.version>
+ </properties>
+ <build>
+ <pluginManagement>
+ <plugins>
+ <!-- trigger run with mvn process-resouces; then see target/generated-sources/antlr4 -->
+ <plugin>
+ <groupId>org.antlr</groupId>
+ <artifactId>antlr4-maven-plugin</artifactId>
+ <version>${antlr.version}</version>
+ <executions>
+ <execution>
+ <id>run-antlr</id>
+ <goals>
+ <goal>antlr4</goal>
+ </goals>
+ <configuration>
+ <visitor>true</visitor>
+ <sourceDirectory>${project.build.sourceDirectory}/../antlr4</sourceDirectory>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <!-- trigger run with mvn process-resouces; then see target/generated-sources/antlr4 -->
+ <plugin>
+ <groupId>org.antlr</groupId>
+ <artifactId>antlr4-maven-plugin</artifactId>
+ </plugin>
+ </plugins>
+ </build>
+ <dependencies>
+ <dependency>
+ <groupId>org.antlr</groupId>
+ <artifactId>antlr4</artifactId>
+ <version>${antlr.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ </dependencies>
+ </profile>
+
+ <profile>
+ <id>add-dependencies</id>
+ <build>
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ <version>3.1.2</version>
+ <executions>
+ <execution>
+ <id>copy-dependencies</id>
+ <phase>prepare-package</phase>
+ <goals>
+ <goal>copy-dependencies</goal>
+ </goals>
+ <configuration>
+ <outputDirectory>${project.build.directory}/lib</outputDirectory>
+ <overWriteReleases>false</overWriteReleases>
+ <overWriteSnapshots>false</overWriteSnapshots>
+ <overWriteIfNewer>true</overWriteIfNewer>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ </build>
+ </profile>
+
+ <!-- Allows skipping of the pre-flight-check as needed when deploying on jenkins -->
+ <profile>
+ <id>skip-prerequisite-check</id>
+ <activation>
+ </activation>
+ <build>
+ <plugins>
+ <!-- Make some additional properties available to simplify keeping some content up to date -->
+ <plugin>
+ <groupId>org.codehaus.gmaven</groupId>
+ <artifactId>groovy-maven-plugin</artifactId>
+ <version>2.1.1</version>
+ <executions>
+ <!-- Do some pre-build checks and report any findings to the user -->
+ <execution>
+ <id>prerequisite-check</id>
+ <phase>validate</phase>
+ <goals>
+ <goal>execute</goal>
+ </goals>
+ <inherited>false</inherited>
+ <configuration>
+ <source>
+ print "\nSkipping prerequisite-check\n\n"
+ </source>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+
+ <!-- Profile for linux (Self-Enabling) -->
+ <profile>
+ <id>os-unix</id>
+ <activation>
+ <os>
+ <family>unix</family>
+ </os>
+ </activation>
+ <properties>
+ <os.suffix>linux</os.suffix>
+ <os.classifier>linux-x86_64</os.classifier>
+ </properties>
+ </profile>
+ <!-- Profile for mac (Self-Enabling) -->
+ <profile>
+ <id>os-mac</id>
+ <activation>
+ <os>
+ <family>mac</family>
+ </os>
+ </activation>
+ <properties>
+ <os.suffix>mac</os.suffix>
+ <os.classifier>mac-x86_64</os.classifier>
+ </properties>
+ </profile>
+ <!-- profile for windows (Self-Enabling) -->
+ <profile>
+ <id>os-windows</id>
+ <activation>
+ <os>
+ <family>windows</family>
+ </os>
+ </activation>
+ <properties>
+ <os.suffix>win</os.suffix>
+ <os.classifier>windows-x86_64</os.classifier>
+ </properties>
+ </profile>
+
+ </profiles>
+
+ <dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>com.fasterxml.jackson</groupId>
+ <artifactId>jackson-bom</artifactId>
+ <version>${jackson.version}</version>
+ <type>pom</type>
+ <scope>import</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.junit</groupId>
+ <artifactId>junit-bom</artifactId>
+ <version>${junit5.version}</version>
+ <type>pom</type>
+ <scope>import</scope>
+ </dependency>
+ <!-- Logging -->
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-bom</artifactId>
+ <version>2.14.0</version>
+ <scope>import</scope>
+ <type>pom</type>
+ </dependency>
+ <dependency>
+ <groupId>org.scala-lang</groupId>
+ <artifactId>scala-library</artifactId>
+ <version>${scala.version}</version>
+ </dependency>
+ </dependencies>
+ </dependencyManagement>
+
+ <dependencies>
+
+ <!-- Test -->
+ <dependency>
+ <groupId>org.junit.jupiter</groupId>
+ <artifactId>junit-jupiter</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.junit.vintage</groupId>
+ <artifactId>junit-vintage-engine</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <version>${mockito.version}</version>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-junit-jupiter</artifactId>
+ <version>${mockito.version}</version>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.assertj</groupId>
+ <artifactId>assertj-core</artifactId>
+ <version>${assertj.version}</version>
+ <scope>test</scope>
+ </dependency>
+
+ <!-- Coverage -->
+ <!-- TODO: add the coverage elements -->
+
+ </dependencies>
+
+ <build>
+
+ <!--
+ Default versioned declarations of managed plugins
+ to be overridden when the distribute profile is active.
+ If this section was missing, Maven would complain about
+ missing version numbers for executions without the
+ profile active.
+ -->
+
+ <pluginManagement>
+
+ <plugins>
+ <plugin>
+ <groupId>org.openclover</groupId>
+ <artifactId>clover-maven-plugin</artifactId>
+ <version>4.2.1</version>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-assembly-plugin</artifactId>
+ <version>2.6</version>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-antrun-plugin</artifactId>
+ <version>3.0.0</version>
+ </plugin>
+
+
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <version>3.8.1</version>
+ <configuration>
+ <forceJavacCompilerUse>true</forceJavacCompilerUse>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-checkstyle-plugin</artifactId>
+ <version>3.1.1</version>
+ <dependencies>
+ <dependency>
+ <groupId>com.puppycrawl.tools</groupId>
+ <artifactId>checkstyle</artifactId>
+ <version>8.33</version>
+ </dependency>
+ </dependencies>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ <version>3.1.2</version>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-deploy-plugin</artifactId>
+ <version>3.0.0-M1</version>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-enforcer-plugin</artifactId>
+ <version>3.0.0-M3</version>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-failsafe-plugin</artifactId>
+ <version>2.19.1</version>
+ <executions>
+ <execution>
+ <goals>
+ <goal>integration-test</goal>
+ <goal>verify</goal>
+ </goals>
+ </execution>
+ </executions>
+ <configuration>
+ <trimStackTrace>false</trimStackTrace>
+ <enableAssertions>true</enableAssertions>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <version>3.2.0</version>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-javadoc-plugin</artifactId>
+ <version>3.3.0</version>
+ <executions>
+ <execution>
+ <id>attach-javadocs</id>
+ <goals>
+ <goal>jar</goal>
+ </goals>
+ </execution>
+ </executions>
+ <configuration>
+ <detectLinks>false</detectLinks>
+ <detectJavaApiLink>true</detectJavaApiLink>
+ <additionalparam>-Xdoclint:none</additionalparam>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <version>3.0.1</version>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <version>3.0.0-M5</version>
+ <configuration>
+ <useSystemClassLoader>false</useSystemClassLoader>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>build-helper-maven-plugin</artifactId>
+ <version>3.1.0</version>
+ </plugin>
+
+ <plugin>
+ <groupId>org.jfrog.buildinfo</groupId>
+ <artifactId>artifactory-maven-plugin</artifactId>
+ <version>2.7.0</version>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-site-plugin</artifactId>
+ <version>3.7</version>
+ <configuration>
+
+ </configuration>
+ </plugin>
+ </plugins>
+
+ </pluginManagement>
+
+ <plugins>
+ <!-- Make some additional properties available to simplify keeping some content up to date -->
+ <plugin>
+ <groupId>org.codehaus.gmaven</groupId>
+ <artifactId>groovy-maven-plugin</artifactId>
+ <version>2.1.1</version>
+ <executions>
+ <!-- Do some pre-build checks and report any findings to the user -->
+ <execution>
+ <id>prerequisite-check</id>
+ <phase>validate</phase>
+ <goals>
+ <goal>execute</goal>
+ </goals>
+ <inherited>false</inherited>
+ <configuration>
+ <source>${project.basedir}/src/main/script/prerequisiteCheck.groovy</source>
+ </configuration>
+ </execution>
+ </executions>
+ <dependencies>
+ <dependency>
+ <groupId>joda-time</groupId>
+ <artifactId>joda-time</artifactId>
+ <version>2.10.8</version>
+ </dependency>
+ </dependencies>
+ </plugin>
+
+ <!-- Fail the build if files don't have a valid header -->
+ <plugin>
+ <groupId>org.apache.rat</groupId>
+ <artifactId>apache-rat-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>license-check</id>
+ <phase>verify</phase>
+ <goals>
+ <goal>check</goal>
+ </goals>
+ </execution>
+ </executions>
+ <configuration>
+ <!-- Right now this would fail the build as not all files have Apache headers -->
+ <!-- TODO: Enable asap -->
+ <useMavenDefaultExcludes>true</useMavenDefaultExcludes>
+ <!--
+ Make rat output the files with missing licensed directly into the
+ build output (This way we don't have to look into the rat.txt to find
+ out which ones)
+ -->
+ <consoleOutput>true</consoleOutput>
+ <excludes>
+ <!-- Git related files -->
+ <exclude>**/.git/**</exclude>
+ <exclude>**/.gitignore</exclude>
+
+ <!-- Maven related files -->
+ <exclude>**/target/**</exclude>
+ <exclude>target/**</exclude>
+
+ <!-- Eclipse related files -->
+ <exclude>**/.project</exclude>
+ <exclude>**/.settings/**</exclude>
+ <exclude>**/.classpath</exclude>
+ <exclude>**/.factorypath</exclude>
+
+ <!-- IntelliJ related files -->
+ <exclude>**/.idea/**</exclude>
+ <exclude>**/*.iml</exclude>
+
+ <exclude>**/*.yaml</exclude>
+ <exclude>**/*.json</exclude>
+
+ <exclude>**/*.log</exclude>
+ <exclude>**/*.input</exclude>
+
+ <exclude>**/*.svg</exclude>
+ <exclude>**/Gemfile.lock</exclude>
+ <exclude>**/.jekyll-cache/**</exclude>
+ <exclude>**/README.md</exclude>
+ </excludes>
+ </configuration>
+ </plugin>
+ <!-- Fail the build if some dependency don't have a valid license -->
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>license-maven-plugin</artifactId>
+ <version>2.0.0</version>
+ <executions>
+ <execution>
+ <id>license-check</id>
+ <phase>verify</phase>
+ <goals>
+ <goal>aggregate-add-third-party</goal>
+ </goals>
+ </execution>
+ </executions>
+ <configuration>
+ <licenseMerges>
+ <licenseMerge>ASF 2.0 | The Apache Software License, Version 2.0 | Apache License, Version 2.0 | Apache 2.0 License | Apache License Version 2.0 | Apache 2.0 | Apache-2.0 | The Apache License, Version 2.0 | Apache License Version 2 | Apache 2 | http://www.apache.org/licenses/LICENSE-2.0.txt | Apache License 2.0 | Apache Software License - Version 2.0</licenseMerge>
+ <licenseMerge>BSD 3-claus | 3-Clause BSD License | BSD 3 Clause License | BSD 3 Clause | BSD 3-Clause "New" or "Revised" License (BSD-3-Clause) | BSD licence | BSD | New BSD License | Revised BSD | The BSD 3-Clause License | The BSD License | The New BSD License | New BSD license | BSD 3-clause | BSD 3-Clause </licenseMerge>
+ <licenseMerge>MIT | MIT License | The MIT License </licenseMerge>
+ <licenseMerge>BSD 2-claus | BSD 2-Clause License | BSD 2-Clause</licenseMerge>
+ <licenseMerge>HSQLDB | HSQLDB License, a BSD open source license</licenseMerge>
+ <licenseMerge>PostgreSQL | The PostgreSQL License</licenseMerge>
+ </licenseMerges>
+ <includedLicenses>
+ ASF 2.0 | BSD 2-claus | BSD 3-claus | MIT | CC0 | HSQLDB | PostgreSQL
+ </includedLicenses>
+ <failOnBlacklist>true</failOnBlacklist>
+ <excludedGroups>
+ org.apache.spark.*|org.apache.hadoop.*|org.apache.giraph.*|org.antlr.*|junit.*|org.graphchi.*
+ </excludedGroups>
+ <excludeTransitiveDependencies>true</excludeTransitiveDependencies>
+ </configuration>
+ </plugin>
+ <!-- Generate the legally required text files in the jars -->
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-remote-resources-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>process-resource-bundles</id>
+ <phase>prepare-package</phase>
+ <goals>
+ <goal>process</goal>
+ </goals>
+ <configuration>
+ <resourceBundles>
+ <!-- Will generate META-INF/{DEPENDENCIES,LICENSE,NOTICE} -->
+ <resourceBundle>org.apache:apache-jar-resource-bundle:1.4</resourceBundle>
+ <!-- Will generate META-INF/DISCLAIMER -->
+ <resourceBundle>org.apache:apache-incubator-disclaimer-resource-bundle:1.1</resourceBundle>
+ </resourceBundles>
+ <!-- Content in this directory will be appended to generated resources -->
+ <appendedResourcesDirectory>${basedir}/src/remote-resources</appendedResourcesDirectory>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <configuration>
+ <source>${source.level}</source>
+ <target>${source.level}</target>
+ </configuration>
+ <executions>
+ <execution>
+ <id>default-compile</id>
+ <phase>none</phase>
+ </execution>
+ <execution>
+ <id>default-testCompile</id>
+ <phase>none</phase>
+ </execution>
+ <execution>
+ <id>java-compile</id>
+ <phase>compile</phase>
+ <goals>
+ <goal>compile</goal>
+ </goals>
+ </execution>
+ <execution>
+ <id>java-test-compile</id>
+ <phase>test-compile</phase>
+ <goals>
+ <goal>testCompile</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <configuration>
+ <useSystemClassLoader>false</useSystemClassLoader>
+ <useFile>false</useFile>
+ <includes>
+ <include>**/*Tests.java</include>
+ <include>**/*Test.java</include>
+ </includes>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Implementation-Title>${project.name}</Implementation-Title>
+ <Implementation-Version>${project.version}</Implementation-Version>
+ <!--Automatic-Module-Name>${java-module-name}</Automatic-Module-Name-->
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <goals>
+ <goal>jar</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-javadoc-plugin</artifactId>
+ <configuration>
+ <locale>en_US</locale>
+ <breakiterator>true</breakiterator>
+ <header>${project.name}</header>
+ <source>${source.level}</source>
+ <quiet>true</quiet>
+ <javadocDirectory>${shared.resources}/javadoc</javadocDirectory>
+ <overview>${shared.resources}/javadoc/overview.html</overview>
+ <!-- copies doc-files subdirectory which contains image resources -->
+ <docfilessubdirs>true</docfilessubdirs>
+ <additionalOptions>
+ <additionalOption>-Xdoclint:none</additionalOption>
+ </additionalOptions>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-enforcer-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>enforce-maven-version</id>
+ <goals>
+ <goal>enforce</goal>
+ </goals>
+ <configuration>
+ <rules>
+ <requireMavenVersion>
+ <version>3.5.0</version>
+ </requireMavenVersion>
+ </rules>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+
+ <plugin>
+ <groupId>org.jacoco</groupId>
+ <artifactId>jacoco-maven-plugin</artifactId>
+ <version>0.8.3</version>
+ <executions>
+ <execution>
+ <id>coverage-initialize</id>
+ <goals>
+ <goal>prepare-agent</goal>
+ </goals>
+ </execution>
+ <execution>
+ <id>coverage-report</id>
+ <phase>post-integration-test</phase>
+ <goals>
+ <goal>report</goal>
+ </goals>
+ </execution>
+<!-- TODO addd the check to the minimum coverage -->
+<!-- <!– Threshold –>-->
+<!-- <execution>-->
+<!-- <id>coverage-check</id>-->
+<!-- <goals>-->
+<!-- <goal>check</goal>-->
+<!-- </goals>-->
+<!-- </execution>-->
+ <execution>
+ <id>report-aggregate</id>
+ <phase>verify</phase>
+ <goals>
+ <goal>report-aggregate</goal>
+ </goals>
+ </execution>
+ <execution>
+ <id>merge-results</id>
+ <phase>verify</phase>
+ <goals>
+ <goal>merge</goal>
+ </goals>
+ <configuration>
+ <fileSets>
+ <fileSet>
+ <directory>${code.coverage.project.folder}</directory>
+ <includes>
+ <include>**/target/jacoco.exec</include>
+ </includes>
+ </fileSet>
+ </fileSets>
+ <destFile>${code.coverage.overall.data.folder}/aggregate.exec</destFile>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+
+ </plugins>
+ </build>
+
+ <modules>
+ <module>wayang-commons</module>
+ <module>wayang-platforms</module>
+ <module>wayang-tests-integration</module>
+ <module>wayang-api</module>
+ <module>wayang-profiler</module>
+ <module>wayang-plugins</module>
+ <module>wayang-resources</module>
+ <module>wayang-benchmark</module>
+ </modules>
+</project>
Added: dev/wayang/0.6.0/rc1/wayang-0.6.0.pom.asc
==============================================================================
--- dev/wayang/0.6.0/rc1/wayang-0.6.0.pom.asc (added)
+++ dev/wayang/0.6.0/rc1/wayang-0.6.0.pom.asc Tue Sep 28 16:30:15 2021
@@ -0,0 +1,16 @@
+-----BEGIN PGP SIGNATURE-----
+
+iQIzBAABCgAdFiEEqhSFtTMF8dlAn4a7T+7DRY0yMSIFAmFTNicACgkQT+7DRY0y
+MSIzLg//U1UbCd6KVlCimqLG9wktGvTdBu+Q+xdawxINYzTBX1lhKppuA5hSS7z8
+Eb7A2uHyB4K4T9M6HTRTBUmJ14f3v6+1xTv+QqbDrm/wxdNiB8bOlFEjKJA+IIMU
+tlJ8RMzH9qn2Fln/3KqEB1T9HI0In7RVgH7Dug6mFrlkplFAQshkDNF08KvB9TWN
+43iCHf44kRwWO7PiLZiWGDLgaX3n19NqU69PcNaECA8wpWQ7K7m1xOiLhr6Mk8Up
+Apq2uRReIhzHhYegRBMRPMeu4WaSBmHUqIxRFxVNhDJcPpUnNgMy9b8cpb3kw/Mj
+zjd1kMxKGEknLJi7S3yXARy79cXuBe4+OJAXQxYYnXx8Uz4Daya9sS8UXcvFsRQA
+9KcmlQpFldKvT4Icov5+oRK9Yfzp6gRT5wsqG7VaHIJ9SXVO6ZE7fSyQ/ygzqD3t
+1kYFYX8n80F+HE/onPTCsSsEXDODpbnWkOwRvmqhFinLSsU4YZhkxXb0baT4m0S8
+gUmfXFZYflLI7lowiGbcS3LSjLw9hWTw0BVQBYWvXEuZTJqe9tcb3X2vLnseGO1U
+0CEvXO+jXr1zjRaod4ZFIJdC2vUgvQ4VCb9/NlbH4l7yuhk6rNmeONvfm3RZl3gI
+OB9srlhnZqmpDLDmnyHQYcgs9u+dHtXXzCzWINpSdVD7tdYC3NY=
+=J+SV
+-----END PGP SIGNATURE-----