You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by dh...@apache.org on 2016/07/22 07:40:15 UTC
[1/2] incubator-beam git commit: Execute RunnableOnService tests only
when runner options provided
Repository: incubator-beam
Updated Branches:
refs/heads/master adb472083 -> 07a313f1a
Execute RunnableOnService tests only when runner options provided
Previously, the situation was this:
- All runners inherit a RunnableOnService integration-test
execution referencing runnableOnServicePipelineOptions
whether or not the variable was set. Basically an unbound
variable reference.
- The Dataflow runner had a profile disabling it if
runnableOnServicePipelineOptions was not set.
- Before they got configured, Flink and Spark had to
do extra work to explicitly prevent the invalid
configuration from being used.
After this change:
- All runners inherit the same integration-test execution
but only if the variable it requires is present.
- Dataflow doesn't have any special profile.
- Flink and Spark are unchanged, since they do set
up the variable themselves. When they move to running
only as postcommit, like Dataflow does, the hardcoding
is expected to either move to a profile or move to
the Jenkins invocation.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/060bc802
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/060bc802
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/060bc802
Branch: refs/heads/master
Commit: 060bc802f0bff8ca1a076e9f93a623058e1d651b
Parents: adb4720
Author: Kenneth Knowles <kl...@google.com>
Authored: Thu Jul 21 21:01:36 2016 -0700
Committer: Dan Halperin <dh...@google.com>
Committed: Fri Jul 22 00:39:58 2016 -0700
----------------------------------------------------------------------
runners/google-cloud-dataflow-java/pom.xml | 31 -----------
runners/pom.xml | 72 ++++++++++++++-----------
2 files changed, 42 insertions(+), 61 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/060bc802/runners/google-cloud-dataflow-java/pom.xml
----------------------------------------------------------------------
diff --git a/runners/google-cloud-dataflow-java/pom.xml b/runners/google-cloud-dataflow-java/pom.xml
index 6a15635..f2c0fe8 100644
--- a/runners/google-cloud-dataflow-java/pom.xml
+++ b/runners/google-cloud-dataflow-java/pom.xml
@@ -37,37 +37,6 @@
<maven.build.timestamp.format>yyyy-MM-dd HH:mm</maven.build.timestamp.format>
</properties>
- <profiles>
- <profile>
- <!-- This profile disables automatic execution of RunnableOnService
- integration tests on Google Cloud Dataflow service. However,
- the tests will be executed if runnableOnServicePipelineOptions
- is specified. -->
- <id>disable-runnable-on-service-tests</id>
- <activation>
- <property>
- <name>!runnableOnServicePipelineOptions</name>
- </property>
- </activation>
- <build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-surefire-plugin</artifactId>
- <executions>
- <execution>
- <id>runnable-on-service-tests</id>
- <configuration>
- <skip>true</skip>
- </configuration>
- </execution>
- </executions>
- </plugin>
- </plugins>
- </build>
- </profile>
- </profiles>
-
<build>
<resources>
<resource>
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/060bc802/runners/pom.xml
----------------------------------------------------------------------
diff --git a/runners/pom.xml b/runners/pom.xml
index 612d493..777325d 100644
--- a/runners/pom.xml
+++ b/runners/pom.xml
@@ -39,34 +39,46 @@
<module>spark</module>
</modules>
- <build>
- <pluginManagement>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-surefire-plugin</artifactId>
- <executions>
- <execution>
- <id>runnable-on-service-tests</id>
- <phase>integration-test</phase>
- <goals>
- <goal>test</goal>
- </goals>
- <configuration>
- <groups>org.apache.beam.sdk.testing.RunnableOnService</groups>
- <parallel>all</parallel>
- <threadCount>4</threadCount>
- <dependenciesToScan>
- <dependency>org.apache.beam:beam-sdks-java-core</dependency>
- </dependenciesToScan>
- <systemPropertyVariables>
- <beamTestPipelineOptions>${runnableOnServicePipelineOptions}</beamTestPipelineOptions>
- </systemPropertyVariables>
- </configuration>
- </execution>
- </executions>
- </plugin>
- </plugins>
- </pluginManagement>
- </build>
+ <profiles>
+
+ <!-- A profile that adds an integration test phase if and only if
+ the runnableOnServicePipelineOptions maven property has been set.
+ It should be set to a valid PipelineOptions JSON string. -->
+ <profile>
+ <id>runnable-on-service-tests</id>
+ <activation>
+ <property><name>runnableOnServicePipelineOptions</name></property>
+ </activation>
+ <build>
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>runnable-on-service-tests</id>
+ <phase>integration-test</phase>
+ <goals>
+ <goal>test</goal>
+ </goals>
+ <configuration>
+ <groups>org.apache.beam.sdk.testing.RunnableOnService</groups>
+ <parallel>all</parallel>
+ <threadCount>4</threadCount>
+ <dependenciesToScan>
+ <dependency>org.apache.beam:beam-sdks-java-core</dependency>
+ </dependenciesToScan>
+ <systemPropertyVariables>
+ <beamTestPipelineOptions>${runnableOnServicePipelineOptions}</beamTestPipelineOptions>
+ </systemPropertyVariables>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ </build>
+ </profile>
+ </profiles>
</project>
[2/2] incubator-beam git commit: Closes #709
Posted by dh...@apache.org.
Closes #709
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/07a313f1
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/07a313f1
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/07a313f1
Branch: refs/heads/master
Commit: 07a313f1af78349040afd9ff6b01e60cdcd567a3
Parents: adb4720 060bc80
Author: Dan Halperin <dh...@google.com>
Authored: Fri Jul 22 00:39:59 2016 -0700
Committer: Dan Halperin <dh...@google.com>
Committed: Fri Jul 22 00:39:59 2016 -0700
----------------------------------------------------------------------
runners/google-cloud-dataflow-java/pom.xml | 31 -----------
runners/pom.xml | 72 ++++++++++++++-----------
2 files changed, 42 insertions(+), 61 deletions(-)
----------------------------------------------------------------------