You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@geode.apache.org by mh...@apache.org on 2020/06/15 06:46:58 UTC
[geode] branch mass-test-run updated (7843270 -> 564f43f)
This is an automated email from the ASF dual-hosted git repository.
mhanson pushed a change to branch mass-test-run
in repository https://gitbox.apache.org/repos/asf/geode.git.
discard 7843270 remove meta-* jobs that we won't use for mass-test-run
discard 631f33b Mass-test-run branch, no windows or publication jobs
add e023b54 GEODE-8226: change publish to work with non-redis servers present (#5215)
add bc0090d GEODE-8029: Delete orphaned drf files (2nd try) (#5099)
add 3a0aeb5 GEODE-8223: Update documentation of expiration accuracy as well as supported, unsupported, and unimplemented commands in Redis API for Geode docs (#5214)
add 9e2891c GEODE-7971: Added extra check in test case to find possible flaky test case (#5221)
add 11e80c3 GEODE-8233: add equals and hashCode to RedisData classes (#5220)
add 20ed5a8 GEODE-8232: add better error message for unimplemented redis commands (#5219)
add 6dabaeb GEODE-8176: Add more checks in the flaky ping test (#5176)
add 6e65b07 GEODE-8234: change redis functions to implement InternalFunction (#5229)
add f65ea45 GEODE-8230: run benchmarks in parallel with other CI jobs (#5217)
add 1231db1 GEODE-8205: add gfsh redis command (#5226)
add ece3a5a GEODE-8238: message loss during shutdown in Shutdown Hook when JVM exits (#5232)
add fe43bc6 GEODE-8205: fix locally failing tests
add 3a2cf76 GEODE-8197: Add launcher acceptance tests for custom logging config (#5196)
add 3897ab1 GEODE-8151: Convert remaining commands to use RedisResponse (#5228)
add 4a0eeeb GEODE-8222: Also ignore EntryDestroyedException during passive expiration (#5239)
add d82e30d GEODE-8243: Use java.exe on Windows in Launcher tests (#5241)
add cb5990c GEODE-8221: Refactor tests to run in appropriate projects. (#5244)
add 4477013 GEODE-8179: gfsh query cmd returns incorrect results if '=' sign is missing (#5187)
new 4e9da8f Mass-test-run branch, no windows or publication jobs
new 564f43f remove meta-* jobs that we won't use for mass-test-run
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (7843270)
\
N -- N -- N refs/heads/mass-test-run (564f43f)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
ci/pipelines/geode-build/jinja.template.yml | 7 +-
ci/pipelines/shared/jinja.variables.yml | 1 +
...SessionsBase.java => AbstractSessionsTest.java} | 143 ++++-----
...t.java => AbstractDeltaSessionManagerTest.java} | 187 +++++------
.../ClientServerSessionCacheJUnitTest.java | 319 -------------------
...ionsJUnitTest.java => Tomcat7SessionsTest.java} | 10 +-
...st.java => Tomcat7DeltaSessionManagerTest.java} | 6 +-
...st.java => Tomcat8DeltaSessionManagerTest.java} | 2 +-
...st.java => Tomcat9DeltaSessionManagerTest.java} | 2 +-
extensions/geode-modules/build.gradle | 4 +-
...ionsJUnitTest.java => Tomcat6SessionsTest.java} | 4 +-
.../catalina/AbstractSessionCacheTest.java} | 29 +-
.../catalina/ClientServerSessionCacheTest.java | 303 +++++++++++++++++-
.../session/catalina/DeltaSessionFacadeTest.java} | 6 +-
.../session/catalina/DeltaSessionTest.java} | 74 ++---
.../catalina/PeerToPeerSessionCacheTest.java} | 26 +-
.../SessionExpirationCacheListenerTest.java} | 10 +-
.../DeltaSessionAttributeEventBatchTest.java} | 17 +-
.../DeltaSessionDestroyAttributeEventTest.java} | 9 +-
.../internal/DeltaSessionStatisticsTest.java} | 11 +-
.../DeltaSessionUpdateAttributeEventTest.java} | 10 +-
.../modules/util/CreateRegionFunctionTest.java} | 17 +-
...TouchPartitionedRegionEntriesFunctionTest.java} | 16 +-
.../TouchReplicatedRegionEntriesFunctionTest.java} | 20 +-
...rLauncherWithCustomLogConfigAcceptanceTest.java | 201 ++++++++++++
.../LocatorLauncherWithPulseAcceptanceTest.java | 136 ++++++++
...rWithPulseAndCustomLogConfigAcceptanceTest.java | 304 ++++++++++++++++++
...rLauncherWithCustomLogConfigAcceptanceTest.java | 179 +++++++++++
...orLauncherWithCustomLogConfigAcceptanceTest.xml | 39 +++
...stomLogConfigAcceptanceTestWithGeodePlugins.xml | 39 +++
...mLogConfigAcceptanceTestWithoutGeodePlugins.xml | 37 +++
...erLauncherWithCustomLogConfigAcceptanceTest.xml | 39 +++
.../source/subnavs/geode-subnav.erb | 11 +-
.../geode/internal/tcp/TCPConduitDUnitTest.java | 21 ++
.../org/apache/geode/cache/wan/GatewaySender.java | 2 +-
.../distributed/internal/direct/DirectChannel.java | 6 +-
.../apache/geode/internal/cache/DiskStoreImpl.java | 18 +-
.../org/apache/geode/internal/cache/Oplog.java | 13 +-
.../org/apache/geode/internal/tcp/Connection.java | 10 +-
geode-docs/tools_modules/book_intro.html.md.erb | 2 +-
.../tools_modules/redis_api_for_geode.html.md.erb | 78 +++--
.../sockets/ClientServerMiscDUnitTestBase.java | 13 +-
.../commands/QueryCommandIntegrationTestBase.java | 15 +-
.../geode/management/internal/cli/GfshParser.java | 24 +-
.../internal/cli/GfshParserJUnitTest.java | 62 ++++
.../redis/RenameNativeRedisAcceptanceTest.java | 7 +
.../apache/geode/redis/GeodeRedisServerRule.java | 2 +-
.../redis/GeodeRedisServerStartupDUnitTest.java | 106 ++++++-
.../org/apache/geode/redis/PubSubDUnitTest.java | 20 +-
.../apache/geode/redis/RenameIntegrationTest.java | 8 +
.../geode/redis/key/TimeIntegrationTest.java | 60 ++++
.../geode/redis/internal/GeodeRedisServer.java | 36 ++-
...Executor.java => RedisCommandSupportLevel.java} | 14 +-
.../geode/redis/internal/RedisCommandType.java | 301 ++++++++++++------
.../geode/redis/internal/RedisConstants.java | 3 +-
.../geode/redis/internal/RegionProvider.java | 9 +-
.../redis/internal/data/AbstractRedisData.java | 17 +
.../geode/redis/internal/data/EmptyRedisSet.java | 4 +
.../geode/redis/internal/data/RedisHash.java | 21 ++
.../apache/geode/redis/internal/data/RedisSet.java | 21 ++
.../geode/redis/internal/data/RedisString.java | 21 ++
.../geode/redis/internal/executor/Executor.java | 15 +-
.../redis/internal/executor/RedisResponse.java | 10 +
.../executor/SingleResultRedisFunction.java | 4 +-
.../redis/internal/executor/UnknownExecutor.java | 9 +-
.../redis/internal/executor/hash/HDelExecutor.java | 2 +-
.../internal/executor/hash/HExistsExecutor.java | 2 +-
.../internal/executor/hash/HGetAllExecutor.java | 2 +-
.../redis/internal/executor/hash/HGetExecutor.java | 2 +-
.../internal/executor/hash/HIncrByExecutor.java | 2 +-
.../executor/hash/HIncrByFloatExecutor.java | 2 +-
.../internal/executor/hash/HKeysExecutor.java | 2 +-
.../redis/internal/executor/hash/HLenExecutor.java | 2 +-
.../internal/executor/hash/HMGetExecutor.java | 2 +-
.../internal/executor/hash/HMSetExecutor.java | 2 +-
.../internal/executor/hash/HScanExecutor.java | 2 +-
.../redis/internal/executor/hash/HSetExecutor.java | 2 +-
.../internal/executor/hash/HValsExecutor.java | 2 +-
.../internal/executor/key/DBSizeExecutor.java | 8 +-
.../redis/internal/executor/key/DelExecutor.java | 7 +-
.../internal/executor/key/ExistsExecutor.java | 8 +-
.../internal/executor/key/ExpireAtExecutor.java | 21 +-
.../internal/executor/key/ExpireExecutor.java | 16 +-
.../internal/executor/key/FlushAllExecutor.java | 7 +-
.../redis/internal/executor/key/KeysExecutor.java | 20 +-
.../internal/executor/key/PersistExecutor.java | 14 +-
.../internal/executor/key/RenameExecutor.java | 19 +-
.../internal/executor/key/RenameFunction.java | 4 +-
.../redis/internal/executor/key/ScanExecutor.java | 31 +-
.../redis/internal/executor/key/TTLExecutor.java | 14 +-
.../redis/internal/executor/key/TypeExecutor.java | 11 +-
.../executor/pubsub/PsubscribeExecutor.java | 36 +--
.../internal/executor/pubsub/PublishExecutor.java | 5 +-
.../executor/pubsub/PunsubscribeExecutor.java | 15 +-
.../executor/pubsub/SubscribeExecutor.java | 36 +--
.../executor/pubsub/UnsubscribeExecutor.java | 20 +-
.../internal/executor/server/AuthExecutor.java | 2 +-
.../internal/executor/server/EchoExecutor.java | 11 +-
.../internal/executor/server/PingExecutor.java | 8 +-
.../internal/executor/server/QuitExecutor.java | 2 +-
.../internal/executor/server/ShutDownExecutor.java | 6 +-
.../internal/executor/server/TimeExecutor.java | 30 +-
.../redis/internal/executor/set/SAddExecutor.java | 2 +-
.../redis/internal/executor/set/SCardExecutor.java | 2 +-
.../internal/executor/set/SIsMemberExecutor.java | 2 +-
.../internal/executor/set/SMembersExecutor.java | 2 +-
.../redis/internal/executor/set/SMoveExecutor.java | 2 +-
.../redis/internal/executor/set/SPopExecutor.java | 2 +-
.../internal/executor/set/SRandMemberExecutor.java | 2 +-
.../redis/internal/executor/set/SRemExecutor.java | 2 +-
.../redis/internal/executor/set/SScanExecutor.java | 2 +-
.../redis/internal/executor/set/SetOpExecutor.java | 2 +-
.../internal/executor/string/AppendExecutor.java | 2 +-
.../internal/executor/string/BitCountExecutor.java | 2 +-
.../internal/executor/string/BitOpExecutor.java | 2 +-
.../internal/executor/string/BitPosExecutor.java | 2 +-
.../internal/executor/string/DecrByExecutor.java | 2 +-
.../internal/executor/string/DecrExecutor.java | 2 +-
.../internal/executor/string/GetBitExecutor.java | 2 +-
.../internal/executor/string/GetExecutor.java | 2 +-
.../internal/executor/string/GetRangeExecutor.java | 2 +-
.../internal/executor/string/GetSetExecutor.java | 2 +-
.../internal/executor/string/IncrByExecutor.java | 2 +-
.../executor/string/IncrByFloatExecutor.java | 2 +-
.../internal/executor/string/IncrExecutor.java | 2 +-
.../internal/executor/string/MGetExecutor.java | 2 +-
.../internal/executor/string/MSetExecutor.java | 2 +-
.../internal/executor/string/MSetNXExecutor.java | 2 +-
.../internal/executor/string/SetBitExecutor.java | 2 +-
.../internal/executor/string/SetEXExecutor.java | 2 +-
.../internal/executor/string/SetExecutor.java | 2 +-
.../internal/executor/string/SetNXExecutor.java | 2 +-
.../internal/executor/string/SetRangeExecutor.java | 2 +-
.../internal/executor/string/StrlenExecutor.java | 2 +-
.../geode/redis/internal/gfsh/RedisCommand.java | 55 ++++
.../redis/internal/gfsh/RedisCommandFunction.java | 53 ++++
.../apache/geode/redis/internal/netty/Coder.java | 47 ++-
.../apache/geode/redis/internal/netty/Command.java | 8 +
.../internal/netty/ExecutionHandlerContext.java | 18 +-
.../apache/geode/redis/internal/pubsub/PubSub.java | 8 +-
.../geode/redis/internal/pubsub/PubSubImpl.java | 24 +-
.../org.springframework.shell.core.CommandMarker | 18 ++
.../sanctioned-geode-redis-serializables.txt | 4 +-
.../internal/ExecutionHandlerContextJUnitTest.java | 74 -----
.../redis/internal/SupportedCommandsJUnitTest.java | 14 +-
.../executor/UnknownExecutorJUnitTest.java | 8 +-
.../executor/key/ExpireAtExecutorJUnitTest.java | 22 +-
.../executor/key/ExpireExecutorJUnitTest.java | 17 +-
.../executor/key/PExpireAtExecutorJUnitTest.java | 22 +-
.../executor/key/PersistExecutorJUnitTest.java | 24 +-
.../executor/string/GetRangeExecutorJUnitTest.java | 8 +-
.../executor/string/GetSetExecutorJUnitTest.java | 4 +-
.../executor/string/IncrExecutorJUnitTest.java | 4 +-
.../string/StringMGetExecutorJUnitTest.java | 4 +-
.../string/StringMSetExecutorJUnitTest.java | 4 +-
.../string/StringSetExecutorJUnitTest.java | 12 +-
.../executor/string/StrlenExecutorJUnitTest.java | 4 +-
...SenderWithIsolatedDiskStoreDistributedTest.java | 348 +++++++++++++++++++++
.../cache/wan/serial/SerialWANStatsDUnitTest.java | 1 +
159 files changed, 3155 insertions(+), 1261 deletions(-)
rename extensions/geode-modules-test/src/main/java/org/apache/geode/modules/session/{TestSessionsBase.java => AbstractSessionsTest.java} (72%)
rename extensions/geode-modules-test/src/main/java/org/apache/geode/modules/session/catalina/{DeltaSessionManagerJUnitTest.java => AbstractDeltaSessionManagerTest.java} (71%)
delete mode 100644 extensions/geode-modules-test/src/main/java/org/apache/geode/modules/session/catalina/ClientServerSessionCacheJUnitTest.java
rename extensions/geode-modules-tomcat7/src/integrationTest/java/org/apache/geode/modules/session/{Tomcat7SessionsJUnitTest.java => Tomcat7SessionsTest.java} (87%)
rename extensions/geode-modules-tomcat7/src/test/java/org/apache/geode/modules/session/catalina/{Tomcat7DeltaSessionManagerJUnitTest.java => Tomcat7DeltaSessionManagerTest.java} (96%)
rename extensions/geode-modules-tomcat8/src/test/java/org/apache/geode/modules/session/catalina/{Tomcat8DeltaSessionManagerJUnitTest.java => Tomcat8DeltaSessionManagerTest.java} (98%)
rename extensions/geode-modules-tomcat9/src/test/java/org/apache/geode/modules/session/catalina/{Tomcat9DeltaSessionManagerJUnitTest.java => Tomcat9DeltaSessionManagerTest.java} (98%)
rename extensions/geode-modules/src/integrationTest/java/org/apache/geode/modules/session/{Tomcat6SessionsJUnitTest.java => Tomcat6SessionsTest.java} (88%)
rename extensions/{geode-modules-test/src/main/java/org/apache/geode/modules/session/catalina/AbstractSessionCacheJUnitTest.java => geode-modules/src/test/java/org/apache/geode/modules/session/catalina/AbstractSessionCacheTest.java} (84%)
rename extensions/{geode-modules-test/src/main/java/org/apache/geode/modules/session/catalina/DeltaSessionFacadeJUnitTest.java => geode-modules/src/test/java/org/apache/geode/modules/session/catalina/DeltaSessionFacadeTest.java} (88%)
rename extensions/{geode-modules-test/src/main/java/org/apache/geode/modules/session/catalina/DeltaSessionJUnitTest.java => geode-modules/src/test/java/org/apache/geode/modules/session/catalina/DeltaSessionTest.java} (71%)
rename extensions/{geode-modules-test/src/main/java/org/apache/geode/modules/session/catalina/PeerToPeerSessionCacheJUnitTest.java => geode-modules/src/test/java/org/apache/geode/modules/session/catalina/PeerToPeerSessionCacheTest.java} (90%)
rename extensions/{geode-modules-test/src/main/java/org/apache/geode/modules/session/catalina/callback/SessionExpirationCacheListenerJUnitTest.java => geode-modules/src/test/java/org/apache/geode/modules/session/catalina/callback/SessionExpirationCacheListenerTest.java} (80%)
rename extensions/{geode-modules-test/src/main/java/org/apache/geode/modules/session/catalina/internal/DeltaSessionAttributeEventBatchJUnitTest.java => geode-modules/src/test/java/org/apache/geode/modules/session/catalina/internal/DeltaSessionAttributeEventBatchTest.java} (77%)
rename extensions/{geode-modules-test/src/main/java/org/apache/geode/modules/session/catalina/internal/DeltaSessionDestroyAttributeEventJUnitTest.java => geode-modules/src/test/java/org/apache/geode/modules/session/catalina/internal/DeltaSessionDestroyAttributeEventTest.java} (79%)
rename extensions/{geode-modules-test/src/main/java/org/apache/geode/modules/session/catalina/internal/DeltaSessionStatisticsJUnitTest.java => geode-modules/src/test/java/org/apache/geode/modules/session/catalina/internal/DeltaSessionStatisticsTest.java} (85%)
rename extensions/{geode-modules-test/src/main/java/org/apache/geode/modules/session/catalina/internal/DeltaSessionUpdateAttributeEventJUnitTest.java => geode-modules/src/test/java/org/apache/geode/modules/session/catalina/internal/DeltaSessionUpdateAttributeEventTest.java} (81%)
rename extensions/{geode-modules-test/src/main/java/org/apache/geode/modules/util/CreateRegionFunctionJUnitTest.java => geode-modules/src/test/java/org/apache/geode/modules/util/CreateRegionFunctionTest.java} (85%)
rename extensions/{geode-modules-test/src/main/java/org/apache/geode/modules/util/TouchPartitionedRegionEntriesFunctionJUnitTest.java => geode-modules/src/test/java/org/apache/geode/modules/util/TouchPartitionedRegionEntriesFunctionTest.java} (84%)
rename extensions/{geode-modules-test/src/main/java/org/apache/geode/modules/util/TouchReplicatedRegionEntriesFunctionJUnitTest.java => geode-modules/src/test/java/org/apache/geode/modules/util/TouchReplicatedRegionEntriesFunctionTest.java} (79%)
create mode 100644 geode-assembly/src/acceptanceTest/java/org/apache/geode/launchers/LocatorLauncherWithCustomLogConfigAcceptanceTest.java
create mode 100644 geode-assembly/src/acceptanceTest/java/org/apache/geode/launchers/LocatorLauncherWithPulseAcceptanceTest.java
create mode 100644 geode-assembly/src/acceptanceTest/java/org/apache/geode/launchers/LocatorLauncherWithPulseAndCustomLogConfigAcceptanceTest.java
create mode 100644 geode-assembly/src/acceptanceTest/java/org/apache/geode/launchers/ServerLauncherWithCustomLogConfigAcceptanceTest.java
create mode 100644 geode-assembly/src/acceptanceTest/resources/org/apache/geode/launchers/LocatorLauncherWithCustomLogConfigAcceptanceTest.xml
create mode 100644 geode-assembly/src/acceptanceTest/resources/org/apache/geode/launchers/LocatorLauncherWithPulseAndCustomLogConfigAcceptanceTestWithGeodePlugins.xml
create mode 100644 geode-assembly/src/acceptanceTest/resources/org/apache/geode/launchers/LocatorLauncherWithPulseAndCustomLogConfigAcceptanceTestWithoutGeodePlugins.xml
create mode 100644 geode-assembly/src/acceptanceTest/resources/org/apache/geode/launchers/ServerLauncherWithCustomLogConfigAcceptanceTest.xml
create mode 100644 geode-redis/src/integrationTest/java/org/apache/geode/redis/key/TimeIntegrationTest.java
copy geode-redis/src/main/java/org/apache/geode/redis/internal/{executor/server/ShutDownExecutor.java => RedisCommandSupportLevel.java} (66%)
mode change 100755 => 100644
create mode 100644 geode-redis/src/main/java/org/apache/geode/redis/internal/gfsh/RedisCommand.java
create mode 100644 geode-redis/src/main/java/org/apache/geode/redis/internal/gfsh/RedisCommandFunction.java
create mode 100644 geode-redis/src/main/resources/META-INF/services/org.springframework.shell.core.CommandMarker
delete mode 100644 geode-redis/src/test/java/org/apache/geode/redis/internal/ExecutionHandlerContextJUnitTest.java
create mode 100644 geode-wan/src/distributedTest/java/org/apache/geode/internal/cache/wan/PersistentGatewaySenderWithIsolatedDiskStoreDistributedTest.java
[geode] 02/02: remove meta-* jobs that we won't use for
mass-test-run
Posted by mh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mhanson pushed a commit to branch mass-test-run
in repository https://gitbox.apache.org/repos/asf/geode.git
commit 564f43fa844b53de16550df9841aa94eb08ad83d
Author: Mark Hanson <mh...@pivotal.io>
AuthorDate: Mon Jan 27 15:27:19 2020 -0800
remove meta-* jobs that we won't use for mass-test-run
---
ci/pipelines/meta/jinja.template.yml | 207 +----------------------------------
1 file changed, 1 insertion(+), 206 deletions(-)
diff --git a/ci/pipelines/meta/jinja.template.yml b/ci/pipelines/meta/jinja.template.yml
index 5fd8ee3..3fd622f 100644
--- a/ci/pipelines/meta/jinja.template.yml
+++ b/ci/pipelines/meta/jinja.template.yml
@@ -52,13 +52,7 @@ resources:
branch: ((!geode-build-branch))
paths:
- ci/pipelines/images/*
-- name: geode-reaper-pipeline
- type: git
- source:
- {{ github_access() | indent(4) }}
- branch: ((!geode-build-branch))
- paths:
- - ci/pipelines/reaper/*
+
- name: meta-mini-dockerfile
type: git
source:
@@ -72,35 +66,6 @@ resources:
username: ((!docker-username))
password: ((!docker-password))
repository: gcr.io/((!gcp-project))/((!sanitized-geode-fork))-((!sanitized-geode-build-branch))-meta-img
-{% if repository.fork == repository.upstream_fork %}
-- name: geode-examples-pipeline
- type: git
- source:
- {{ github_access() | indent(4) }}
- branch: ((!geode-build-branch))
- paths:
- - ci/pipelines/examples/*
- - ci/pipelines/shared/*
- - ci/pipelines/render.py
- - ci/pipelines/meta/meta.properties
-- name: geode-pr-pipeline
- type: git
- source:
- {{ github_access() | indent(4) }}
- branch: ((!geode-build-branch))
- paths:
- - ci/pipelines/pull-request/*
- - ci/pipelines/shared/*
- - ci/pipelines/render.py
- - ci/pipelines/meta/meta.properties
-{% endif %}
-- name: geode-metrics-pipeline
- type: git
- source:
- {{ github_access() | indent(4) }}
- branch: ((!geode-build-branch))
- paths:
- - ci/pipelines/metrics/*
jobs:
- name: set-images-pipeline
@@ -188,176 +153,6 @@ jobs:
- results/pipeline-vars.yml
-{% if repository.fork == repository.upstream_fork %}
-- name: set-examples-pipeline
- serial: true
- public: ((!public-pipelines))
- plan:
- - get: geode-examples-pipeline
- trigger: true
- - get: meta-mini-image
- trigger: true
- passed: [build-meta-mini-docker-image]
- - task: deploy-build
- image: meta-mini-image
- config:
- platform: linux
- inputs:
- - name: geode-examples-pipeline
- outputs:
- - name: results
- params:
- OUTPUT_DIRECTORY: results
- GEODE_BRANCH: ((geode-build-branch))
- GEODE_FORK: ((geode-fork))
- GEODE_REPO_NAME: ((geode-repo-name))
- REPOSITORY_PUBLIC: {{ repository.public }}
- ARTIFACT_BUCKET: ((artifact-bucket))
- UPSTREAM_FORK: {{ repository.upstream_fork }}
- PIPELINE_PREFIX: ((pipeline-prefix))
- SANITIZED_GEODE_BRANCH: ((sanitized-geode-build-branch))
- SANITIZED_GEODE_FORK: ((sanitized-geode-fork))
- GCP_PROJECT: ((gcp-project))
- PUBLIC_PIPELINES: ((public-pipelines))
- MAVEN_SNAPSHOT_BUCKET: ((maven-snapshot-bucket))
- SEMVER_PRERELEASE_TOKEN: ((semver-prerelease-token))
- run:
- path: geode-examples-pipeline/ci/pipelines/examples/deploy_pipeline.sh
- - put: concourse
- params:
- pipelines:
- - name: ((!pipeline-prefix))examples
- team: ((!concourse-team))
- config_file: results/generated-pipeline.yml
- vars_files:
- - results/pipeline-vars.yml
-
-- name: set-pr-pipeline
- serial: true
- public: ((!public-pipelines))
- plan:
- - get: geode-pr-pipeline
- trigger: true
- - get: meta-mini-image
- trigger: true
- passed: [build-meta-mini-docker-image]
- - task: deploy-pr
- image: meta-mini-image
- config:
- platform: linux
- inputs:
- - name: geode-pr-pipeline
- outputs:
- - name: results
- params:
- OUTPUT_DIRECTORY: results
- GEODE_BRANCH: ((!geode-build-branch))
- GEODE_FORK: ((!geode-fork))
- GEODE_REPO_NAME: ((geode-repo-name))
- REPOSITORY_PUBLIC: {{ repository.public }}
- ARTIFACT_BUCKET: ((artifact-bucket))
- UPSTREAM_FORK: {{ repository.upstream_fork }}
- PIPELINE_PREFIX: ((pipeline-prefix))
- GCP_PROJECT: ((gcp-project))
- PUBLIC_PIPELINES: ((public-pipelines))
- GRADLE_GLOBAL_ARGS: ((gradle-global-args))
- run:
- path: geode-pr-pipeline/ci/pipelines/pull-request/deploy_pr_pipeline.sh
- - put: concourse
- params:
- pipelines:
- - name: ((!pipeline-prefix))pr
- team: ((!concourse-team))
- config_file: results/generated-pipeline.yml
- vars_files:
- - results/pipeline-vars.yml
-
-{% endif %}
-
-- name: set-metrics-pipeline
- serial: true
- public: ((!public-pipelines))
- plan:
- - get: geode-metrics-pipeline
- trigger: true
- - get: meta-mini-image
- trigger: true
- passed: [build-meta-mini-docker-image]
- - task: create-metrics-pipeline-yml
- image: meta-mini-image
- config:
- platform: linux
- inputs:
- - name: geode-metrics-pipeline
- outputs:
- - name: results
- params:
- OUTPUT_DIRECTORY: results
- GEODE_BRANCH: ((geode-build-branch))
- GEODE_FORK: ((geode-fork))
- GEODE_REPO_NAME: ((geode-repo-name))
- REPOSITORY_PUBLIC: {{ repository.public }}
- ARTIFACT_BUCKET: ((artifact-bucket))
- UPSTREAM_FORK: {{ repository.upstream_fork }}
- PIPELINE_PREFIX: ((pipeline-prefix))
- SANITIZED_GEODE_BRANCH: ((sanitized-geode-build-branch))
- SANITIZED_GEODE_FORK: ((sanitized-geode-fork))
- GCP_PROJECT: ((gcp-project))
- PUBLIC_PIPELINES: ((public-pipelines))
- CONCOURSE_URL: ((concourse-url))
- CONCOURSE_HOST: ((concourse-host))
- CONCOURSE_TEAM: ((concourse-team))
- run:
- path: geode-metrics-pipeline/ci/pipelines/metrics/deploy_metrics_pipeline.sh
- - put: concourse
- params:
- pipelines:
- - name: ((!pipeline-prefix))metrics
- team: ((!concourse-team))
- config_file: results/generated-pipeline.yml
- vars_files:
- - results/pipeline-vars.yml
-
-- name: set-reaper-pipeline
- serial: true
- public: ((!public-pipelines))
- plan:
- - get: geode-reaper-pipeline
- trigger: true
- - get: meta-mini-image
- trigger: true
- passed: [build-meta-mini-docker-image]
- - task: create-image-yml
- image: meta-mini-image
- config:
- platform: linux
- inputs:
- - name: geode-reaper-pipeline
- outputs:
- - name: results
- params:
- OUTPUT_DIRECTORY: results
- GEODE_BRANCH: ((geode-build-branch))
- GEODE_FORK: ((geode-fork))
- GEODE_REPO_NAME: ((geode-repo-name))
- REPOSITORY_PUBLIC: {{ repository.public }}
- UPSTREAM_FORK: {{ repository.upstream_fork }}
- PIPELINE_PREFIX: ((pipeline-prefix))
- SANITIZED_GEODE_BRANCH: ((sanitized-geode-build-branch))
- SANITIZED_GEODE_FORK: ((sanitized-geode-fork))
- GCP_PROJECT: ((gcp-project))
- PUBLIC_PIPELINES: ((public-pipelines))
- run:
- path: geode-reaper-pipeline/ci/pipelines/reaper/deploy_reaper_pipeline.sh
- - put: concourse
- params:
- pipelines:
- - name: ((!pipeline-prefix))reaper
- team: ((!concourse-team))
- config_file: results/generated-pipeline.yml
- vars_files:
- - results/pipeline-vars.yml
-
- name: build-meta-mini-docker-image
public: ((!public-pipelines))
serial: true
[geode] 01/02: Mass-test-run branch, no windows or publication jobs
Posted by mh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mhanson pushed a commit to branch mass-test-run
in repository https://gitbox.apache.org/repos/asf/geode.git
commit 4e9da8f048458184c745098a3f8394efaa69a30c
Author: Robert Houghton <rh...@pivotal.io>
AuthorDate: Mon Jan 27 15:20:46 2020 -0800
Mass-test-run branch, no windows or publication jobs
Co-authored-by: Robert Houghton <rh...@pivotal.io>
Co-authored-by: Mark Hanson <mh...@pivotal.io>
---
ci/pipelines/geode-build/jinja.template.yml | 302 +---------------------------
ci/pipelines/shared/jinja.variables.yml | 90 +--------
2 files changed, 7 insertions(+), 385 deletions(-)
diff --git a/ci/pipelines/geode-build/jinja.template.yml b/ci/pipelines/geode-build/jinja.template.yml
index bc71092..d4ca428 100644
--- a/ci/pipelines/geode-build/jinja.template.yml
+++ b/ci/pipelines/geode-build/jinja.template.yml
@@ -95,61 +95,6 @@ GRADLE_GLOBAL_ARGS: ((gradle-global-args))
{%- endfor -%}
{% endmacro %}
-groups:
-- name: main
- jobs:
- - {{ build_test.name }}
- {{- all_gating_jobs() | indent(2) }}
- - UpdatePassingTokens
- {%- if repository.upstream_fork != "apache" or repository.branch == "develop" or repository.branch.startswith("support/") %}
- - PublishArtifacts
- {%- endif %}
-- name: complete
- jobs:
- - {{ build_test.name }}
- {{- all_gating_jobs() | indent(2) }}
- {%- if repository.upstream_fork != "apache" or repository.branch == "develop" or repository.branch.startswith("support/") %}
- - PublishArtifacts
- {%- endif %}
- - UpdatePassingTokens
-- name: linux
- jobs:
- - {{ build_test.name }}
- {%- for test in (tests) if test.PLATFORM=="linux" and not test.name=="StressNew" -%}
- {% for java_test_version in (java_test_versions) if ((not test.ONLY_JDK is defined) or test.ONLY_JDK==java_test_version.version) %}
- - {{test.name}}Test{{java_test_version.name}}
- {%- endfor -%}
- {%- endfor %}
- {%- for flavor in (benchmarks.flavors) %}
- - Benchmark{{flavor.title}}
- {%- endfor %}
-- name: windows
- jobs:
- - {{ build_test.name }}
- {%- for test in (tests) if test.PLATFORM=="windows" -%}
- {% for java_test_version in (java_test_versions) if ((not test.ONLY_JDK is defined) or test.ONLY_JDK==java_test_version.version) %}
- - {{test.name}}Test{{java_test_version.name}}
- {%- endfor -%}
- {%- endfor %}
-{%- for java_test_version in (java_test_versions) %}
-- name: {{java_test_version.name}}
- jobs:
- - {{ build_test.name }}
- {%- for test in (tests) if (not test.name=="StressNew") and ((not test.ONLY_JDK is defined) or test.ONLY_JDK==java_test_version.version) %}
- - {{test.name}}Test{{java_test_version.name}}
- {%- endfor -%}
-{%- endfor %}
-- name: Benchmarks
- jobs:
- {%- for run_var in (benchmarks.flavors) %}
- - Benchmark{{ run_var.title }}
- {%- endfor %}
-- name: Semver Management
- jobs:
- {%- for semverPiece in ['major', 'minor', 'patch'] %}
- - Bump{{ semverPiece.title() }}
- {%- endfor %}
-
resources:
- name: concourse-metadata-resource
type: concourse-metadata-resource
@@ -171,12 +116,7 @@ resources:
paths:
- ci/*
{{ github_access() | indent(4) }}
-- name: geode-benchmarks
- type: git
- source:
- branch: {{benchmarks.benchmark_branch}}
- depth: 1
- uri: https://github.com/apache/geode-benchmarks.git
+
- name: geode-build-version
type: semver
source:
@@ -185,12 +125,7 @@ resources:
initial_version: {{ metadata.initial_version }}
json_key: ((!concourse-gcp-key))
key: semvers/((pipeline-prefix))((geode-build-branch))/number
-- name: geode-passing-tokens
- type: json-gcs-resource
- source:
- bucket: ((artifact-bucket))
- json_key: ((concourse-gcp-key))
- versioned_file: semvers/((pipeline-prefix))((geode-build-branch))/passing-build-tokens.json
+
- name: alpine-tools-image
type: docker-image
source:
@@ -199,12 +134,6 @@ resources:
repository: gcr.io/((gcp-project))/((pipeline-prefix))alpine-tools
tag: latest
-- name: windows-builder-image-family
- type: gci
- source:
- key: ((concourse-gcp-key))
- family_project: ((gcp-project))
- family: ((pipeline-prefix))windows-geode-builder
- name: linux-builder-image-family
type: gci
@@ -238,18 +167,6 @@ resource_types:
repository: gcr.io/((gcp-project))/((pipeline-prefix))gcs-resource-with-json-content
tag: latest
jobs:
-{% for semverPiece in ['major', 'minor', 'patch'] -%}
-- name: Bump{{ semverPiece.title() }}
- serial: true
- plan:
- - get: geode-build-version
- params: { bump: {{ semverPiece }},
- pre: ((semver-prerelease-token)),
- pre_without_version: true }
- - put: geode-build-version
- params:
- file: geode-build-version/number
-{% endfor %}
- name: {{build_test.name}}
public: true
max_in_flight: {{build_test.MAX_IN_FLIGHT}}
@@ -395,221 +312,6 @@ jobs:
- name: instance-data
timeout: 1h
-- name: UpdatePassingTokens
- public: true
- serial: true
- plan:
- - aggregate:
- - get: alpine-tools-image
- - get: geode
- passed: &update-token-passed-anchor
-{%- if repository.upstream_fork != "apache" or repository.branch == "develop" or repository.branch.startswith("support/") %}
- - PublishArtifacts
-{% else %}
- {{- all_gating_jobs() | indent(6) }}
-{% endif %}
- trigger: true
- - get: geode-build-version
- trigger: true
- passed: *update-token-passed-anchor
- - task: couple-sha-and-build-id
- image: alpine-tools-image
- config:
- platform: linux
- inputs:
- - name: geode
- - name: geode-build-version
- outputs:
- - name: geode-passing-tokens
- run:
- path: bash
- args:
- - -cx
- - |
- pushd geode
- GEODE_SHA=$(git rev-parse HEAD)
- popd
- GEODE_SEMVER=$(cat geode-build-version/number)
-
- GS_PATH=gs://((artifact-bucket))/semvers/((pipeline-prefix))((geode-build-branch))/passing-build-tokens.json
- CURRENT_PASSING_SHA=$(gsutil cat ${GS_PATH} | jq -r .ref)
- set -e
- # Check that the incoming GEODE_SHA is a descendent of the currently stored value.
- # Keeps us from winding back the repository in the case of an out-of-order pipeline pass
- if [ -n "${CURRENT_PASSING_SHA}" ]; then
- cd geode
- if git merge-base --is-ancestor ${CURRENT_PASSING_SHA} ${GEODE_SHA}; then
- cat > ../geode-passing-tokens/passing-build-tokens.json <<JSON
- {
- "ref": "${GEODE_SHA}",
- "semver": "${GEODE_SEMVER}"
- }
- JSON
- fi
- fi
- - aggregate:
- - put: geode-passing-tokens
- params:
- file: geode-passing-tokens/passing-build-tokens.json
-
-{% for run_var in (benchmarks.flavors) %}
-- name: Benchmark{{ run_var.title }}
- public: true
- max_in_flight: 2
- plan:
- - get: geode-ci
- passed: &benchmark-inputs
- - Build
- - get: alpine-tools-image
- - aggregate:
- - get: geode
- passed: *benchmark-inputs
- trigger: true
- - get: geode-benchmarks
- - get: geode-build-version
- trigger: true
- passed: *benchmark-inputs
- - put: concourse-metadata-resource
- - do:
- - task: run_benchmarks{{ run_var.title }}
- image: alpine-tools-image
- config:
- platform: linux
- params:
- AWS_ACCESS_KEY_ID: ((benchmarks-access-key-id))
- AWS_SECRET_ACCESS_KEY: ((benchmarks-secret-access-key))
- AWS_DEFAULT_REGION: us-west-2
- AWS_REGION: us-west-2
- ARTIFACT_BUCKET: ((artifact-bucket))
- BENCHMARKS_BRANCH: {{benchmarks.benchmark_branch}}
- BASELINE_BRANCH: {{benchmarks.baseline_branch}}
- BASELINE_VERSION: {{benchmarks.baseline_version}}
- FLAGS: {{ run_var.flag }}
- TAG_POSTFIX: {{ run_var.title }}
- TEST_OPTIONS: {{ run_var.options }}
- run:
- path: geode-ci/ci/scripts/run_benchmarks.sh
- inputs:
- - name: geode
- - name: geode-ci
- - name: geode-benchmarks
- - name: concourse-metadata-resource
- outputs:
- - name: results
- timeout: 8h
- ensure:
- do:
- - task: cleanup_benchmarks
- image: alpine-tools-image
- config:
- platform: linux
- params:
- AWS_ACCESS_KEY_ID: ((benchmarks-access-key-id))
- AWS_SECRET_ACCESS_KEY: ((benchmarks-secret-access-key))
- AWS_DEFAULT_REGION: us-west-2
- AWS_REGION: us-west-2
- ARTIFACT_BUCKET: ((artifact-bucket))
- BASELINE_BRANCH: {{benchmarks.baseline_branch}}
- BASELINE_VERSION: {{benchmarks.baseline_version}}
- FLAGS: {{ run_var.flag }}
- TAG_POSTFIX: {{ run_var.title }}
- TEST_OPTIONS: {{ run_var.options }}
- run:
- path: geode-ci/ci/scripts/cleanup_benchmarks.sh
- inputs:
- - name: geode
- - name: geode-ci
- - name: geode-benchmarks
- - name: concourse-metadata-resource
- - name: results
-{% endfor %}
-{% if repository.upstream_fork != "apache" or repository.branch == "develop" or repository.branch.startswith("support/") %}
-- name: PublishArtifacts
- public: true
- plan:
- - aggregate:
- - get: geode-ci
- passed: &publish-passed-inputs
- {{- all_gating_jobs() | indent(6) }}
- - get: alpine-tools-image
- - get: geode
- passed: *publish-passed-inputs
- trigger: true
- - get: linux-builder-image-family
- - aggregate:
- - get: geode-build-version
- trigger: true
- passed: *publish-passed-inputs
- - put: concourse-metadata-resource
- {{ init_retry()|indent(4) }}
- - task: create_instance
- image: alpine-tools-image
- config:
- platform: linux
- params:
- {{ common_instance_params(publish_artifacts) | indent(8) }}
- GEODE_BRANCH: {{repository.branch}}
- GEODE_FORK: {{repository.fork}}
- JAVA_BUILD_VERSION: {{ java_build_version.version }}
- IMAGE_FAMILY_NAME: ((pipeline-prefix))linux-geode-builder
- run:
- path: geode-ci/ci/scripts/create_instance.sh
- inputs:
- - name: concourse-metadata-resource
- - name: geode-ci
- - name: geode
- - name: attempts-log
- path: old
- outputs:
- - name: instance-data
- - name: attempts-log
- path: new
- timeout: 20m
- attempts: 5
- - task: rsync_code_up
- image: alpine-tools-image
- config:
- platform: linux
- run:
- path: geode-ci/ci/scripts/rsync_code_up.sh
- inputs:
- - name: geode-ci
- - name: geode
- - name: instance-data
- timeout: 5m
- - task: publish
- image: alpine-tools-image
- config:
- platform: linux
- params:
- MAINTENANCE_VERSION: ((geode-build-branch))
- ARTIFACT_BUCKET: ((artifact-bucket))
- SERVICE_ACCOUNT: ((!concourse-gcp-account))
- JAVA_BUILD_VERSION: {{ java_build_version.version }}
- MAVEN_SNAPSHOT_BUCKET: ((maven-snapshot-bucket))
- GRADLE_GLOBAL_ARGS: ((gradle-global-args))
- run:
- path: geode-ci/ci/scripts/execute_publish.sh
- inputs:
- - name: instance-data
- - name: geode
- - name: geode-ci
- - name: geode-build-version
- ensure:
- do:
- - task: delete_instance
- image: alpine-tools-image
- config:
- platform: linux
- run:
- path: geode-ci/ci/scripts/delete_instance.sh
- inputs:
- - name: geode-ci
- - name: instance-data
- timeout: 1h
-
-{% endif %}
-
{%- for test in tests if not test.name=="StressNew" %}
{%- set parameters = {} %}
{%- do deep_merge(parameters, test) %}
diff --git a/ci/pipelines/shared/jinja.variables.yml b/ci/pipelines/shared/jinja.variables.yml
index a23324c..be9a2e4 100644
--- a/ci/pipelines/shared/jinja.variables.yml
+++ b/ci/pipelines/shared/jinja.variables.yml
@@ -84,7 +84,7 @@ tests:
DUNIT_PARALLEL_FORKS: '0'
EXECUTE_TEST_TIMEOUT: 30m
GRADLE_TASK: test
- MAX_IN_FLIGHT: 1
+ MAX_IN_FLIGHT: 5
ONLY_JDK: 8
PARALLEL_DUNIT: 'false'
PARALLEL_GRADLE: 'true'
@@ -97,7 +97,7 @@ tests:
DUNIT_PARALLEL_FORKS: '0'
EXECUTE_TEST_TIMEOUT: 1h30m
GRADLE_TASK: acceptanceTest
- MAX_IN_FLIGHT: 2
+ MAX_IN_FLIGHT: 5
PARALLEL_DUNIT: 'false'
PARALLEL_GRADLE: 'false'
PLATFORM: linux
@@ -109,7 +109,7 @@ tests:
DUNIT_PARALLEL_FORKS: '24'
EXECUTE_TEST_TIMEOUT: 3h00m
GRADLE_TASK: distributedTest
- MAX_IN_FLIGHT: 3
+ MAX_IN_FLIGHT: 5
PARALLEL_DUNIT: 'true'
PLATFORM: linux
RAM: '250'
@@ -120,7 +120,7 @@ tests:
DUNIT_PARALLEL_FORKS: '48'
EXECUTE_TEST_TIMEOUT: 40m
GRADLE_TASK: integrationTest
- MAX_IN_FLIGHT: 1
+ MAX_IN_FLIGHT: 5
PARALLEL_DUNIT: 'true'
PLATFORM: linux
RAM: '90'
@@ -131,88 +131,8 @@ tests:
DUNIT_PARALLEL_FORKS: '48'
EXECUTE_TEST_TIMEOUT: 1h
GRADLE_TASK: upgradeTest
- MAX_IN_FLIGHT: 2
+ MAX_IN_FLIGHT: 5
PARALLEL_DUNIT: 'true'
PLATFORM: linux
RAM: '210'
name: Upgrade
-- ARTIFACT_SLUG: stressnewtestfiles
- CALL_STACK_TIMEOUT: '20700'
- CPUS: '96'
- DUNIT_PARALLEL_FORKS: '24'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: repeatTest
- PARALLEL_DUNIT: 'true'
- PARALLEL_GRADLE: 'false'
- PLATFORM: linux
- RAM: '280'
- name: StressNew
-- ARTIFACT_SLUG: windows-acceptancetestfiles
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: :geode-assembly:acceptanceTest
- MAX_IN_FLIGHT: 3
- PARALLEL_DUNIT: 'false'
- PARALLEL_GRADLE: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsAcceptance
-- ARTIFACT_SLUG: windows-gfshdistributedtest
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: distributedTest
- GRADLE_TASK_OPTIONS: -PtestCategory=org.apache.geode.test.junit.categories.GfshTest
- MAX_IN_FLIGHT: 5
- PARALLEL_DUNIT: 'false'
- PARALLEL_GRADLE: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsGfshDistributed
-- ARTIFACT_SLUG: windows-integrationtestfiles
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: integrationTest
- GRADLE_TASK_OPTIONS: -x geode-core:integrationTest
- MAX_IN_FLIGHT: 2
- PARALLEL_DUNIT: 'false'
- PARALLEL_GRADLE: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsIntegration
-- ARTIFACT_SLUG: windows-coreintegrationtestfiles
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: geode-core:integrationTest
- MAX_IN_FLIGHT: 6
- PARALLEL_DUNIT: 'false'
- PARALLEL_GRADLE: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsCoreIntegration
-- ARTIFACT_SLUG: windows-unittestfiles
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: test
- MAX_IN_FLIGHT: 1
- PARALLEL_DUNIT: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsUnit
-- ARTIFACT_SLUG: apicheck
- CALL_STACK_TIMEOUT: '20700'
- CPUS: '4'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 1h
- GRADLE_TASK: geode-assembly:japicmp
- MAX_IN_FLIGHT: 1
- ONLY_JDK: 8
- PARALLEL_DUNIT: 'false'
- PARALLEL_GRADLE: 'false'
- PLATFORM: linux
- RAM: '16'
- name: ApiCheck