You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@geode.apache.org by ig...@apache.org on 2020/04/09 22:59:29 UTC
[geode] branch mass-test-run updated (ccedf05 -> e30246c)
This is an automated email from the ASF dual-hosted git repository.
igodwin pushed a change to branch mass-test-run
in repository https://gitbox.apache.org/repos/asf/geode.git.
discard ccedf05 remove meta-* jobs that we won't use for mass-test-run
discard 2a828ad Mass-test-run branch, no windows or publication jobs
add 2ac3de7 GEODE-7921: NullPointerExceptions logged during auto-reconnect (#4898)
add 88c165c GEODE-7926: GMSMemberData is doing unnecessary reverse-DNS lookups
add 5864f2a GEODE-7926: GMSMemberData is doing unnecessary reverse-DNS lookups (#4894)
add 8f24abb GEODE-7920: Geode UDP INT thread found processing cache operations
add 6f8c74b removing commented-out code
add 6312b32 Merge pull request #4893 from apache/feature/GEODE-7920
add 189ae76 GEODE-7939: membership heartbeat messages aren't designated high-priority
add 8fd0f6f Merge pull request #4892 from apache/feature/GEODE-7939
add 3c99931 GEODE-7946: Fix redis publish/subscribe leaking netty buffers (#4902)
add abd0f54 GEODE-7098: Tomcat8SessionsClientServerDUnitTest Tests were getting bind failures (#4903)
add 24203d8 GEODE-7892: Code improvements in ConnectionProxyJUnitTest
add 087e47a Merge pull request #4827 from Nordix/feature/GEODE-7892
add 552cdea GEODE-7917: change thrown exception type for SSL to IOException (#4858)
add e7f6faf GEODE-7950: Force TLS1.2 for installing PackageProvider=NuGet (#4908)
add d2f18fd GEODE-7156: add docs for security-auth-token-enabled-components (#4910)
add d663864 GEODE-7864: Remove null checks that are not required Part-2. (#4881)
add d4b7c14 GEODE-7864: Closing the query statements after execution. (#4873)
add 1b1ad6b GEODE-7947 Implement tests for EXPIRE-related functionality (#4904)
add 27614c2 GEODE-7864: Prevented non synchronized override of synchronized methods. (#4872)
add 7d8834f GEODE-7460: Disable network partition detection, which is not needed for this test and is causing flakiness (#4911)
add 6263f1f GEODE-7938: added keys query param to GET, PUT, and DELETE /{region} endpoints (#4901)
add ead319c GEODE-7970: bump spring to latest (#4927)
add e8a5dbf GEODE-7966: User Guide - properties list - reformat oversized table (#4923)
add b4fb6c8 Revert "GEODE-7938: added keys query param to GET, PUT, and DELETE /{region} endpoints (#4901)"
add 8e3c7a2 Merge pull request #4929 from jmelchio/slash-revert
add 37e2d98 GEODE-7945: Replace Map with ConcurrentMap for GemFireCacheImpl.rootRegions (#4899)
add 8f9f4b7 GEODE-7969: bump netty version to latest (#4933)
add 2765825 GEODE-7852: SNI extension support
add 1260318 addressing reviews
add d301de9 addressing reviews
add 381714e fixing new assertions in unit test and retriggering CI tasks
add 47cc74f reverting ProcessManager changes - evidently the rolling upgrade test code needs some of this stuff
add a12d1fe Merge pull request #4920 from apache/feature/GEODE-7852doc
add a73dffc GEODE-7976: make "config" param optional when deploy using CMS. (#4930)
add 4f2a604 GEODE-7852: test SNI client against a geode (2-node) cluster (#4925)
add 464c749 GEODE-7965: Modify Redis SADD command to use region.compute (#4918)
add 3062423 Geode 7751: fix for Tomcat9CachingClientServerTest.containersShouldExpireInSetTimeframe (#4931)
add 63c681d GEODE-7664: calling RegionConfigRealizer.exists methods doesn't need … (#4932)
new 80321e8 Mass-test-run branch, no windows or publication jobs
new e30246c remove meta-* jobs that we won't use for mass-test-run
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (ccedf05)
\
N -- N -- N refs/heads/mass-test-run (e30246c)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../src/test/resources/expected-pom.xml | 38 ++---
.../gradle/plugins/DependencyConstraints.groovy | 6 +-
.../google-windows-geode-builder/build_image.sh | 3 +-
.../windows-packer.json | 1 +
.../modules/session/TestSessionsTomcat8Base.java | 23 +--
.../Tomcat8SessionsClientServerDUnitTest.java | 52 +++++-
.../modules/session/Tomcat8SessionsDUnitTest.java | 5 +-
...eTest.java => DualServerSNIAcceptanceTest.java} | 120 +++++++++++---
...est.java => SingleServerSNIAcceptanceTest.java} | 23 ++-
.../client/sni/geode-config/locator-keystore.jks | Bin 2028 -> 0 bytes
.../sni/geode-config/locator-maeve-keystore.jks | Bin 0 -> 2048 bytes
.../geode-config/server-clementine-keystore.jks | Bin 0 -> 2059 bytes
.../sni/geode-config/server-dolores-keystore.jks | Bin 0 -> 2050 bytes
.../client/sni/geode-config/server-keystore.jks | Bin 2025 -> 0 bytes
.../geode/client/sni/geode-config/truststore.jks | Bin 6023 -> 8095 bytes
.../org/apache/geode/client/sni/haproxy.cfg | 14 +-
.../geode/client/sni/scripts/geode-starter-2.gfsh | 23 +++
.../geode/client/sni/scripts/geode-starter.gfsh | 4 +-
.../apache/geode/session/tests/CargoTestBase.java | 121 +++++++-------
.../session/tests/TomcatClientServerTest.java | 7 +-
.../integrationTest/resources/assembly_content.txt | 12 +-
.../resources/dependency_classpath.txt | 6 +-
.../integrationTest/resources/expected_jars.txt | 2 -
.../distributed/DistributedMemberDUnitTest.java | 2 +
.../cache/client/ClientCacheFactoryJUnitTest.java | 10 +-
.../tcpserver/TCPClientSSLIntegrationTest.java | 49 ++++++
.../tier/sockets/ConnectionProxyJUnitTest.java | 97 ++++-------
.../ClientCacheFactoryJUnitTest_single_pool.xml | 9 ++
.../admin/internal/ManagedSystemMemberImpl.java | 17 +-
.../geode/cache/client/proxy/SniSocketFactory.java | 30 +++-
.../internal/ClusterOperationExecutors.java | 2 +-
.../distributed/internal/DistributionMessage.java | 19 ++-
.../distributed/internal/ShutdownMessage.java | 19 ---
.../ThrottlingMemLinkedQueueWithDMStats.java | 2 +-
.../org/apache/geode/internal/AbstractConfig.java | 2 +-
.../apache/geode/internal/cache/BucketAdvisor.java | 2 +-
.../cache/DestroyPartitionedRegionMessage.java | 2 +-
.../geode/internal/cache/DistTXCommitMessage.java | 2 +-
.../internal/cache/DistTXPrecommitMessage.java | 2 +-
.../internal/cache/DistTXRollbackMessage.java | 2 +-
.../apache/geode/internal/cache/FilterProfile.java | 42 ++---
.../geode/internal/cache/GemFireCacheImpl.java | 177 ++++++++++-----------
.../cache/LatestLastAccessTimeMessage.java | 26 +--
.../geode/internal/cache/PartitionedRegion.java | 2 +-
.../geode/internal/cache/TXCommitMessage.java | 2 +-
.../partitioned/BecomePrimaryBucketMessage.java | 2 +-
.../partitioned/DeposePrimaryBucketMessage.java | 2 +-
.../partitioned/FetchPartitionDetailsMessage.java | 2 +-
.../cache/partitioned/MoveBucketMessage.java | 2 +-
.../partitioned/PersistentBucketRecoverer.java | 6 +-
.../cache/partitioned/RemoveBucketMessage.java | 2 +-
.../internal/cache/partitioned/SizeMessage.java | 2 +-
.../geode/internal/cache/tx/RemoteSizeMessage.java | 2 +-
.../cache/wan/serial/BatchDestroyOperation.java | 2 +-
.../geode/internal/statistics/ArchiveSplitter.java | 2 +-
.../tcp/ImmutableByteBufferInputStream.java | 4 +-
.../realizers/RegionConfigRealizer.java | 18 ++-
.../pdx/internal/AutoSerializableManager.java | 21 +--
.../internal/DistributionMessageTest.java | 12 ++
.../cache/LatestLastAccessTimeMessageTest.java | 43 +++++
.../realizers/RegionConfigRealizerTest.java | 27 ++++
.../running/cluster-management-service.html.md.erb | 9 +-
.../topics/client-cache-elements-list.html.md.erb | 1 +
.../reference/topics/client-cache.html.md.erb | 26 +++
.../topics/gemfire_properties.html.md.erb | 52 +++---
.../setting_up_a_client_server_system.html.md.erb | 17 +-
.../geode/test/dunit/internal/ProcessManager.java | 7 +-
.../org/apache/geode/internal/jta/CacheUtils.java | 16 +-
.../CommandAvailabilityIndicatorTestHelper.java | 2 +-
.../internal/filesystem/FileInputStream.java | 2 +-
.../geode/management/configuration/HasFile.java | 2 +
.../membership/gms/GMSMemberDataJUnitTest.java | 49 +++---
.../membership/gms/GMSMembershipJUnitTest.java | 30 ----
.../internal/membership/gms/TestMessage.java} | 47 ++----
.../gms/messenger/JGroupsMessengerJUnitTest.java | 21 +++
.../internal/membership/gms/GMSMemberData.java | 4 +-
.../membership/gms/messages/HeartbeatMessage.java | 5 +
.../gms/messages/HeartbeatRequestMessage.java | 6 +
.../membership/gms/messenger/JGroupsMessenger.java | 12 +-
...Test.java => ExpireAtDockerAcceptanceTest.java} | 20 +--
...ceTest.java => ExpireDockerAcceptanceTest.java} | 20 +--
.../geode/redis/GeoDockerAcceptanceTest.java | 2 +-
.../geode/redis/HashesDockerAcceptanceTest.java | 2 +-
.../geode/redis/ListsDockerAcceptanceTest.java | 2 +-
...eTest.java => PexpireDockerAcceptanceTest.java} | 20 +--
.../geode/redis/PubSubDockerAcceptanceTest.java | 2 +-
.../geode/redis/RenameDockerAcceptanceTest.java | 2 +-
.../geode/redis/SetsDockerAcceptanceTest.java | 2 +-
.../redis/SortedSetsDockerAcceptanceTest.java | 2 +-
.../geode/redis/StringsDockerAcceptanceTest.java | 2 +-
.../org/apache/geode/redis/RedisDistDUnitTest.java | 93 +++++++++--
.../apache/geode/redis/SetsIntegrationTest.java | 43 ++++-
.../redis/general/ExpireAtIntegrationTest.java | 128 +++++++++++++++
.../geode/redis/general/ExpireIntegrationTest.java | 14 +-
.../redis/general/PexpireIntegrationTest.java | 12 +-
.../geode/redis/internal/AbstractSubscription.java | 10 +-
.../redis/internal/ExecutionHandlerContext.java | 6 +-
.../redis/internal/executor/ExpireAtExecutor.java | 46 ++++--
.../redis/internal/executor/ExpireExecutor.java | 21 ++-
.../executor/pubsub/PsubscribeExecutor.java | 5 +-
.../executor/pubsub/SubscribeExecutor.java | 5 +-
.../redis/internal/executor/set/SAddExecutor.java | 57 +++----
...nitTest.java => ExpireAtExecutorJUnitTest.java} | 53 +++---
.../executor/general/ExpireExecutorJUnitTest.java | 2 +-
...itTest.java => PExpireAtExecutorJUnitTest.java} | 54 +++----
.../tcpserver/AdvancedSocketCreatorImpl.java | 10 +-
.../distributed/internal/tcpserver/TcpClient.java | 8 +
.../internal/rest/DeploymentIntegrationTest.java | 86 ++++++++++
.../DeploymentManagementController.java | 14 +-
.../DeploymentManagementControllerTest.java | 4 +-
110 files changed, 1397 insertions(+), 784 deletions(-)
copy geode-assembly/src/acceptanceTest/java/org/apache/geode/client/sni/{ClientSNIAcceptanceTest.java => DualServerSNIAcceptanceTest.java} (50%)
rename geode-assembly/src/acceptanceTest/java/org/apache/geode/client/sni/{ClientSNIAcceptanceTest.java => SingleServerSNIAcceptanceTest.java} (77%)
delete mode 100644 geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/geode-config/locator-keystore.jks
create mode 100644 geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/geode-config/locator-maeve-keystore.jks
create mode 100644 geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/geode-config/server-clementine-keystore.jks
create mode 100644 geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/geode-config/server-dolores-keystore.jks
delete mode 100644 geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/geode-config/server-keystore.jks
create mode 100644 geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/geode-starter-2.gfsh
rename geode-core/src/{test => integrationTest}/resources/org/apache/geode/cache/client/ClientCacheFactoryJUnitTest_single_pool.xml (82%)
create mode 100644 geode-core/src/test/java/org/apache/geode/internal/cache/LatestLastAccessTimeMessageTest.java
copy geode-membership/src/{main/java/org/apache/geode/distributed/internal/membership/gms/messages/HeartbeatMessage.java => integrationTest/java/org/apache/geode/distributed/internal/membership/gms/TestMessage.java} (50%)
mode change 100755 => 100644
copy geode-redis/src/acceptanceTest/java/org/apache/geode/redis/{ListsDockerAcceptanceTest.java => ExpireAtDockerAcceptanceTest.java} (68%)
mode change 100755 => 100644
copy geode-redis/src/acceptanceTest/java/org/apache/geode/redis/{ListsDockerAcceptanceTest.java => ExpireDockerAcceptanceTest.java} (68%)
mode change 100755 => 100644
copy geode-redis/src/acceptanceTest/java/org/apache/geode/redis/{ListsDockerAcceptanceTest.java => PexpireDockerAcceptanceTest.java} (68%)
mode change 100755 => 100644
create mode 100644 geode-redis/src/integrationTest/java/org/apache/geode/redis/general/ExpireAtIntegrationTest.java
copy geode-redis/src/test/java/org/apache/geode/redis/internal/executor/general/{ExpireExecutorJUnitTest.java => ExpireAtExecutorJUnitTest.java} (66%)
copy geode-redis/src/test/java/org/apache/geode/redis/internal/executor/general/{ExpireExecutorJUnitTest.java => PExpireAtExecutorJUnitTest.java} (66%)
create mode 100644 geode-web-management/src/integrationTest/java/org/apache/geode/management/internal/rest/DeploymentIntegrationTest.java
[geode] 01/02: Mass-test-run branch, no windows or publication jobs
Posted by ig...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
igodwin pushed a commit to branch mass-test-run
in repository https://gitbox.apache.org/repos/asf/geode.git
commit 80321e8f80946ce5f87efea0c723ec165d17dc76
Author: Robert Houghton <rh...@pivotal.io>
AuthorDate: Mon Jan 27 15:20:46 2020 -0800
Mass-test-run branch, no windows or publication jobs
Co-authored-by: Robert Houghton <rh...@pivotal.io>
Co-authored-by: Mark Hanson <mh...@pivotal.io>
---
ci/pipelines/geode-build/jinja.template.yml | 310 +---------------------------
ci/pipelines/shared/jinja.variables.yml | 77 +------
2 files changed, 9 insertions(+), 378 deletions(-)
diff --git a/ci/pipelines/geode-build/jinja.template.yml b/ci/pipelines/geode-build/jinja.template.yml
index da6e399..f9dcbda 100644
--- a/ci/pipelines/geode-build/jinja.template.yml
+++ b/ci/pipelines/geode-build/jinja.template.yml
@@ -92,66 +92,6 @@ GRADLE_GLOBAL_ARGS: ((gradle-global-args))
{%- endfor -%}
{% endmacro %}
-groups:
-- name: main
- jobs:
- - {{ build_test.name }}
- {{- all_gating_jobs() | indent(2) }}
- - UpdatePassingTokens
- {%- if repository.upstream_fork != "apache" or repository.branch == "develop" %}
- - PublishArtifacts
- {%- endif %}
- {%- for flavor in (benchmarks.flavors) %}
- - Benchmark{{flavor.title}}
- {%- endfor %}
-- name: complete
- jobs:
- - {{ build_test.name }}
- {%- for test in (tests) if not test.name=="StressNew" -%}
- {%- for java_test_version in (java_test_versions) %}
- - {{test.name}}Test{{java_test_version.name}}
- {%- endfor -%}
- {%- endfor %}
- - UpdatePassingTokens
- {%- if repository.upstream_fork != "apache" or repository.branch == "develop" %}
- - PublishArtifacts
- {%- endif %}
- {%- for flavor in (benchmarks.flavors) %}
- - Benchmark{{flavor.title}}
- {%- endfor %}
-- name: linux
- jobs:
- - {{ build_test.name }}
- {%- for test in (tests) if test.PLATFORM=="linux" and not test.name=="StressNew" -%}
- {% for java_test_version in (java_test_versions) %}
- - {{test.name}}Test{{java_test_version.name}}
- {%- endfor -%}
- {%- endfor %}
- {%- for flavor in (benchmarks.flavors) %}
- - Benchmark{{flavor.title}}
- {%- endfor %}
-- name: windows
- jobs:
- - {{ build_test.name }}
- {%- for test in (tests) if test.PLATFORM=="windows" -%}
- {% for java_test_version in (java_test_versions) %}
- - {{test.name}}Test{{java_test_version.name}}
- {%- endfor -%}
- {%- endfor %}
-{%- for java_test_version in (java_test_versions) %}
-- name: {{java_test_version.name}}
- jobs:
- - {{ build_test.name }}
- {%- for test in (tests) if not test.name=="StressNew" %}
- - {{test.name}}Test{{java_test_version.name}}
- {%- endfor -%}
-{%- endfor %}
-- name: Semver Management
- jobs:
- {%- for semverPiece in ['major', 'minor', 'patch'] %}
- - Bump{{ semverPiece.title() }}
- {%- endfor %}
-
resources:
- name: concourse-metadata-resource
type: concourse-metadata-resource
@@ -173,12 +113,7 @@ resources:
paths:
- ci/*
{{ github_access() | indent(4) }}
-- name: geode-benchmarks
- type: git
- source:
- branch: {{benchmarks.benchmark_branch}}
- depth: 1
- uri: https://github.com/apache/geode-benchmarks.git
+
- name: geode-build-version
type: semver
source:
@@ -187,12 +122,7 @@ resources:
initial_version: {{ metadata.initial_version }}
json_key: ((!concourse-gcp-key))
key: semvers/((pipeline-prefix))((geode-build-branch))/version
-- name: geode-passing-tokens
- type: gcs-resource
- source:
- bucket: ((artifact-bucket))
- json_key: ((concourse-gcp-key))
- versioned_file: semvers/((pipeline-prefix))((geode-build-branch))/passing-build-tokens.json
+
- name: alpine-tools-image
type: docker-image
source:
@@ -201,12 +131,6 @@ resources:
repository: gcr.io/((gcp-project))/((pipeline-prefix))alpine-tools
tag: latest
-- name: windows-builder-image-family
- type: gci
- source:
- key: ((concourse-gcp-key))
- family_project: ((gcp-project))
- family: ((pipeline-prefix))windows-geode-builder
- name: linux-builder-image-family
type: gci
@@ -233,16 +157,8 @@ resource_types:
source:
repository: frodenas/gcs-resource
jobs:
-{% for semverPiece in ['major', 'minor', 'patch'] -%}
-- name: Bump{{ semverPiece.title() }}
- serial: true
- plan:
- - get: geode-build-version
- params: { bump: {{ semverPiece }} }
- - put: geode-build-version
- params:
- file: geode-build-version/number
-{% endfor %}
+
+
- name: {{build_test.name}}
public: true
max_in_flight: {{build_test.MAX_IN_FLIGHT}}
@@ -388,224 +304,6 @@ jobs:
- name: instance-data
timeout: 1h
-- name: UpdatePassingTokens
- public: true
- serial: true
- plan:
- - aggregate:
- - get: alpine-tools-image
- - get: geode
- passed: &update-token-passed-anchor
-{%- if repository.upstream_fork != "apache" or repository.branch == "develop" %}
- - PublishArtifacts
-{% else %}
- {% for flavor in (benchmarks.flavors) %}
- - Benchmark{{flavor.title}}
- {% endfor %}
-{% endif %}
- trigger: true
- - get: geode-build-version
- trigger: true
- passed: *update-token-passed-anchor
- - task: couple-sha-and-build-id
- image: alpine-tools-image
- config:
- platform: linux
- inputs:
- - name: geode
- - name: geode-build-version
- outputs:
- - name: geode-passing-tokens
- run:
- path: bash
- args:
- - -cx
- - |
- pushd geode
- GEODE_SHA=$(git rev-parse HEAD)
- popd
- GEODE_SEMVER=$(cat geode-build-version/number)
-
- GS_PATH=gs://((artifact-bucket))/semvers/((pipeline-prefix))((geode-build-branch))/passing-build-tokens.json
- CURRENT_PASSING_SHA=$(gsutil cat ${GS_PATH} | jq -r .ref)
- set -e
- # Check that the incoming GEODE_SHA is a descendent of the currently stored value.
- # Keeps us from winding back the repository in the case of an out-of-order pipeline pass
- if [ -z "${CURRENT_PASSING_SHA}" ] || (cd geode; git merge-base --is-ancestor ${CURRENT_PASSING_SHA} ${GEODE_SHA}); then
- cat > geode-passing-tokens/passing-build-tokens.json <<JSON
- {
- "ref": "${GEODE_SHA}",
- "semver": "${GEODE_SEMVER}"
- }
- JSON
- fi
- - aggregate:
- - put: geode-passing-tokens
- params:
- file: geode-passing-tokens/passing-build-tokens.json
-
-{% for run_var in (benchmarks.flavors) %}
-- name: Benchmark{{ run_var.title }}
- public: true
- max_in_flight: 3
- plan:
- - get: geode-ci
- passed:
- {{ all_gating_jobs() | indent(4) }}
- - get: alpine-tools-image
- - aggregate:
- - get: geode
- passed:
- {{ all_gating_jobs() | indent(6) }}
- trigger: true
- - get: geode-benchmarks
- - get: geode-build-version
- trigger: true
- passed:
- {{ all_gating_jobs() | indent(6) }}
- - put: concourse-metadata-resource
- - do:
- - task: run_benchmarks{{ run_var.title }}
- image: alpine-tools-image
- config:
- platform: linux
- params:
- AWS_ACCESS_KEY_ID: ((benchmarks-access-key-id))
- AWS_SECRET_ACCESS_KEY: ((benchmarks-secret-access-key))
- AWS_DEFAULT_REGION: us-west-2
- AWS_REGION: us-west-2
- ARTIFACT_BUCKET: ((artifact-bucket))
- BENCHMARKS_BRANCH: {{benchmarks.benchmark_branch}}
- BASELINE_BRANCH: {{benchmarks.baseline_branch}}
- BASELINE_VERSION: {{benchmarks.baseline_version}}
- FLAGS: {{ run_var.flag }}
- TAG_POSTFIX: {{ run_var.title }}
- TEST_OPTIONS: {{ run_var.options }}
- run:
- path: geode-ci/ci/scripts/run_benchmarks.sh
- inputs:
- - name: geode
- - name: geode-ci
- - name: geode-benchmarks
- - name: concourse-metadata-resource
- outputs:
- - name: results
- timeout: 8h
- ensure:
- do:
- - task: cleanup_benchmarks
- image: alpine-tools-image
- config:
- platform: linux
- params:
- AWS_ACCESS_KEY_ID: ((benchmarks-access-key-id))
- AWS_SECRET_ACCESS_KEY: ((benchmarks-secret-access-key))
- AWS_DEFAULT_REGION: us-west-2
- AWS_REGION: us-west-2
- ARTIFACT_BUCKET: ((artifact-bucket))
- BASELINE_BRANCH: {{benchmarks.baseline_branch}}
- BASELINE_VERSION: {{benchmarks.baseline_version}}
- FLAGS: {{ run_var.flag }}
- TAG_POSTFIX: {{ run_var.title }}
- TEST_OPTIONS: {{ run_var.options }}
- run:
- path: geode-ci/ci/scripts/cleanup_benchmarks.sh
- inputs:
- - name: geode
- - name: geode-ci
- - name: geode-benchmarks
- - name: concourse-metadata-resource
- - name: results
-{% endfor %}
-{% if repository.upstream_fork != "apache" or repository.branch == "develop" %}
-- name: PublishArtifacts
- public: true
- plan:
- - aggregate:
- - get: geode-ci
- passed: &publish-passed-inputs
- {% for flavor in (benchmarks.flavors) %}
- - Benchmark{{flavor.title}}
- {% endfor %}
- - get: alpine-tools-image
- - get: geode
- passed: *publish-passed-inputs
- trigger: true
- - get: linux-builder-image-family
- - aggregate:
- - get: geode-build-version
- trigger: true
- passed: *publish-passed-inputs
- - put: concourse-metadata-resource
- {{ init_retry()|indent(4) }}
- - task: create_instance
- image: alpine-tools-image
- config:
- platform: linux
- params:
- {{ common_instance_params(publish_artifacts) | indent(8) }}
- GEODE_BRANCH: {{repository.branch}}
- GEODE_FORK: {{repository.fork}}
- JAVA_BUILD_VERSION: {{ java_build_version.version }}
- IMAGE_FAMILY_NAME: ((pipeline-prefix))linux-geode-builder
- run:
- path: geode-ci/ci/scripts/create_instance.sh
- inputs:
- - name: concourse-metadata-resource
- - name: geode-ci
- - name: geode
- - name: attempts-log
- path: old
- outputs:
- - name: instance-data
- - name: attempts-log
- path: new
- timeout: 20m
- attempts: 5
- - task: rsync_code_up
- image: alpine-tools-image
- config:
- platform: linux
- run:
- path: geode-ci/ci/scripts/rsync_code_up.sh
- inputs:
- - name: geode-ci
- - name: geode
- - name: instance-data
- timeout: 5m
- - task: publish
- image: alpine-tools-image
- config:
- platform: linux
- params:
- MAINTENANCE_VERSION: ((geode-build-branch))
- ARTIFACT_BUCKET: ((artifact-bucket))
- SERVICE_ACCOUNT: ((!concourse-gcp-account))
- JAVA_BUILD_VERSION: {{ java_build_version.version }}
- MAVEN_SNAPSHOT_BUCKET: ((maven-snapshot-bucket))
- GRADLE_GLOBAL_ARGS: ((gradle-global-args))
- run:
- path: geode-ci/ci/scripts/execute_publish.sh
- inputs:
- - name: instance-data
- - name: geode
- - name: geode-ci
- - name: geode-build-version
- ensure:
- do:
- - task: delete_instance
- image: alpine-tools-image
- config:
- platform: linux
- run:
- path: geode-ci/ci/scripts/delete_instance.sh
- inputs:
- - name: geode-ci
- - name: instance-data
- timeout: 1h
-
-{% endif %}
-
{%- for test in tests if not test.name=="StressNew" %}
{%- set parameters = {} %}
{%- do deep_merge(parameters, test) %}
diff --git a/ci/pipelines/shared/jinja.variables.yml b/ci/pipelines/shared/jinja.variables.yml
index a82cdca..3931254 100644
--- a/ci/pipelines/shared/jinja.variables.yml
+++ b/ci/pipelines/shared/jinja.variables.yml
@@ -84,7 +84,7 @@ tests:
DUNIT_PARALLEL_FORKS: '0'
EXECUTE_TEST_TIMEOUT: 30m
GRADLE_TASK: test
- MAX_IN_FLIGHT: 1
+ MAX_IN_FLIGHT: 5
PARALLEL_DUNIT: 'false'
PARALLEL_GRADLE: 'true'
PLATFORM: linux
@@ -96,7 +96,7 @@ tests:
DUNIT_PARALLEL_FORKS: '0'
EXECUTE_TEST_TIMEOUT: 1h30m
GRADLE_TASK: acceptanceTest
- MAX_IN_FLIGHT: 2
+ MAX_IN_FLIGHT: 5
PARALLEL_DUNIT: 'false'
PARALLEL_GRADLE: 'false'
PLATFORM: linux
@@ -108,7 +108,7 @@ tests:
DUNIT_PARALLEL_FORKS: '24'
EXECUTE_TEST_TIMEOUT: 3h00m
GRADLE_TASK: distributedTest
- MAX_IN_FLIGHT: 3
+ MAX_IN_FLIGHT: 5
PARALLEL_DUNIT: 'true'
PLATFORM: linux
RAM: '250'
@@ -119,7 +119,7 @@ tests:
DUNIT_PARALLEL_FORKS: '48'
EXECUTE_TEST_TIMEOUT: 40m
GRADLE_TASK: integrationTest
- MAX_IN_FLIGHT: 1
+ MAX_IN_FLIGHT: 5
PARALLEL_DUNIT: 'true'
PLATFORM: linux
RAM: '90'
@@ -130,75 +130,8 @@ tests:
DUNIT_PARALLEL_FORKS: '48'
EXECUTE_TEST_TIMEOUT: 1h
GRADLE_TASK: upgradeTest
- MAX_IN_FLIGHT: 2
+ MAX_IN_FLIGHT: 5
PARALLEL_DUNIT: 'true'
PLATFORM: linux
RAM: '210'
name: Upgrade
-- ARTIFACT_SLUG: stressnewtestfiles
- CALL_STACK_TIMEOUT: '20700'
- CPUS: '96'
- DUNIT_PARALLEL_FORKS: '24'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: repeatTest
- PARALLEL_DUNIT: 'true'
- PARALLEL_GRADLE: 'false'
- PLATFORM: linux
- RAM: '280'
- name: StressNew
-- ARTIFACT_SLUG: windows-acceptancetestfiles
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: :geode-assembly:acceptanceTest
- MAX_IN_FLIGHT: 3
- PARALLEL_DUNIT: 'false'
- PARALLEL_GRADLE: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsAcceptance
-- ARTIFACT_SLUG: windows-gfshdistributedtest
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: distributedTest
- GRADLE_TASK_OPTIONS: -PtestCategory=org.apache.geode.test.junit.categories.GfshTest
- MAX_IN_FLIGHT: 5
- PARALLEL_DUNIT: 'false'
- PARALLEL_GRADLE: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsGfshDistributed
-- ARTIFACT_SLUG: windows-integrationtestfiles
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: integrationTest
- GRADLE_TASK_OPTIONS: -x geode-core:integrationTest
- MAX_IN_FLIGHT: 2
- PARALLEL_DUNIT: 'false'
- PARALLEL_GRADLE: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsIntegration
-- ARTIFACT_SLUG: windows-coreintegrationtestfiles
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: geode-core:integrationTest
- MAX_IN_FLIGHT: 6
- PARALLEL_DUNIT: 'false'
- PARALLEL_GRADLE: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsCoreIntegration
-- ARTIFACT_SLUG: windows-unittestfiles
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: test
- MAX_IN_FLIGHT: 1
- PARALLEL_DUNIT: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsUnit
[geode] 02/02: remove meta-* jobs that we won't use for
mass-test-run
Posted by ig...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
igodwin pushed a commit to branch mass-test-run
in repository https://gitbox.apache.org/repos/asf/geode.git
commit e30246c714ea20d6f9e50ccac7b2190a81cfbd7a
Author: Mark Hanson <mh...@pivotal.io>
AuthorDate: Mon Jan 27 15:27:19 2020 -0800
remove meta-* jobs that we won't use for mass-test-run
---
ci/pipelines/meta/jinja.template.yml | 206 +----------------------------------
1 file changed, 1 insertion(+), 205 deletions(-)
diff --git a/ci/pipelines/meta/jinja.template.yml b/ci/pipelines/meta/jinja.template.yml
index 792254f..a67e377 100644
--- a/ci/pipelines/meta/jinja.template.yml
+++ b/ci/pipelines/meta/jinja.template.yml
@@ -51,13 +51,7 @@ resources:
branch: ((!geode-build-branch))
paths:
- ci/pipelines/images/*
-- name: geode-reaper-pipeline
- type: git
- source:
- {{ github_access() | indent(4) }}
- branch: ((!geode-build-branch))
- paths:
- - ci/pipelines/reaper/*
+
- name: meta-mini-dockerfile
type: git
source:
@@ -71,35 +65,6 @@ resources:
username: ((!docker-username))
password: ((!docker-password))
repository: gcr.io/((!gcp-project))/((!sanitized-geode-fork))-((!sanitized-geode-build-branch))-meta-img
-{% if repository.fork == repository.upstream_fork %}
-- name: geode-examples-pipeline
- type: git
- source:
- {{ github_access() | indent(4) }}
- branch: ((!geode-build-branch))
- paths:
- - ci/pipelines/examples/*
- - ci/pipelines/shared/*
- - ci/pipelines/render.py
- - ci/pipelines/meta/meta.properties
-- name: geode-pr-pipeline
- type: git
- source:
- {{ github_access() | indent(4) }}
- branch: ((!geode-build-branch))
- paths:
- - ci/pipelines/pull-request/*
- - ci/pipelines/shared/*
- - ci/pipelines/render.py
- - ci/pipelines/meta/meta.properties
-{% endif %}
-- name: geode-metrics-pipeline
- type: git
- source:
- {{ github_access() | indent(4) }}
- branch: ((!geode-build-branch))
- paths:
- - ci/pipelines/metrics/*
jobs:
- name: set-images-pipeline
@@ -187,175 +152,6 @@ jobs:
- results/pipeline-vars.yml
-{% if repository.fork == repository.upstream_fork %}
-- name: set-examples-pipeline
- serial: true
- public: ((!public-pipelines))
- plan:
- - get: geode-examples-pipeline
- trigger: true
- - get: meta-mini-image
- trigger: true
- passed: [build-meta-mini-docker-image]
- - task: deploy-build
- image: meta-mini-image
- config:
- platform: linux
- inputs:
- - name: geode-examples-pipeline
- outputs:
- - name: results
- params:
- OUTPUT_DIRECTORY: results
- GEODE_BRANCH: ((geode-build-branch))
- GEODE_FORK: ((geode-fork))
- GEODE_REPO_NAME: ((geode-repo-name))
- REPOSITORY_PUBLIC: {{ repository.public }}
- ARTIFACT_BUCKET: ((artifact-bucket))
- UPSTREAM_FORK: {{ repository.upstream_fork }}
- PIPELINE_PREFIX: ((pipeline-prefix))
- SANITIZED_GEODE_BRANCH: ((sanitized-geode-build-branch))
- SANITIZED_GEODE_FORK: ((sanitized-geode-fork))
- GCP_PROJECT: ((gcp-project))
- PUBLIC_PIPELINES: ((public-pipelines))
- SEMVER_PRERELEASE_TOKEN: ((semver-prerelease-token))
- run:
- path: geode-examples-pipeline/ci/pipelines/examples/deploy_pipeline.sh
- - put: concourse
- params:
- pipelines:
- - name: ((!pipeline-prefix))examples
- team: ((!concourse-team))
- config_file: results/generated-pipeline.yml
- vars_files:
- - results/pipeline-vars.yml
-
-- name: set-pr-pipeline
- serial: true
- public: ((!public-pipelines))
- plan:
- - get: geode-pr-pipeline
- trigger: true
- - get: meta-mini-image
- trigger: true
- passed: [build-meta-mini-docker-image]
- - task: deploy-pr
- image: meta-mini-image
- config:
- platform: linux
- inputs:
- - name: geode-pr-pipeline
- outputs:
- - name: results
- params:
- OUTPUT_DIRECTORY: results
- GEODE_BRANCH: ((!geode-build-branch))
- GEODE_FORK: ((!geode-fork))
- GEODE_REPO_NAME: ((geode-repo-name))
- REPOSITORY_PUBLIC: {{ repository.public }}
- ARTIFACT_BUCKET: ((artifact-bucket))
- UPSTREAM_FORK: {{ repository.upstream_fork }}
- PIPELINE_PREFIX: ((pipeline-prefix))
- GCP_PROJECT: ((gcp-project))
- PUBLIC_PIPELINES: ((public-pipelines))
- GRADLE_GLOBAL_ARGS: ((gradle-global-args))
- run:
- path: geode-pr-pipeline/ci/pipelines/pull-request/deploy_pr_pipeline.sh
- - put: concourse
- params:
- pipelines:
- - name: ((!pipeline-prefix))pr
- team: ((!concourse-team))
- config_file: results/generated-pipeline.yml
- vars_files:
- - results/pipeline-vars.yml
-
-{% endif %}
-
-- name: set-metrics-pipeline
- serial: true
- public: ((!public-pipelines))
- plan:
- - get: geode-metrics-pipeline
- trigger: true
- - get: meta-mini-image
- trigger: true
- passed: [build-meta-mini-docker-image]
- - task: create-metrics-pipeline-yml
- image: meta-mini-image
- config:
- platform: linux
- inputs:
- - name: geode-metrics-pipeline
- outputs:
- - name: results
- params:
- OUTPUT_DIRECTORY: results
- GEODE_BRANCH: ((geode-build-branch))
- GEODE_FORK: ((geode-fork))
- GEODE_REPO_NAME: ((geode-repo-name))
- REPOSITORY_PUBLIC: {{ repository.public }}
- ARTIFACT_BUCKET: ((artifact-bucket))
- UPSTREAM_FORK: {{ repository.upstream_fork }}
- PIPELINE_PREFIX: ((pipeline-prefix))
- SANITIZED_GEODE_BRANCH: ((sanitized-geode-build-branch))
- SANITIZED_GEODE_FORK: ((sanitized-geode-fork))
- GCP_PROJECT: ((gcp-project))
- PUBLIC_PIPELINES: ((public-pipelines))
- CONCOURSE_URL: ((concourse-url))
- CONCOURSE_HOST: ((concourse-host))
- CONCOURSE_TEAM: ((concourse-team))
- run:
- path: geode-metrics-pipeline/ci/pipelines/metrics/deploy_metrics_pipeline.sh
- - put: concourse
- params:
- pipelines:
- - name: ((!pipeline-prefix))metrics
- team: ((!concourse-team))
- config_file: results/generated-pipeline.yml
- vars_files:
- - results/pipeline-vars.yml
-
-- name: set-reaper-pipeline
- serial: true
- public: ((!public-pipelines))
- plan:
- - get: geode-reaper-pipeline
- trigger: true
- - get: meta-mini-image
- trigger: true
- passed: [build-meta-mini-docker-image]
- - task: create-image-yml
- image: meta-mini-image
- config:
- platform: linux
- inputs:
- - name: geode-reaper-pipeline
- outputs:
- - name: results
- params:
- OUTPUT_DIRECTORY: results
- GEODE_BRANCH: ((geode-build-branch))
- GEODE_FORK: ((geode-fork))
- GEODE_REPO_NAME: ((geode-repo-name))
- REPOSITORY_PUBLIC: {{ repository.public }}
- UPSTREAM_FORK: {{ repository.upstream_fork }}
- PIPELINE_PREFIX: ((pipeline-prefix))
- SANITIZED_GEODE_BRANCH: ((sanitized-geode-build-branch))
- SANITIZED_GEODE_FORK: ((sanitized-geode-fork))
- GCP_PROJECT: ((gcp-project))
- PUBLIC_PIPELINES: ((public-pipelines))
- run:
- path: geode-reaper-pipeline/ci/pipelines/reaper/deploy_reaper_pipeline.sh
- - put: concourse
- params:
- pipelines:
- - name: ((!pipeline-prefix))reaper
- team: ((!concourse-team))
- config_file: results/generated-pipeline.yml
- vars_files:
- - results/pipeline-vars.yml
-
- name: build-meta-mini-docker-image
public: ((!public-pipelines))
serial: true