You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@geode.apache.org by mh...@apache.org on 2020/03/09 16:15:29 UTC
[geode] branch mass-test-run updated (52bde64 -> 533de13)
This is an automated email from the ASF dual-hosted git repository.
mhanson pushed a change to branch mass-test-run
in repository https://gitbox.apache.org/repos/asf/geode.git.
discard 52bde64 remove meta-* jobs that we won't use for mass-test-run
discard ac7ded2 Mass-test-run branch, no windows or publication jobs
add c8198f1 Revert "GEODE-6536: Added retry in borrowConnection/single hop (#4719)" (#4749)
add bf0012d Feature/geode 6807 1 (#4711)
add cceaa66 Revert fluent style API fix in ClientRegionFactory (#4748)
add 0af6264 GEODE-7808: standardize on use of HostAndPort to form client-side connections (#4743)
add d603b21 GEODE-7835: Correct doc typo of file name gfsecurity.properties (#4756)
add 3ad4f17 GEODE-7839: add gfsh oauth token support (#4736)
add bcd3655 GEODE-7819: Add debugging support to SerializableTemporaryFolder (#4740)
add a61f951 GEODE-7421: deployment should use PUT instead of POST in REST request
add 4b06a7a Revert "GEODE-7808: standardize on use of HostAndPort to form client-side connections (#4743)" (#4761)
add 2f6bf01 GEODE-7828: Convert backing store for Redis Hashes and Sets to single regions (#4745)
add 2a3e09f GEODE-7832: Remove Connection Semaphores (#4754)
add b3af99e GEODE-7752: Delete the extra GeodeClusterManagementServiceBuilder (#4760)
add f15bbc7 GEODE-7421: use RequestParam instead of RequestPart to be compatible with all rest client (#4766)
add a5c8e2b GEODE-7842: Improve concurrent region creation (#4767)
add 5c2b959 GEODE-7808: standardize on use of HostAndPort for creating connections (#4765)
add 838e070 GEODE-7840: Redis pub/sub does not handle binary messages correctly (#4762)
add b57456e GEODE-7841 Add benchmarking scripts for Geode-Redis module (#4763)
add a9a7d15 GEODE-7853: Remove unused field (#4777)
add f0982cd Revert "GEODE-7828: Convert backing store for Redis Hashes and Sets to single regions (#4745)" (#4780)
add d5adfbe GEODE-7849: change release urls from www.apache.org/dist/ to downloads.apache.org (#4770)
add 69ade72 GEODE-7851: configure pulse security using java code instead of using xml (#4772)
new ab40070 Mass-test-run branch, no windows or publication jobs
new 533de13 remove meta-* jobs that we won't use for mass-test-run
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (52bde64)
\
N -- N -- N refs/heads/mass-test-run (533de13)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
dev-tools/release/promote_rc.sh | 9 +-
docker/Dockerfile | 3 +-
.../apache/geode/metrics/CacheCommonTagsTest.java | 3 +-
.../rest/ClientClusterManagementSSLTest.java | 2 +-
.../ClientClusterManagementServiceDunitTest.java | 2 +-
.../rest/ClusterManagementServiceOnServerTest.java | 2 +-
.../rest/GeodeClientClusterManagementSSLTest.java | 2 +-
.../GeodeClientClusterManagementSecurityTest.java | 2 +-
.../geode/tools/pulse/PulseConnectivityTest.java | 14 -
.../PulseSecurityConfigDefaultProfileTest.java | 81 +++++
.../PulseSecurityConfigGemfireProfileTest.java | 101 ++++++
.../internal/LocatorLoadBalancingDUnitTest.java | 6 +-
.../geode/cache30/ClientMembershipDUnitTest.java | 3 -
.../internal/cache/DeltaPropagationDUnitTest.java | 64 ++--
.../AutoConnectionSourceImplJUnitTest.java | 28 +-
.../pooling/ConnectionManagerImplTest.java | 32 +-
.../pooling/ConnectionManagerJUnitTest.java | 6 +-
.../distributed/LauncherIntegrationTestCase.java | 3 +-
.../geode/distributed/LocatorIntegrationTest.java | 4 +-
.../tcpserver/TCPClientSSLIntegrationTest.java | 12 +-
.../internal/tcpserver/TCPServerSSLJUnitTest.java | 5 +-
.../CacheDistributionAdvisorConcurrentTest.java | 103 ++++++
.../net/ClientSocketFactoryIntegrationTest.java | 4 +-
.../internal/net/SSLSocketIntegrationTest.java | 13 +-
.../GemFireStatSamplerIntegrationTest.java | 3 +-
.../SimpleStatSamplerIntegrationTest.java | 3 +-
...SSLSocketParameterExtensionIntegrationTest.java | 11 +-
.../internal/DistributionLocatorConfigImpl.java | 8 +-
.../admin/internal/DistributionLocatorImpl.java | 3 +-
.../admin/internal/ManagedEntityConfigImpl.java | 3 +-
.../jmx/internal/MX4JServerSocketFactory.java | 10 +-
.../geode/cache/client/ClientRegionFactory.java | 2 +-
.../client/internal/AutoConnectionSourceImpl.java | 47 ++-
.../client/internal/ClientRegionFactoryImpl.java | 3 +-
.../cache/client/internal/ConnectionImpl.java | 7 +-
.../internal/ExplicitConnectionSourceImpl.java | 2 +-
.../cache/client/internal/OpExecutorImpl.java | 2 +-
.../geode/cache/client/internal/PoolImpl.java | 32 +-
.../client/internal/pooling/ConnectionManager.java | 5 +-
.../internal/pooling/ConnectionManagerImpl.java | 48 +--
.../java/org/apache/geode/distributed/Locator.java | 3 +-
.../apache/geode/distributed/LocatorLauncher.java | 4 +-
.../internal/ClusterOperationExecutors.java | 8 -
.../distributed/internal/DistributionAdvisor.java | 12 +-
.../internal/DistributionConfigImpl.java | 3 +-
.../distributed/internal/direct/DirectChannel.java | 200 +++---------
.../membership/InternalDistributedMember.java | 28 +-
.../org/apache/geode/internal/AbstractConfig.java | 2 +-
.../org/apache/geode/internal/DSFIDFactory.java | 2 +
.../apache/geode/internal/DistributionLocator.java | 6 +-
.../geode/internal/InternalDataSerializer.java | 5 +-
.../org/apache/geode/internal/SystemAdmin.java | 9 +-
.../admin/remote/DistributionLocatorId.java | 26 +-
.../internal/cache/CacheDistributionAdvisor.java | 98 ++++--
.../internal/cache/DistributedCacheOperation.java | 2 +-
.../geode/internal/cache/DistributedRegion.java | 7 +-
.../geode/internal/cache/GemFireCacheImpl.java | 8 +-
.../geode/internal/cache/PoolFactoryImpl.java | 79 +++--
.../apache/geode/internal/cache/properties.html | 34 --
.../internal/cache/tier/sockets/AcceptorImpl.java | 5 +-
.../cache/tier/sockets/CacheClientUpdater.java | 4 +-
.../tier/sockets/ProtobufServerConnection.java | 2 +-
.../internal/cache/wan/AbstractGatewaySender.java | 2 +-
.../internal/cache/xmlcache/CacheXmlGenerator.java | 4 +-
.../internal/net/SCAdvancedSocketCreator.java | 119 +++++++
.../geode/internal/net/SCClientSocketCreator.java | 20 +-
.../geode/internal/net/SCServerSocketCreator.java | 109 +++++++
.../apache/geode/internal/net/SocketCreator.java | 361 ++++++---------------
.../security/shiro/JMXShiroAuthenticator.java | 5 +-
.../geode/internal/statistics/HostStatSampler.java | 3 +-
.../internal/statistics/StatArchiveWriter.java | 3 +-
.../org/apache/geode/internal/tcp/Connection.java | 84 +----
.../org/apache/geode/internal/tcp/TCPConduit.java | 7 +-
.../ContextAwareSSLRMIClientSocketFactory.java | 5 +-
.../internal/JmxManagerLocatorRequest.java | 8 +-
.../geode/management/internal/ManagementAgent.java | 8 +-
.../internal/api/GeodeConnectionConfig.java | 4 +-
.../utils/ClusterConfigurationStatusRetriever.java | 11 +-
.../geode/management/internal/i18n/CliStrings.java | 2 +
.../sanctioned-geode-core-serializables.txt | 8 +-
.../client/internal/OpExecutorImplJUnitTest.java | 2 +-
.../geode/cache/client/internal/PoolImplTest.java | 4 +-
.../cache/CacheDistributionAdvisorTest.java | 176 ++++++++++
.../geode/internal/cache/CacheServerImplTest.java | 5 +
.../tier/sockets/CacheClientUpdaterJUnitTest.java | 6 +-
.../geode/internal/net/SocketCreatorJUnitTest.java | 8 +-
.../apache/geode/internal/tcp/TCPConduitTest.java | 3 +-
.../cache/PutAllClientServerDistributedTest.java | 15 +-
.../cache/tier/sockets/DurableClientTestBase.java | 3 +-
.../cache/tier/sockets/DurableClientTestCase.java | 11 +-
.../gfsh/command-pages/connect.html.md.erb | 19 +-
.../tools_modules/pulse/pulse-auth.html.md.erb | 2 +-
.../cache/client/internal/LocatorTestBase.java | 10 +-
.../geode/test/junit/rules/GfshCommandRule.java | 6 +
.../internal/cli/commands/ConnectCommand.java | 26 +-
.../cli/functions/RegionCreateFunction.java | 37 ++-
.../internal/cli/shell/JmxOperationInvoker.java | 7 +-
.../internal/cli/commands/ConnectCommandTest.java | 52 +++
.../functions/RegionCreateFunctionJUnitTest.java | 75 +++++
...SerializableTemporaryFolderIntegrationTest.java | 334 +++++++++++++++++++
.../serializable/SerializableTemporaryFolder.java | 148 ++++++++-
.../geode/test/junit/runners/TestRunner.java | 96 +++++-
.../SerializableTemporaryFolderTest.java | 12 +-
.../apache/geode/management/api/CommandType.java | 1 +
...tTemplateClusterManagementServiceTransport.java | 9 +-
.../configuration/AbstractConfiguration.java | 6 +
.../geode/management/configuration/Deployment.java | 6 +
.../internal/ClientClusterManagementService.java | 3 +-
.../ClientClusterManagementServiceTest.java | 1 +
.../internal/membership/gms/GMSUtilTest.java | 36 +-
.../gms/membership/GMSJoinLeaveJUnitTest.java | 12 +-
.../internal/membership/gms/GMSUtil.java | 14 +-
.../membership/gms/MemberDataBuilderImpl.java | 4 +-
.../membership/gms/MemberIdentifierImpl.java | 8 +-
.../membership/gms/fd/GMSHealthMonitor.java | 7 +-
.../membership/gms/locator/GMSLocator.java | 13 +-
.../gms/locator/MembershipLocatorImpl.java | 5 +-
.../membership/gms/membership/GMSJoinLeave.java | 9 +-
.../acceptance/CacheConnectionIntegrationTest.java | 4 +-
.../v1/acceptance/CacheOperationsJUnitTest.java | 4 +-
.../controllers/PulseControllerJUnitTest.java | 2 +-
.../internal/security/CustomSecurityConfig.java | 41 +++
.../internal/security/DefaultSecurityConfig.java | 105 ++++++
.../internal/security/GemfireSecurityConfig.java | 43 +++
...vc-dispatcher-servlet.xml => pulse-servlet.xml} | 0
.../src/main/webapp/WEB-INF/spring-security.xml | 108 ------
geode-pulse/src/main/webapp/WEB-INF/web.xml | 19 +-
geode-redis/README.md | 71 ++++
.../java/org/apache/geode/redis/PubSubTest.java | 28 ++
.../geode/redis/mocks/MockBinarySubscriber.java | 34 +-
.../geode/redis/internal/AbstractSubscription.java | 4 +-
.../geode/redis/internal/ChannelSubscription.java | 2 +-
.../geode/redis/internal/PatternSubscription.java | 2 +-
.../org/apache/geode/redis/internal/PubSub.java | 2 +-
.../apache/geode/redis/internal/PubSubImpl.java | 15 +-
.../apache/geode/redis/internal/Subscription.java | 4 +-
.../internal/executor/pubsub/PublishExecutor.java | 3 +-
geode-redis/src/performanceTest/aggregator.sh | 91 ++++++
geode-redis/src/performanceTest/benchmark.sh | 116 +++++++
geode-redis/src/performanceTest/shacompare.sh | 59 ++++
.../geode/redis/internal/PubSubImplJUnitTest.java | 2 +-
.../serialization/DataSerializableFixedID.java | 1 +
.../serialization/StaticSerialization.java | 1 +
.../tcpserver/TcpServerGossipVersionDUnitTest.java | 3 +-
.../internal/tcpserver/TcpServerJUnitTest.java | 30 +-
.../TcpServerProductVersionDUnitTest.java | 17 +-
.../internal/tcpserver/AdvancedSocketCreator.java | 61 ++++
...torImpl.java => AdvancedSocketCreatorImpl.java} | 148 +++------
...SocketCreator.java => ClientSocketCreator.java} | 44 +--
.../tcpserver/ClientSocketCreatorImpl.java | 54 +++
...ocketCreator.java => ClusterSocketCreator.java} | 50 ++-
.../internal/tcpserver/HostAndPort.java | 154 +++++++++
.../internal/tcpserver/LocatorAddress.java | 91 ------
.../tcpserver/ServerSocketCreatorImpl.java | 96 ++++++
.../distributed/internal/tcpserver/TcpClient.java | 64 ++--
.../distributed/internal/tcpserver/TcpServer.java | 6 +-
.../internal/tcpserver/TcpSocketCreator.java | 38 +--
.../internal/tcpserver/TcpSocketCreatorImpl.java | 191 ++---------
...ocatorAddressTest.java => HostAndPortTest.java} | 40 +--
.../WANHostNameVerificationDistributedTest.java | 31 +-
.../internal/locator/wan/LocatorDiscovery.java | 11 +-
.../locator/wan/LocatorMembershipListenerImpl.java | 6 +-
.../locator/wan/LocatorMembershipListenerTest.java | 4 +-
...SenderEventRemoteDispatcherIntegrationTest.java | 17 -
.../DeploymentManagementController.java | 11 +-
.../rest/security/RestSecurityConfiguration.java | 19 ++
.../src/main/webapp/WEB-INF/management-servlet.xml | 2 -
.../DeploymentManagementControllerTest.java | 50 +++
.../commands/ConnectCommandWithSecurityTest.java | 22 ++
.../support/LoginHandlerInterceptor.java | 15 +-
170 files changed, 3466 insertions(+), 1764 deletions(-)
create mode 100644 geode-assembly/src/integrationTest/java/org/apache/geode/tools/pulse/PulseSecurityConfigDefaultProfileTest.java
create mode 100644 geode-assembly/src/integrationTest/java/org/apache/geode/tools/pulse/PulseSecurityConfigGemfireProfileTest.java
create mode 100644 geode-core/src/integrationTest/java/org/apache/geode/internal/cache/CacheDistributionAdvisorConcurrentTest.java
create mode 100644 geode-core/src/main/java/org/apache/geode/internal/net/SCAdvancedSocketCreator.java
rename geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/membership/HostAddress.java => geode-core/src/main/java/org/apache/geode/internal/net/SCClientSocketCreator.java (58%)
create mode 100644 geode-core/src/main/java/org/apache/geode/internal/net/SCServerSocketCreator.java
create mode 100644 geode-core/src/test/java/org/apache/geode/internal/cache/CacheDistributionAdvisorTest.java
create mode 100644 geode-gfsh/src/test/java/org/apache/geode/management/internal/cli/functions/RegionCreateFunctionJUnitTest.java
create mode 100644 geode-junit/src/integrationTest/java/org/apache/geode/test/junit/rules/serializable/SerializableTemporaryFolderIntegrationTest.java
create mode 100644 geode-pulse/src/main/java/org/apache/geode/tools/pulse/internal/security/CustomSecurityConfig.java
create mode 100644 geode-pulse/src/main/java/org/apache/geode/tools/pulse/internal/security/DefaultSecurityConfig.java
create mode 100644 geode-pulse/src/main/java/org/apache/geode/tools/pulse/internal/security/GemfireSecurityConfig.java
rename geode-pulse/src/main/webapp/WEB-INF/{mvc-dispatcher-servlet.xml => pulse-servlet.xml} (100%)
delete mode 100644 geode-pulse/src/main/webapp/WEB-INF/spring-security.xml
create mode 100644 geode-redis/README.md
rename geode-core/src/main/java/org/apache/geode/management/internal/builder/GeodeClusterManagementServiceBuilder.java => geode-redis/src/integrationTest/java/org/apache/geode/redis/mocks/MockBinarySubscriber.java (52%)
create mode 100644 geode-redis/src/performanceTest/aggregator.sh
create mode 100644 geode-redis/src/performanceTest/benchmark.sh
create mode 100644 geode-redis/src/performanceTest/shacompare.sh
create mode 100644 geode-tcp-server/src/main/java/org/apache/geode/distributed/internal/tcpserver/AdvancedSocketCreator.java
copy geode-tcp-server/src/main/java/org/apache/geode/distributed/internal/tcpserver/{TcpSocketCreatorImpl.java => AdvancedSocketCreatorImpl.java} (62%)
copy geode-tcp-server/src/main/java/org/apache/geode/distributed/internal/tcpserver/{TcpSocketCreator.java => ClientSocketCreator.java} (54%)
create mode 100644 geode-tcp-server/src/main/java/org/apache/geode/distributed/internal/tcpserver/ClientSocketCreatorImpl.java
copy geode-tcp-server/src/main/java/org/apache/geode/distributed/internal/tcpserver/{TcpSocketCreator.java => ClusterSocketCreator.java} (50%)
create mode 100644 geode-tcp-server/src/main/java/org/apache/geode/distributed/internal/tcpserver/HostAndPort.java
delete mode 100644 geode-tcp-server/src/main/java/org/apache/geode/distributed/internal/tcpserver/LocatorAddress.java
create mode 100644 geode-tcp-server/src/main/java/org/apache/geode/distributed/internal/tcpserver/ServerSocketCreatorImpl.java
rename geode-tcp-server/src/test/java/org/apache/geode/distributed/internal/tcpserver/{LocatorAddressTest.java => HostAndPortTest.java} (56%)
create mode 100644 geode-web-management/src/test/java/org/apache/geode/management/internal/rest/controllers/DeploymentManagementControllerTest.java
[geode] 01/02: Mass-test-run branch, no windows or publication jobs
Posted by mh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mhanson pushed a commit to branch mass-test-run
in repository https://gitbox.apache.org/repos/asf/geode.git
commit ab40070ac0e0de1e1422bf2a165b8caaee4914d3
Author: Robert Houghton <rh...@pivotal.io>
AuthorDate: Mon Jan 27 15:20:46 2020 -0800
Mass-test-run branch, no windows or publication jobs
Co-authored-by: Robert Houghton <rh...@pivotal.io>
Co-authored-by: Mark Hanson <mh...@pivotal.io>
---
ci/pipelines/geode-build/jinja.template.yml | 310 +---------------------------
ci/pipelines/shared/jinja.variables.yml | 77 +------
2 files changed, 9 insertions(+), 378 deletions(-)
diff --git a/ci/pipelines/geode-build/jinja.template.yml b/ci/pipelines/geode-build/jinja.template.yml
index da6e399..f9dcbda 100644
--- a/ci/pipelines/geode-build/jinja.template.yml
+++ b/ci/pipelines/geode-build/jinja.template.yml
@@ -92,66 +92,6 @@ GRADLE_GLOBAL_ARGS: ((gradle-global-args))
{%- endfor -%}
{% endmacro %}
-groups:
-- name: main
- jobs:
- - {{ build_test.name }}
- {{- all_gating_jobs() | indent(2) }}
- - UpdatePassingTokens
- {%- if repository.upstream_fork != "apache" or repository.branch == "develop" %}
- - PublishArtifacts
- {%- endif %}
- {%- for flavor in (benchmarks.flavors) %}
- - Benchmark{{flavor.title}}
- {%- endfor %}
-- name: complete
- jobs:
- - {{ build_test.name }}
- {%- for test in (tests) if not test.name=="StressNew" -%}
- {%- for java_test_version in (java_test_versions) %}
- - {{test.name}}Test{{java_test_version.name}}
- {%- endfor -%}
- {%- endfor %}
- - UpdatePassingTokens
- {%- if repository.upstream_fork != "apache" or repository.branch == "develop" %}
- - PublishArtifacts
- {%- endif %}
- {%- for flavor in (benchmarks.flavors) %}
- - Benchmark{{flavor.title}}
- {%- endfor %}
-- name: linux
- jobs:
- - {{ build_test.name }}
- {%- for test in (tests) if test.PLATFORM=="linux" and not test.name=="StressNew" -%}
- {% for java_test_version in (java_test_versions) %}
- - {{test.name}}Test{{java_test_version.name}}
- {%- endfor -%}
- {%- endfor %}
- {%- for flavor in (benchmarks.flavors) %}
- - Benchmark{{flavor.title}}
- {%- endfor %}
-- name: windows
- jobs:
- - {{ build_test.name }}
- {%- for test in (tests) if test.PLATFORM=="windows" -%}
- {% for java_test_version in (java_test_versions) %}
- - {{test.name}}Test{{java_test_version.name}}
- {%- endfor -%}
- {%- endfor %}
-{%- for java_test_version in (java_test_versions) %}
-- name: {{java_test_version.name}}
- jobs:
- - {{ build_test.name }}
- {%- for test in (tests) if not test.name=="StressNew" %}
- - {{test.name}}Test{{java_test_version.name}}
- {%- endfor -%}
-{%- endfor %}
-- name: Semver Management
- jobs:
- {%- for semverPiece in ['major', 'minor', 'patch'] %}
- - Bump{{ semverPiece.title() }}
- {%- endfor %}
-
resources:
- name: concourse-metadata-resource
type: concourse-metadata-resource
@@ -173,12 +113,7 @@ resources:
paths:
- ci/*
{{ github_access() | indent(4) }}
-- name: geode-benchmarks
- type: git
- source:
- branch: {{benchmarks.benchmark_branch}}
- depth: 1
- uri: https://github.com/apache/geode-benchmarks.git
+
- name: geode-build-version
type: semver
source:
@@ -187,12 +122,7 @@ resources:
initial_version: {{ metadata.initial_version }}
json_key: ((!concourse-gcp-key))
key: semvers/((pipeline-prefix))((geode-build-branch))/version
-- name: geode-passing-tokens
- type: gcs-resource
- source:
- bucket: ((artifact-bucket))
- json_key: ((concourse-gcp-key))
- versioned_file: semvers/((pipeline-prefix))((geode-build-branch))/passing-build-tokens.json
+
- name: alpine-tools-image
type: docker-image
source:
@@ -201,12 +131,6 @@ resources:
repository: gcr.io/((gcp-project))/((pipeline-prefix))alpine-tools
tag: latest
-- name: windows-builder-image-family
- type: gci
- source:
- key: ((concourse-gcp-key))
- family_project: ((gcp-project))
- family: ((pipeline-prefix))windows-geode-builder
- name: linux-builder-image-family
type: gci
@@ -233,16 +157,8 @@ resource_types:
source:
repository: frodenas/gcs-resource
jobs:
-{% for semverPiece in ['major', 'minor', 'patch'] -%}
-- name: Bump{{ semverPiece.title() }}
- serial: true
- plan:
- - get: geode-build-version
- params: { bump: {{ semverPiece }} }
- - put: geode-build-version
- params:
- file: geode-build-version/number
-{% endfor %}
+
+
- name: {{build_test.name}}
public: true
max_in_flight: {{build_test.MAX_IN_FLIGHT}}
@@ -388,224 +304,6 @@ jobs:
- name: instance-data
timeout: 1h
-- name: UpdatePassingTokens
- public: true
- serial: true
- plan:
- - aggregate:
- - get: alpine-tools-image
- - get: geode
- passed: &update-token-passed-anchor
-{%- if repository.upstream_fork != "apache" or repository.branch == "develop" %}
- - PublishArtifacts
-{% else %}
- {% for flavor in (benchmarks.flavors) %}
- - Benchmark{{flavor.title}}
- {% endfor %}
-{% endif %}
- trigger: true
- - get: geode-build-version
- trigger: true
- passed: *update-token-passed-anchor
- - task: couple-sha-and-build-id
- image: alpine-tools-image
- config:
- platform: linux
- inputs:
- - name: geode
- - name: geode-build-version
- outputs:
- - name: geode-passing-tokens
- run:
- path: bash
- args:
- - -cx
- - |
- pushd geode
- GEODE_SHA=$(git rev-parse HEAD)
- popd
- GEODE_SEMVER=$(cat geode-build-version/number)
-
- GS_PATH=gs://((artifact-bucket))/semvers/((pipeline-prefix))((geode-build-branch))/passing-build-tokens.json
- CURRENT_PASSING_SHA=$(gsutil cat ${GS_PATH} | jq -r .ref)
- set -e
- # Check that the incoming GEODE_SHA is a descendent of the currently stored value.
- # Keeps us from winding back the repository in the case of an out-of-order pipeline pass
- if [ -z "${CURRENT_PASSING_SHA}" ] || (cd geode; git merge-base --is-ancestor ${CURRENT_PASSING_SHA} ${GEODE_SHA}); then
- cat > geode-passing-tokens/passing-build-tokens.json <<JSON
- {
- "ref": "${GEODE_SHA}",
- "semver": "${GEODE_SEMVER}"
- }
- JSON
- fi
- - aggregate:
- - put: geode-passing-tokens
- params:
- file: geode-passing-tokens/passing-build-tokens.json
-
-{% for run_var in (benchmarks.flavors) %}
-- name: Benchmark{{ run_var.title }}
- public: true
- max_in_flight: 3
- plan:
- - get: geode-ci
- passed:
- {{ all_gating_jobs() | indent(4) }}
- - get: alpine-tools-image
- - aggregate:
- - get: geode
- passed:
- {{ all_gating_jobs() | indent(6) }}
- trigger: true
- - get: geode-benchmarks
- - get: geode-build-version
- trigger: true
- passed:
- {{ all_gating_jobs() | indent(6) }}
- - put: concourse-metadata-resource
- - do:
- - task: run_benchmarks{{ run_var.title }}
- image: alpine-tools-image
- config:
- platform: linux
- params:
- AWS_ACCESS_KEY_ID: ((benchmarks-access-key-id))
- AWS_SECRET_ACCESS_KEY: ((benchmarks-secret-access-key))
- AWS_DEFAULT_REGION: us-west-2
- AWS_REGION: us-west-2
- ARTIFACT_BUCKET: ((artifact-bucket))
- BENCHMARKS_BRANCH: {{benchmarks.benchmark_branch}}
- BASELINE_BRANCH: {{benchmarks.baseline_branch}}
- BASELINE_VERSION: {{benchmarks.baseline_version}}
- FLAGS: {{ run_var.flag }}
- TAG_POSTFIX: {{ run_var.title }}
- TEST_OPTIONS: {{ run_var.options }}
- run:
- path: geode-ci/ci/scripts/run_benchmarks.sh
- inputs:
- - name: geode
- - name: geode-ci
- - name: geode-benchmarks
- - name: concourse-metadata-resource
- outputs:
- - name: results
- timeout: 8h
- ensure:
- do:
- - task: cleanup_benchmarks
- image: alpine-tools-image
- config:
- platform: linux
- params:
- AWS_ACCESS_KEY_ID: ((benchmarks-access-key-id))
- AWS_SECRET_ACCESS_KEY: ((benchmarks-secret-access-key))
- AWS_DEFAULT_REGION: us-west-2
- AWS_REGION: us-west-2
- ARTIFACT_BUCKET: ((artifact-bucket))
- BASELINE_BRANCH: {{benchmarks.baseline_branch}}
- BASELINE_VERSION: {{benchmarks.baseline_version}}
- FLAGS: {{ run_var.flag }}
- TAG_POSTFIX: {{ run_var.title }}
- TEST_OPTIONS: {{ run_var.options }}
- run:
- path: geode-ci/ci/scripts/cleanup_benchmarks.sh
- inputs:
- - name: geode
- - name: geode-ci
- - name: geode-benchmarks
- - name: concourse-metadata-resource
- - name: results
-{% endfor %}
-{% if repository.upstream_fork != "apache" or repository.branch == "develop" %}
-- name: PublishArtifacts
- public: true
- plan:
- - aggregate:
- - get: geode-ci
- passed: &publish-passed-inputs
- {% for flavor in (benchmarks.flavors) %}
- - Benchmark{{flavor.title}}
- {% endfor %}
- - get: alpine-tools-image
- - get: geode
- passed: *publish-passed-inputs
- trigger: true
- - get: linux-builder-image-family
- - aggregate:
- - get: geode-build-version
- trigger: true
- passed: *publish-passed-inputs
- - put: concourse-metadata-resource
- {{ init_retry()|indent(4) }}
- - task: create_instance
- image: alpine-tools-image
- config:
- platform: linux
- params:
- {{ common_instance_params(publish_artifacts) | indent(8) }}
- GEODE_BRANCH: {{repository.branch}}
- GEODE_FORK: {{repository.fork}}
- JAVA_BUILD_VERSION: {{ java_build_version.version }}
- IMAGE_FAMILY_NAME: ((pipeline-prefix))linux-geode-builder
- run:
- path: geode-ci/ci/scripts/create_instance.sh
- inputs:
- - name: concourse-metadata-resource
- - name: geode-ci
- - name: geode
- - name: attempts-log
- path: old
- outputs:
- - name: instance-data
- - name: attempts-log
- path: new
- timeout: 20m
- attempts: 5
- - task: rsync_code_up
- image: alpine-tools-image
- config:
- platform: linux
- run:
- path: geode-ci/ci/scripts/rsync_code_up.sh
- inputs:
- - name: geode-ci
- - name: geode
- - name: instance-data
- timeout: 5m
- - task: publish
- image: alpine-tools-image
- config:
- platform: linux
- params:
- MAINTENANCE_VERSION: ((geode-build-branch))
- ARTIFACT_BUCKET: ((artifact-bucket))
- SERVICE_ACCOUNT: ((!concourse-gcp-account))
- JAVA_BUILD_VERSION: {{ java_build_version.version }}
- MAVEN_SNAPSHOT_BUCKET: ((maven-snapshot-bucket))
- GRADLE_GLOBAL_ARGS: ((gradle-global-args))
- run:
- path: geode-ci/ci/scripts/execute_publish.sh
- inputs:
- - name: instance-data
- - name: geode
- - name: geode-ci
- - name: geode-build-version
- ensure:
- do:
- - task: delete_instance
- image: alpine-tools-image
- config:
- platform: linux
- run:
- path: geode-ci/ci/scripts/delete_instance.sh
- inputs:
- - name: geode-ci
- - name: instance-data
- timeout: 1h
-
-{% endif %}
-
{%- for test in tests if not test.name=="StressNew" %}
{%- set parameters = {} %}
{%- do deep_merge(parameters, test) %}
diff --git a/ci/pipelines/shared/jinja.variables.yml b/ci/pipelines/shared/jinja.variables.yml
index a82cdca..3931254 100644
--- a/ci/pipelines/shared/jinja.variables.yml
+++ b/ci/pipelines/shared/jinja.variables.yml
@@ -84,7 +84,7 @@ tests:
DUNIT_PARALLEL_FORKS: '0'
EXECUTE_TEST_TIMEOUT: 30m
GRADLE_TASK: test
- MAX_IN_FLIGHT: 1
+ MAX_IN_FLIGHT: 5
PARALLEL_DUNIT: 'false'
PARALLEL_GRADLE: 'true'
PLATFORM: linux
@@ -96,7 +96,7 @@ tests:
DUNIT_PARALLEL_FORKS: '0'
EXECUTE_TEST_TIMEOUT: 1h30m
GRADLE_TASK: acceptanceTest
- MAX_IN_FLIGHT: 2
+ MAX_IN_FLIGHT: 5
PARALLEL_DUNIT: 'false'
PARALLEL_GRADLE: 'false'
PLATFORM: linux
@@ -108,7 +108,7 @@ tests:
DUNIT_PARALLEL_FORKS: '24'
EXECUTE_TEST_TIMEOUT: 3h00m
GRADLE_TASK: distributedTest
- MAX_IN_FLIGHT: 3
+ MAX_IN_FLIGHT: 5
PARALLEL_DUNIT: 'true'
PLATFORM: linux
RAM: '250'
@@ -119,7 +119,7 @@ tests:
DUNIT_PARALLEL_FORKS: '48'
EXECUTE_TEST_TIMEOUT: 40m
GRADLE_TASK: integrationTest
- MAX_IN_FLIGHT: 1
+ MAX_IN_FLIGHT: 5
PARALLEL_DUNIT: 'true'
PLATFORM: linux
RAM: '90'
@@ -130,75 +130,8 @@ tests:
DUNIT_PARALLEL_FORKS: '48'
EXECUTE_TEST_TIMEOUT: 1h
GRADLE_TASK: upgradeTest
- MAX_IN_FLIGHT: 2
+ MAX_IN_FLIGHT: 5
PARALLEL_DUNIT: 'true'
PLATFORM: linux
RAM: '210'
name: Upgrade
-- ARTIFACT_SLUG: stressnewtestfiles
- CALL_STACK_TIMEOUT: '20700'
- CPUS: '96'
- DUNIT_PARALLEL_FORKS: '24'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: repeatTest
- PARALLEL_DUNIT: 'true'
- PARALLEL_GRADLE: 'false'
- PLATFORM: linux
- RAM: '280'
- name: StressNew
-- ARTIFACT_SLUG: windows-acceptancetestfiles
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: :geode-assembly:acceptanceTest
- MAX_IN_FLIGHT: 3
- PARALLEL_DUNIT: 'false'
- PARALLEL_GRADLE: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsAcceptance
-- ARTIFACT_SLUG: windows-gfshdistributedtest
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: distributedTest
- GRADLE_TASK_OPTIONS: -PtestCategory=org.apache.geode.test.junit.categories.GfshTest
- MAX_IN_FLIGHT: 5
- PARALLEL_DUNIT: 'false'
- PARALLEL_GRADLE: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsGfshDistributed
-- ARTIFACT_SLUG: windows-integrationtestfiles
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: integrationTest
- GRADLE_TASK_OPTIONS: -x geode-core:integrationTest
- MAX_IN_FLIGHT: 2
- PARALLEL_DUNIT: 'false'
- PARALLEL_GRADLE: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsIntegration
-- ARTIFACT_SLUG: windows-coreintegrationtestfiles
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: geode-core:integrationTest
- MAX_IN_FLIGHT: 6
- PARALLEL_DUNIT: 'false'
- PARALLEL_GRADLE: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsCoreIntegration
-- ARTIFACT_SLUG: windows-unittestfiles
- CPUS: '16'
- DUNIT_PARALLEL_FORKS: '0'
- EXECUTE_TEST_TIMEOUT: 6h
- GRADLE_TASK: test
- MAX_IN_FLIGHT: 1
- PARALLEL_DUNIT: 'false'
- PLATFORM: windows
- RAM: '64'
- name: WindowsUnit
[geode] 02/02: remove meta-* jobs that we won't use for
mass-test-run
Posted by mh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mhanson pushed a commit to branch mass-test-run
in repository https://gitbox.apache.org/repos/asf/geode.git
commit 533de1308cf408aa4e8704b9649cdaa7ba37b189
Author: Mark Hanson <mh...@pivotal.io>
AuthorDate: Mon Jan 27 15:27:19 2020 -0800
remove meta-* jobs that we won't use for mass-test-run
---
ci/pipelines/meta/jinja.template.yml | 206 +----------------------------------
1 file changed, 1 insertion(+), 205 deletions(-)
diff --git a/ci/pipelines/meta/jinja.template.yml b/ci/pipelines/meta/jinja.template.yml
index 792254f..a67e377 100644
--- a/ci/pipelines/meta/jinja.template.yml
+++ b/ci/pipelines/meta/jinja.template.yml
@@ -51,13 +51,7 @@ resources:
branch: ((!geode-build-branch))
paths:
- ci/pipelines/images/*
-- name: geode-reaper-pipeline
- type: git
- source:
- {{ github_access() | indent(4) }}
- branch: ((!geode-build-branch))
- paths:
- - ci/pipelines/reaper/*
+
- name: meta-mini-dockerfile
type: git
source:
@@ -71,35 +65,6 @@ resources:
username: ((!docker-username))
password: ((!docker-password))
repository: gcr.io/((!gcp-project))/((!sanitized-geode-fork))-((!sanitized-geode-build-branch))-meta-img
-{% if repository.fork == repository.upstream_fork %}
-- name: geode-examples-pipeline
- type: git
- source:
- {{ github_access() | indent(4) }}
- branch: ((!geode-build-branch))
- paths:
- - ci/pipelines/examples/*
- - ci/pipelines/shared/*
- - ci/pipelines/render.py
- - ci/pipelines/meta/meta.properties
-- name: geode-pr-pipeline
- type: git
- source:
- {{ github_access() | indent(4) }}
- branch: ((!geode-build-branch))
- paths:
- - ci/pipelines/pull-request/*
- - ci/pipelines/shared/*
- - ci/pipelines/render.py
- - ci/pipelines/meta/meta.properties
-{% endif %}
-- name: geode-metrics-pipeline
- type: git
- source:
- {{ github_access() | indent(4) }}
- branch: ((!geode-build-branch))
- paths:
- - ci/pipelines/metrics/*
jobs:
- name: set-images-pipeline
@@ -187,175 +152,6 @@ jobs:
- results/pipeline-vars.yml
-{% if repository.fork == repository.upstream_fork %}
-- name: set-examples-pipeline
- serial: true
- public: ((!public-pipelines))
- plan:
- - get: geode-examples-pipeline
- trigger: true
- - get: meta-mini-image
- trigger: true
- passed: [build-meta-mini-docker-image]
- - task: deploy-build
- image: meta-mini-image
- config:
- platform: linux
- inputs:
- - name: geode-examples-pipeline
- outputs:
- - name: results
- params:
- OUTPUT_DIRECTORY: results
- GEODE_BRANCH: ((geode-build-branch))
- GEODE_FORK: ((geode-fork))
- GEODE_REPO_NAME: ((geode-repo-name))
- REPOSITORY_PUBLIC: {{ repository.public }}
- ARTIFACT_BUCKET: ((artifact-bucket))
- UPSTREAM_FORK: {{ repository.upstream_fork }}
- PIPELINE_PREFIX: ((pipeline-prefix))
- SANITIZED_GEODE_BRANCH: ((sanitized-geode-build-branch))
- SANITIZED_GEODE_FORK: ((sanitized-geode-fork))
- GCP_PROJECT: ((gcp-project))
- PUBLIC_PIPELINES: ((public-pipelines))
- SEMVER_PRERELEASE_TOKEN: ((semver-prerelease-token))
- run:
- path: geode-examples-pipeline/ci/pipelines/examples/deploy_pipeline.sh
- - put: concourse
- params:
- pipelines:
- - name: ((!pipeline-prefix))examples
- team: ((!concourse-team))
- config_file: results/generated-pipeline.yml
- vars_files:
- - results/pipeline-vars.yml
-
-- name: set-pr-pipeline
- serial: true
- public: ((!public-pipelines))
- plan:
- - get: geode-pr-pipeline
- trigger: true
- - get: meta-mini-image
- trigger: true
- passed: [build-meta-mini-docker-image]
- - task: deploy-pr
- image: meta-mini-image
- config:
- platform: linux
- inputs:
- - name: geode-pr-pipeline
- outputs:
- - name: results
- params:
- OUTPUT_DIRECTORY: results
- GEODE_BRANCH: ((!geode-build-branch))
- GEODE_FORK: ((!geode-fork))
- GEODE_REPO_NAME: ((geode-repo-name))
- REPOSITORY_PUBLIC: {{ repository.public }}
- ARTIFACT_BUCKET: ((artifact-bucket))
- UPSTREAM_FORK: {{ repository.upstream_fork }}
- PIPELINE_PREFIX: ((pipeline-prefix))
- GCP_PROJECT: ((gcp-project))
- PUBLIC_PIPELINES: ((public-pipelines))
- GRADLE_GLOBAL_ARGS: ((gradle-global-args))
- run:
- path: geode-pr-pipeline/ci/pipelines/pull-request/deploy_pr_pipeline.sh
- - put: concourse
- params:
- pipelines:
- - name: ((!pipeline-prefix))pr
- team: ((!concourse-team))
- config_file: results/generated-pipeline.yml
- vars_files:
- - results/pipeline-vars.yml
-
-{% endif %}
-
-- name: set-metrics-pipeline
- serial: true
- public: ((!public-pipelines))
- plan:
- - get: geode-metrics-pipeline
- trigger: true
- - get: meta-mini-image
- trigger: true
- passed: [build-meta-mini-docker-image]
- - task: create-metrics-pipeline-yml
- image: meta-mini-image
- config:
- platform: linux
- inputs:
- - name: geode-metrics-pipeline
- outputs:
- - name: results
- params:
- OUTPUT_DIRECTORY: results
- GEODE_BRANCH: ((geode-build-branch))
- GEODE_FORK: ((geode-fork))
- GEODE_REPO_NAME: ((geode-repo-name))
- REPOSITORY_PUBLIC: {{ repository.public }}
- ARTIFACT_BUCKET: ((artifact-bucket))
- UPSTREAM_FORK: {{ repository.upstream_fork }}
- PIPELINE_PREFIX: ((pipeline-prefix))
- SANITIZED_GEODE_BRANCH: ((sanitized-geode-build-branch))
- SANITIZED_GEODE_FORK: ((sanitized-geode-fork))
- GCP_PROJECT: ((gcp-project))
- PUBLIC_PIPELINES: ((public-pipelines))
- CONCOURSE_URL: ((concourse-url))
- CONCOURSE_HOST: ((concourse-host))
- CONCOURSE_TEAM: ((concourse-team))
- run:
- path: geode-metrics-pipeline/ci/pipelines/metrics/deploy_metrics_pipeline.sh
- - put: concourse
- params:
- pipelines:
- - name: ((!pipeline-prefix))metrics
- team: ((!concourse-team))
- config_file: results/generated-pipeline.yml
- vars_files:
- - results/pipeline-vars.yml
-
-- name: set-reaper-pipeline
- serial: true
- public: ((!public-pipelines))
- plan:
- - get: geode-reaper-pipeline
- trigger: true
- - get: meta-mini-image
- trigger: true
- passed: [build-meta-mini-docker-image]
- - task: create-image-yml
- image: meta-mini-image
- config:
- platform: linux
- inputs:
- - name: geode-reaper-pipeline
- outputs:
- - name: results
- params:
- OUTPUT_DIRECTORY: results
- GEODE_BRANCH: ((geode-build-branch))
- GEODE_FORK: ((geode-fork))
- GEODE_REPO_NAME: ((geode-repo-name))
- REPOSITORY_PUBLIC: {{ repository.public }}
- UPSTREAM_FORK: {{ repository.upstream_fork }}
- PIPELINE_PREFIX: ((pipeline-prefix))
- SANITIZED_GEODE_BRANCH: ((sanitized-geode-build-branch))
- SANITIZED_GEODE_FORK: ((sanitized-geode-fork))
- GCP_PROJECT: ((gcp-project))
- PUBLIC_PIPELINES: ((public-pipelines))
- run:
- path: geode-reaper-pipeline/ci/pipelines/reaper/deploy_reaper_pipeline.sh
- - put: concourse
- params:
- pipelines:
- - name: ((!pipeline-prefix))reaper
- team: ((!concourse-team))
- config_file: results/generated-pipeline.yml
- vars_files:
- - results/pipeline-vars.yml
-
- name: build-meta-mini-docker-image
public: ((!public-pipelines))
serial: true