You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@geode.apache.org by zh...@apache.org on 2018/03/21 22:32:09 UTC
[geode] branch feature/GEODE-4647 updated (a8d035e -> 02be598)
This is an automated email from the ASF dual-hosted git repository.
zhouxj pushed a change to branch feature/GEODE-4647
in repository https://gitbox.apache.org/repos/asf/geode.git.
omit a8d035e GEODE-4647: spotlessApply
omit d09492c GEODE-4647: recovery will increase secondary queue stats
omit cf104e2 GEODE-4647: add a stats eventSecondaryQueueSizeId to track events in secondary gateway sender queue
add 42ad145 GEODE-4846: fixup javadocs on ExecutorServiceRule
add f8bab2a GEODE-4845: make FieldSerializationUtils package-private
add ae25c7b GEODE-4789: upgrade system-rules from 1.16.1 to 1.17.1
add 71d1d8a GEODE-4789: add VM count constructor to ClusterStartupRule
add 6b56b5f GEODE-4789: fixup tests that use more than 4 VMs with ClusterStarterRule
add 8863386 GEODE-4789: fix static usage of DistributedRestoreSystemProperties
add 1298f76 GEODE-4821: rename RegisterInterestIntegrationTest as RegisterInterestDistributedTest
add 62f83e3 GEODE-4820: use RestoreSystemProperties instead of DistributedRestoreSystemProperties
add 6e2a754 GEODE-4782: Add category annotation for Gfsh tests (#1630)
add 7402122 GEODE-4783: Add categor annotation for Management tests (#1629)
add ad77bde GEODE-4384/3875: refactor create/destroy jndi binding commands to use CacheConfig objects (#1627)
add 6073a15 GEODE-4875: make a public ClusterConfigurationService interface
add bf079a1 GEODE-4880: Ignore Pulse UI acceptance tests that are flaky (#1635)
add c74a83f GEODE-4497: fix Gfsh text wrapping error
add 07a7b05 GEODE-4883: ServerLauncherDUnitTest fails with BindException
add 794af72 GEODE-4884: Remove Thread.sleep from ServerLauncherDUnitTest
add 0fe3839 GEODE-4885: Disabling DistributedRestoreSystemPropertiesTest until it can be fixed
add 694ee1d GEODE-4857: add interface methods to ClusterConfigurationService (#1637)
add 5018feb Commit then review: update documentation to be for version 1.6.0
add ce0c965 GEODE-4878: Remove Thread.sleep from ClusterConfigLocatorRestartDUnitTest (#1633)
add 8444bd0 [GEODE-4891] Fixes for concourse API changes.
add b4ad610 GEODE-4833: JdbcWriter and JdbcAsyncWriter may fail to write null fields to database (#1636)
add 752bc23 GEODE-4815: Forced locator members to be LOCATOR_DM_TYPES
add c6dee63 GEODE-4882: Add FlakyTest category to AnalyzeConnectorsSerializablesJUnitTest
add d8e29f0 GEODE-4869: Move all the Configuration objects to a public package (#1638)
add 4b5d42d GEODE-4888 Document security-udp-dhalgo property (#1642)
add bafab6b GEODE-4771: Defaults in ConfigurePDXCommand
add 54b18ed GEODE-4771: Changes requested by reviewers.
add 3fa3074 GEODE-4771: Internal API was renamed
add 1564f45 GEODE-4896: Delete com.gemstone.gemfire duplicate of FunctionExceptionTest
add 5bfccee GEODE-4832: Add FlakyTest category to CacheWriterGetOldValueIntegrationTest
add 8f36acf GEODE-4832: Cleanup CacheWriterGetOldValueIntegrationTest
add 05feafb GEODE-4864 Document the gfsh create jndi-binding command (#1644)
add 8da6377 GEODE-4869: Update sanctioned-geode-core-serializables.txt since moving configuration classes
add 68f4ca4 GEODE-4898: Fix field names in DestroyJndiBindingCommand (#1648)
add dca52c8 GEODE-1279: Rename several Bug*Test tests to have descriptive names (#1645)
add 888c7ab GEODE-4885: Change ClusterStartupRule to launch DUnit during setUp
add f39342d GEODE-4885: Fix ClientCQAuthDUnitTest by invoking getVM during setUp
add 461a53d GEODE-4885: Fix JMXMBeanDUnitTest by adding DistributedTestRule
add 2480ed8 GEODE-4832: use TemporaryFolder in CacheWriterGetOldValueIntegrationTest (#1652)
add 220eff1 GEODE-4885: Reenable DistributedRestoreSystemPropertiesTest
add a6092e9 Feature/geode 4810: Move rebalance model classes to own package (#1623)
add ccfa6bb GEODE-4901: increase the wait time for the newly created queue to be ready. (#1651)
add ee328b9 GEODE-4873: Remove link to test code from main code. (#1641)
add e1dc682 GEODE-4907: Fix analyze serializables for moved enum RefusalReason (#1655)
add c689dfd GEODE-2999, GEODE-4836: Add PutIfAbsent to the Protobuf protocol.
add 8e8ec93 GEODE-4451: Changed sender startup to retry when a remote security exception occurs
new 02be598 GEODE-4647: add a stats eventSecondaryQueueSizeId to track events in secondary gateway sender queue
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (a8d035e)
\
N -- N -- N refs/heads/feature/GEODE-4647 (02be598)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
ci/bin/concourse_job_performance.py | 2 +-
.../tools/pulse/ui/PulseAcceptanceTestBase.java | 3 +
geode-book/config.yml | 6 +-
.../source/subnavs/geode-subnav.erb | 3 +
geode-book/redirects.rb | 4 +-
.../jdbc/internal/AbstractJdbcCallback.java | 4 +-
.../connectors/jdbc/internal/ColumnValue.java | 13 +-
.../geode/connectors/jdbc/internal/SqlHandler.java | 31 +-
.../connectors/jdbc/internal/TableMetaData.java | 54 ++
...olumnManager.java => TableMetaDataManager.java} | 52 +-
.../jdbc/internal/TableMetaDataView.java | 8 +-
.../AnalyzeConnectorsSerializablesJUnitTest.java | 5 +-
.../jdbc/JdbcAsyncWriterIntegrationTest.java | 5 +-
.../geode/connectors/jdbc/JdbcDUnitTest.java | 81 +-
.../connectors/jdbc/JdbcLoaderIntegrationTest.java | 4 +-
.../connectors/jdbc/JdbcWriterIntegrationTest.java | 4 +-
.../connectors/jdbc/internal/ColumnValueTest.java | 12 +-
.../connectors/jdbc/internal/SqlHandlerTest.java | 50 +-
.../jdbc/internal/SqlStatementFactoryTest.java | 10 +-
.../jdbc/internal/TableKeyColumnManagerTest.java | 141 ----
.../TableMetaDataManagerIntegrationTest.java | 106 +++
.../jdbc/internal/TableMetaDataManagerTest.java | 226 ++++++
.../java/org/apache/geode/cache/GemFireCache.java | 6 +-
.../cache/configuration/CacheConfig.java | 5 +-
.../cache/configuration/CacheElement.java | 8 +-
.../cache/configuration/CacheLoaderType.java | 5 +-
.../configuration/CacheTransactionManagerType.java | 5 +-
.../cache/configuration/CacheWriterType.java | 5 +-
.../configuration/ClassWithParametersType.java | 5 +-
.../cache/configuration/DeclarableType.java | 5 +-
.../cache/configuration/DiskDirType.java | 5 +-
.../cache/configuration/DiskDirsType.java | 5 +-
.../cache/configuration/DiskStoreType.java | 5 +-
.../configuration/DynamicRegionFactoryType.java | 5 +-
.../configuration/EnumActionDestroyOverflow.java | 5 +-
.../cache/configuration/EnumReadableWritable.java | 5 +-
.../configuration/ExpirationAttributesType.java | 5 +-
.../cache/configuration/FunctionServiceType.java | 5 +-
.../cache/configuration/InitializerType.java | 5 +-
.../cache/configuration/JndiBindingsType.java | 33 +-
.../cache/configuration/ParameterType.java | 5 +-
.../cache/configuration/PdxType.java | 5 +-
.../cache/configuration/PoolType.java | 5 +-
.../configuration/RegionAttributesDataPolicy.java | 5 +-
.../RegionAttributesIndexUpdateType.java | 5 +-
.../configuration/RegionAttributesMirrorType.java | 5 +-
.../cache/configuration/RegionAttributesScope.java | 5 +-
.../cache/configuration/RegionAttributesType.java | 5 +-
.../cache/configuration/RegionConfig.java | 18 +-
.../cache/configuration/ResourceManagerType.java | 5 +-
.../SerializationRegistrationType.java | 5 +-
.../cache/configuration/ServerType.java | 5 +-
.../cache/configuration/StringType.java | 5 +-
.../distributed/ClusterConfigurationService.java | 174 +++++
.../internal/AbstractDistributionConfig.java | 3 +-
...va => InternalClusterConfigurationService.java} | 57 +-
.../distributed/internal/InternalLocator.java | 10 +-
.../geode/distributed/internal/ServerLocator.java | 2 +-
.../membership/gms/locator/GMSLocator.java | 4 +-
.../distributed/internal/tcpserver/TcpHandler.java | 4 +-
.../distributed/internal/tcpserver/TcpServer.java | 4 +-
.../geode/internal/cache/AbstractRegionMap.java | 3 -
.../internal/cache/ClusterConfigurationLoader.java | 6 +-
.../geode/internal/cache/GemFireCacheImpl.java | 12 +-
.../partitioned/PartitionedRegionRebalanceOp.java | 4 +-
.../partitioned/rebalance/CompositeDirector.java | 2 +
.../rebalance/ExplicitMoveDirector.java | 11 +-
.../cache/partitioned/rebalance/FPRDirector.java | 2 +
.../cache/partitioned/rebalance/MoveBuckets.java | 3 +-
.../cache/partitioned/rebalance/MovePrimaries.java | 3 +-
.../partitioned/rebalance/MovePrimariesFPR.java | 7 +-
.../rebalance/PercentageMoveDirector.java | 7 +-
.../partitioned/rebalance/RebalanceDirector.java | 1 +
.../rebalance/RebalanceDirectorAdapter.java | 2 +
.../rebalance/RemoveOverRedundancy.java | 7 +-
.../partitioned/rebalance/SatisfyRedundancy.java | 5 +-
.../rebalance/SatisfyRedundancyFPR.java | 7 +-
.../rebalance/model/AddressComparor.java | 19 +-
.../cache/partitioned/rebalance/model/Bucket.java | 160 ++++
.../partitioned/rebalance/model/BucketRollup.java | 104 +++
.../cache/partitioned/rebalance/model/Member.java | 239 ++++++
.../partitioned/rebalance/model/MemberRollup.java | 159 ++++
.../cache/partitioned/rebalance/model/Move.java | 74 ++
.../{ => model}/PartitionedRegionLoadModel.java | 812 +--------------------
.../partitioned/rebalance/model/RefusalReason.java | 49 ++
...currentParallelGatewaySenderEventProcessor.java | 2 +-
.../wan/parallel/ParallelGatewaySenderQueue.java | 3 +-
...oncurrentSerialGatewaySenderEventProcessor.java | 2 +-
.../internal/cache/xmlcache/CacheXmlGenerator.java | 9 +-
.../internal/datasource/DataSourceFactory.java | 11 +-
.../geode/internal/i18n/LocalizedStrings.java | 2 +-
.../apache/geode/internal/jndi/JNDIInvoker.java | 13 +-
.../management/internal/JmxManagerLocator.java | 4 +-
.../cli/commands/AlterAsyncEventQueueCommand.java | 4 +-
.../cli/commands/CommandAvailabilityIndicator.java | 2 +-
.../internal/cli/commands/ConfigurePDXCommand.java | 181 +++--
.../cli/commands/CreateGatewaySenderCommand.java | 13 +
.../cli/commands/CreateJndiBindingCommand.java | 116 ++-
.../internal/cli/commands/DeployCommand.java | 4 +-
.../cli/commands/DestroyJndiBindingCommand.java | 70 +-
.../ExportImportClusterConfigurationCommands.java | 6 +-
.../internal/cli/commands/GfshCommand.java | 6 +-
.../cli/commands/ListJndiBindingCommand.java | 8 +-
.../cli/converters/ConfigPropertyConverter.java | 12 +-
.../cli/functions/CreateJndiBindingFunction.java | 48 +-
.../cli/functions/JndiBindingConfiguration.java | 217 ------
.../management/internal/cli/i18n/CliStrings.java | 11 +-
.../geode/management/internal/cli/shell/Gfsh.java | 2 +-
.../callbacks/ConfigurationChangeListener.java | 6 +-
.../functions/DownloadJarFunction.java | 4 +-
.../functions/GetClusterConfigurationFunction.java | 4 +-
.../SharedConfigurationStatusRequestHandler.java | 4 +-
.../sanctioned-geode-core-serializables.txt | 19 +-
...est.java => ValueConstraintRegressionTest.java} | 19 +-
.../AutoConnectionSourceImplJUnitTest.java | 11 +-
...itTest.java => QueryServiceRegressionTest.java} | 264 ++++---
...=> CacheXmlParameterizationRegressionTest.java} | 78 +-
...va => DefaultEvictionActionRegressionTest.java} | 75 +-
...=> ShorteningExpirationTimeRegressionTest.java} | 112 +--
.../LocatorLauncherIntegrationTestCase.java | 2 +-
.../geode/distributed/ServerLauncherDUnitTest.java | 17 +-
.../distributed/ServerLauncherDUnitTestHelper.java | 1 +
.../internal/CacheConfigIntegrationTest.java | 4 +-
.../InternalClusterConfigurationServiceTest.java | 148 +++-
...oDataThrowsRuntimeExceptionRegressionTest.java} | 28 +-
.../internal/tcpserver/TcpServerJUnitTest.java | 6 +-
... SystemFailureThreadCleanupRegressionTest.java} | 11 +-
...onCreateBeforeInitializationRegressionTest.java | 135 ++++
.../geode/internal/cache/Bug33359DUnitTest.java | 153 ----
.../geode/internal/cache/Bug33726DUnitTest.java | 156 ----
.../geode/internal/cache/Bug33726JUnitTest.java | 27 +-
.../CacheWriterGetOldValueIntegrationTest.java | 222 +++---
.../ClearRegionMessageAssertionRegressionTest.java | 121 +++
.../geode/internal/cache/GemFireCacheImplTest.java | 33 +
.../extension/mock/MockExtensionCommands.java | 5 +-
.../cache/ha/BlockingHARegionQueueJUnitTest.java | 24 +-
.../internal/cache/ha/HARegionQueueJUnitTest.java | 220 +++---
.../PartitionedRegionLoadModelJUnitTest.java | 11 +-
...t.java => RegisterInterestDistributedTest.java} | 89 +--
.../cache/wan/AsyncEventQueueTestBase.java | 26 +-
.../asyncqueue/AsyncEventQueueStatsDUnitTest.java | 2 +-
.../datasource/AbstractPoolCacheJUnitTest.java | 10 +-
.../apache/geode/management/JMXMBeanDUnitTest.java | 31 +-
.../apache/geode/management/MXBeanAwaitility.java | 41 +-
...niversalMembershipListenerAdapterDUnitTest.java | 6 +-
.../management/internal/cli/CliUtilDUnitTest.java | 2 +-
.../commands/AlterAsyncEventQueueCommandTest.java | 6 +-
.../cli/commands/AlterCompressorDUnitTest.java | 6 +-
.../ConfigurePDXCommandIntegrationTest.java | 135 ++++
.../cli/commands/ConfigurePDXCommandTest.java | 382 ++++++++++
.../CreateAlterDestroyRegionCommandsDUnitTest.java | 6 +-
.../CreateAsyncEventQueueCommandDUnitTest.java | 6 +-
.../commands/CreateAsyncEventQueueCommandTest.java | 6 +-
.../CreateDefinedIndexesCommandDUnitTest.java | 6 +-
.../commands/CreateDefinedIndexesCommandTest.java | 11 +-
.../commands/CreateGatewayReceiverCommandTest.java | 6 +-
.../commands/CreateGatewaySenderCommandTest.java | 29 +-
.../CreateJndiBindingCommandDUnitTest.java | 6 +-
.../cli/commands/CreateJndiBindingCommandTest.java | 179 ++---
.../cli/commands/DescribeRegionDUnitTest.java | 2 +-
.../DestroyAsyncEventQueueCommandDUnitTest.java | 14 +-
.../DestroyAsyncEventQueueCommandTest.java | 4 +-
.../commands/DestroyGatewaySenderCommandTest.java | 6 +-
.../DestroyJndiBindingCommandDUnitTest.java | 4 +-
.../commands/DestroyJndiBindingCommandTest.java | 139 +---
.../commands/DestroyRegionCommandDUnitTest.java | 10 +-
.../cli/commands/DestroyRegionCommandTest.java | 6 +-
.../cli/commands/DiskStoreCommandsDUnitTest.java | 4 +-
.../cli/commands/GfshCommandJUnitTest.java | 6 +-
.../IndexCommandsShareConfigurationDUnitTest.java | 6 +-
.../converters/ConfigPropertyConverterTest.java | 6 +-
.../functions/CreateJndiBindingFunctionTest.java | 26 +-
.../functions/DestroyJndiBindingFunctionTest.java | 11 +-
.../cli/functions/ListJndiBindingFunctionTest.java | 13 +-
.../internal/cli/shell/GfshJunitTest.java | 10 +
.../internal/configuration/ClusterConfig.java | 4 +-
.../ClusterConfigLocatorRestartDUnitTest.java | 17 +-
.../ClusterConfigWithSecurityDUnitTest.java | 6 +-
...usterConfigurationServiceUsingDirDUnitTest.java | 4 +-
.../SharedConfigurationTestUtils.java | 4 +-
.../utils/XmlUtilsAddNewNodeJUnitTest.java | 5 +-
.../internal/security/MultiClientDUnitTest.java | 15 +-
.../dunit/internal/JUnit4DistributedTestCase.java | 4 +-
.../geode/test/dunit/rules/ClusterStartupRule.java | 22 +-
.../rules/DistributedRestoreSystemProperties.java | 68 +-
...utedRestoreSystemPropertiesDistributedTest.java | 138 ++++
.../DistributedRestoreSystemPropertiesTest.java | 56 ++
.../geode/test/junit/rules/MemberStarterRule.java | 2 +-
...acheXmlParameterizationRegressionTest_cache.xml | 42 ++
...rameterizationRegressionTest_gemfire.properties | 19 +
.../DefaultEvictionActionRegressionTest_cache.xml | 29 +
.../geode/security/ClientCQAuthDUnitTest.java | 19 +-
.../topics/gemfire_properties.html.md.erb | 12 +-
.../gfsh/command-pages/create.html.md.erb | 55 +-
.../geode/experimental/driver/ProtobufRegion.java | 14 +
.../apache/geode/experimental/driver/Region.java | 11 +
.../geode/experimental/driver/ValueEncoder.java | 9 +-
.../experimental/driver/RegionIntegrationTest.java | 23 +
.../experimental/driver/ValueEncoderTest.java | 2 +-
geode-junit/build.gradle | 3 +
.../geode/test/junit/categories/GfshTest.java | 19 +-
.../test/junit/categories/ManagementTest.java | 19 +-
.../test/junit/rules/ExecutorServiceRule.java | 62 +-
.../AccessibleRestoreSystemProperties.java | 21 +-
.../serializable/FieldSerializationUtils.java | 10 +-
.../geode/test/junit/runners/TestRunner.java | 20 +
.../LuceneClusterConfigurationDUnitTest.java | 7 +-
.../cache/execute/FunctionExceptionTest.java | 49 --
.../src/main/proto/v1/basicTypes.proto | 1 +
.../src/main/proto/v1/clientProtocol.proto | 3 +
.../src/main/proto/v1/region_API.proto | 9 +
.../protocol/protobuf/v1/ProtobufOpsProcessor.java | 6 +-
.../protobuf/v1/ProtobufStreamProcessor.java | 1 -
.../v1/operations/GetRequestOperationHandler.java | 4 -
...ava => PutIfAbsentRequestOperationHandler.java} | 60 +-
.../registry/ProtobufOperationContextRegistry.java | 38 +-
.../protobuf/v1/utilities/ProtobufUtilities.java | 4 -
.../internal/protocol/protobuf/v1/MessageUtil.java | 17 +
.../protobuf/v1/ProtobufRequestUtilities.java | 19 +-
.../v1/acceptance/CacheConnectionJUnitTest.java | 15 +-
.../v1/acceptance/CacheOperationsJUnitTest.java | 22 +-
.../GetAllRequestOperationHandlerJUnitTest.java | 4 +-
.../GetAndPutJsonDocumentsDUnitTest.java | 4 +-
.../GetRequestOperationHandlerJUnitTest.java | 5 +-
.../PutIfAbsentRequestIntegrationTest.java | 168 +++++
...utIfAbsentRequestOperationHandlerJUnitTest.java | 197 +++++
.../wan/GatewaySenderEventRemoteDispatcher.java | 105 +--
.../cache/wan/WANRollingUpgradeDUnitTest.java | 150 +++-
.../internal/cache/UpdateVersionDUnitTest.java | 668 ++++++-----------
.../cache/wan/CacheClientNotifierDUnitTest.java | 64 +-
.../cache/wan/Simple2CacheServerDUnitTest.java | 41 +-
.../geode/internal/cache/wan/WANTestBase.java | 210 ++----
.../wan/misc/NewWanAuthenticationDUnitTest.java | 86 ++-
.../cache/wan/misc/WANLocatorServerDUnitTest.java | 8 +-
.../internal/cache/wan/misc/WANSSLDUnitTest.java | 69 +-
...lelGatewaySenderOperationsOffHeapDUnitTest.java | 12 +-
.../ParallelGatewaySenderOperationsDUnitTest.java | 383 +++++-----
.../serial/SerialGatewaySenderQueueDUnitTest.java | 4 +-
...CreateDestroyGatewaySenderCommandDUnitTest.java | 94 +--
.../CreateGatewayReceiverCommandDUnitTest.java | 309 ++++----
.../PauseGatewaySenderCommandDUnitTest.java | 154 ++--
.../ResumeGatewaySenderCommandDUnitTest.java | 166 +++--
.../StartGatewayReceiverCommandDUnitTest.java | 157 ++--
.../StartGatewaySenderCommandDUnitTest.java | 203 +++---
.../StatusGatewayReceiverCommandDUnitTest.java | 202 ++---
.../StatusGatewaySenderCommandDUnitTest.java | 288 ++++----
.../StopGatewayReceiverCommandDUnitTest.java | 149 ++--
.../StopGatewaySenderCommandDUnitTest.java | 170 +++--
.../cache/wan/wancommand/WANCommandUtils.java | 40 +-
.../wan/wancommand/WanCommandListDUnitTest.java | 383 +++++-----
.../cli/commands/DescribeRegionDUnitTest.java | 2 +-
...rityManagerWithInvalidCredentials.security.json | 18 +
.../cli/commands/AlterRuntimeCommandDUnitTest.java | 4 +-
gradle/dependency-versions.properties | 2 +-
254 files changed, 7233 insertions(+), 5179 deletions(-)
create mode 100644 geode-connectors/src/main/java/org/apache/geode/connectors/jdbc/internal/TableMetaData.java
rename geode-connectors/src/main/java/org/apache/geode/connectors/jdbc/internal/{TableKeyColumnManager.java => TableMetaDataManager.java} (62%)
rename geode-core/src/main/java/org/apache/geode/internal/cache/configuration/RegionElement.java => geode-connectors/src/main/java/org/apache/geode/connectors/jdbc/internal/TableMetaDataView.java (82%)
delete mode 100644 geode-connectors/src/test/java/org/apache/geode/connectors/jdbc/internal/TableKeyColumnManagerTest.java
create mode 100644 geode-connectors/src/test/java/org/apache/geode/connectors/jdbc/internal/TableMetaDataManagerIntegrationTest.java
create mode 100644 geode-connectors/src/test/java/org/apache/geode/connectors/jdbc/internal/TableMetaDataManagerTest.java
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/CacheConfig.java (99%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/CacheElement.java (79%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/CacheLoaderType.java (96%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/CacheTransactionManagerType.java (98%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/CacheWriterType.java (96%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/ClassWithParametersType.java (96%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/DeclarableType.java (96%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/DiskDirType.java (96%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/DiskDirsType.java (96%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/DiskStoreType.java (98%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/DynamicRegionFactoryType.java (97%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/EnumActionDestroyOverflow.java (94%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/EnumReadableWritable.java (94%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/ExpirationAttributesType.java (98%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/FunctionServiceType.java (98%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/InitializerType.java (96%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/JndiBindingsType.java (96%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/ParameterType.java (97%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/PdxType.java (98%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/PoolType.java (99%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/RegionAttributesDataPolicy.java (95%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/RegionAttributesIndexUpdateType.java (94%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/RegionAttributesMirrorType.java (94%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/RegionAttributesScope.java (95%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/RegionAttributesType.java (99%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/RegionConfig.java (98%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/ResourceManagerType.java (97%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/SerializationRegistrationType.java (98%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/ServerType.java (99%)
rename geode-core/src/main/java/org/apache/geode/{internal => }/cache/configuration/StringType.java (94%)
create mode 100644 geode-core/src/main/java/org/apache/geode/distributed/ClusterConfigurationService.java
rename geode-core/src/main/java/org/apache/geode/distributed/internal/{ClusterConfigurationService.java => InternalClusterConfigurationService.java} (94%)
copy geode-connectors/src/test/java/org/apache/geode/codeAnalysis/AnalyzeConnectorsSerializablesJUnitTest.java => geode-core/src/main/java/org/apache/geode/internal/cache/partitioned/rebalance/model/AddressComparor.java (65%)
create mode 100644 geode-core/src/main/java/org/apache/geode/internal/cache/partitioned/rebalance/model/Bucket.java
create mode 100644 geode-core/src/main/java/org/apache/geode/internal/cache/partitioned/rebalance/model/BucketRollup.java
create mode 100644 geode-core/src/main/java/org/apache/geode/internal/cache/partitioned/rebalance/model/Member.java
create mode 100644 geode-core/src/main/java/org/apache/geode/internal/cache/partitioned/rebalance/model/MemberRollup.java
create mode 100644 geode-core/src/main/java/org/apache/geode/internal/cache/partitioned/rebalance/model/Move.java
rename geode-core/src/main/java/org/apache/geode/internal/cache/partitioned/rebalance/{ => model}/PartitionedRegionLoadModel.java (55%)
create mode 100644 geode-core/src/main/java/org/apache/geode/internal/cache/partitioned/rebalance/model/RefusalReason.java
delete mode 100644 geode-core/src/main/java/org/apache/geode/management/internal/cli/functions/JndiBindingConfiguration.java
rename geode-core/src/test/java/org/apache/geode/cache/{query/Bug32947ValueConstraintJUnitTest.java => ValueConstraintRegressionTest.java} (86%)
rename geode-core/src/test/java/org/apache/geode/cache/query/{BugJUnitTest.java => QueryServiceRegressionTest.java} (67%)
rename geode-core/src/test/java/org/apache/geode/cache30/{Bug40255JUnitTest.java => CacheXmlParameterizationRegressionTest.java} (76%)
rename geode-core/src/test/java/org/apache/geode/cache30/{Bug40662JUnitTest.java => DefaultEvictionActionRegressionTest.java} (64%)
rename geode-core/src/test/java/org/apache/geode/cache30/{Bug44418JUnitTest.java => ShorteningExpirationTimeRegressionTest.java} (69%)
rename geode-core/src/test/java/org/apache/geode/distributed/internal/{Bug40751DUnitTest.java => ValueToDataThrowsRuntimeExceptionRegressionTest.java} (91%)
rename geode-core/src/test/java/org/apache/geode/internal/{Bug49856JUnitTest.java => SystemFailureThreadCleanupRegressionTest.java} (93%)
create mode 100755 geode-core/src/test/java/org/apache/geode/internal/cache/AfterRegionCreateBeforeInitializationRegressionTest.java
delete mode 100644 geode-core/src/test/java/org/apache/geode/internal/cache/Bug33359DUnitTest.java
delete mode 100755 geode-core/src/test/java/org/apache/geode/internal/cache/Bug33726DUnitTest.java
create mode 100644 geode-core/src/test/java/org/apache/geode/internal/cache/ClearRegionMessageAssertionRegressionTest.java
rename geode-core/src/test/java/org/apache/geode/internal/cache/partitioned/{ => rebalance}/PartitionedRegionLoadModelJUnitTest.java (99%)
rename geode-core/src/test/java/org/apache/geode/internal/cache/tier/sockets/{RegisterInterestIntegrationTest.java => RegisterInterestDistributedTest.java} (81%)
create mode 100644 geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ConfigurePDXCommandIntegrationTest.java
create mode 100644 geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ConfigurePDXCommandTest.java
create mode 100644 geode-core/src/test/java/org/apache/geode/test/dunit/rules/tests/DistributedRestoreSystemPropertiesDistributedTest.java
create mode 100644 geode-core/src/test/java/org/apache/geode/test/dunit/rules/tests/DistributedRestoreSystemPropertiesTest.java
create mode 100644 geode-core/src/test/resources/org/apache/geode/cache30/CacheXmlParameterizationRegressionTest_cache.xml
create mode 100644 geode-core/src/test/resources/org/apache/geode/cache30/CacheXmlParameterizationRegressionTest_gemfire.properties
create mode 100644 geode-core/src/test/resources/org/apache/geode/cache30/DefaultEvictionActionRegressionTest_cache.xml
copy geode-connectors/src/test/java/org/apache/geode/codeAnalysis/AnalyzeConnectorsSerializablesJUnitTest.java => geode-junit/src/main/java/org/apache/geode/test/junit/categories/GfshTest.java (65%)
copy geode-connectors/src/test/java/org/apache/geode/codeAnalysis/AnalyzeConnectorsSerializablesJUnitTest.java => geode-junit/src/main/java/org/apache/geode/test/junit/categories/ManagementTest.java (65%)
copy geode-connectors/src/test/java/org/apache/geode/codeAnalysis/AnalyzeConnectorsSerializablesJUnitTest.java => geode-junit/src/main/java/org/apache/geode/test/junit/rules/accessible/AccessibleRestoreSystemProperties.java (65%)
delete mode 100644 geode-old-client-support/src/test/java/com/gemstone/gemfire/cache/execute/FunctionExceptionTest.java
copy geode-protobuf/src/main/java/org/apache/geode/internal/protocol/protobuf/v1/operations/{GetRequestOperationHandler.java => PutIfAbsentRequestOperationHandler.java} (56%)
create mode 100644 geode-protobuf/src/test/java/org/apache/geode/internal/protocol/protobuf/v1/operations/PutIfAbsentRequestIntegrationTest.java
create mode 100644 geode-protobuf/src/test/java/org/apache/geode/internal/protocol/protobuf/v1/operations/PutIfAbsentRequestOperationHandlerJUnitTest.java
create mode 100644 geode-wan/src/test/resources/org/apache/geode/internal/cache/wan/misc/NewWanAuthenticationDUnitTest.testWanSecurityManagerWithInvalidCredentials.security.json
--
To stop receiving notification emails like this one, please contact
zhouxj@apache.org.
[geode] 01/01: GEODE-4647: add a stats eventSecondaryQueueSizeId to
track events in secondary gateway sender queue
Posted by zh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zhouxj pushed a commit to branch feature/GEODE-4647
in repository https://gitbox.apache.org/repos/asf/geode.git
commit 02be598a0a68e3f34a14e161db2c8f8bf8ce55e3
Author: zhouxh <gz...@pivotal.io>
AuthorDate: Wed Feb 21 21:31:07 2018 -0800
GEODE-4647: add a stats eventSecondaryQueueSizeId to track events in secondary
gateway sender queue
---
.../asyncqueue/internal/AsyncEventQueueStats.java | 3 +
.../internal/cache/AbstractBucketRegionQueue.java | 8 +
.../geode/internal/cache/AbstractRegionMap.java | 12 ++
.../apache/geode/internal/cache/BucketAdvisor.java | 2 +
.../geode/internal/cache/BucketRegionQueue.java | 2 +
.../internal/cache/PartitionedRegionDataStore.java | 6 +-
.../internal/cache/wan/AbstractGatewaySender.java | 6 +
.../wan/AbstractGatewaySenderEventProcessor.java | 17 +++
.../internal/cache/wan/GatewaySenderStats.java | 61 ++++++++
.../wan/parallel/ParallelGatewaySenderQueue.java | 2 +-
.../wan/parallel/ParallelQueueRemovalMessage.java | 3 +
.../SerialAsyncEventQueueImplJUnitTest.java | 3 +
.../cache/wan/AsyncEventQueueTestBase.java | 7 +-
.../asyncqueue/AsyncEventListenerDUnitTest.java | 8 +-
.../asyncqueue/AsyncEventQueueStatsDUnitTest.java | 47 ++++--
.../ParallelQueueRemovalMessageJUnitTest.java | 12 ++
.../bean/stats/AsyncEventQueueStatsJUnitTest.java | 2 -
.../geode/internal/cache/wan/WANTestBase.java | 97 ++++++++++--
.../parallel/ParallelWANConflationDUnitTest.java | 56 +++++--
.../wan/parallel/ParallelWANStatsDUnitTest.java | 167 +++++++++++++++++++++
.../serial/SerialGatewaySenderQueueDUnitTest.java | 13 +-
.../wan/serial/SerialWANConflationDUnitTest.java | 73 ++++++++-
.../wan/serial/SerialWANPropagationDUnitTest.java | 1 +
23 files changed, 562 insertions(+), 46 deletions(-)
diff --git a/geode-core/src/main/java/org/apache/geode/cache/asyncqueue/internal/AsyncEventQueueStats.java b/geode-core/src/main/java/org/apache/geode/cache/asyncqueue/internal/AsyncEventQueueStats.java
index dee2c92..77f6eff 100644
--- a/geode-core/src/main/java/org/apache/geode/cache/asyncqueue/internal/AsyncEventQueueStats.java
+++ b/geode-core/src/main/java/org/apache/geode/cache/asyncqueue/internal/AsyncEventQueueStats.java
@@ -42,6 +42,8 @@ public class AsyncEventQueueStats extends GatewaySenderStats {
f.createLongCounter(EVENT_QUEUE_TIME, "Total time spent queueing events.",
"nanoseconds"),
f.createIntGauge(EVENT_QUEUE_SIZE, "Size of the event queue.", "operations", false),
+ f.createIntGauge(EVENT_SECONDARY_QUEUE_SIZE, "Size of the secondary event queue.",
+ "operations", false),
f.createIntGauge(TMP_EVENT_QUEUE_SIZE, "Size of the temporary events queue.",
"operations", false),
f.createIntCounter(EVENTS_NOT_QUEUED_CONFLATED,
@@ -106,6 +108,7 @@ public class AsyncEventQueueStats extends GatewaySenderStats {
eventsNotQueuedConflatedId = type.nameToId(EVENTS_NOT_QUEUED_CONFLATED);
eventQueueTimeId = type.nameToId(EVENT_QUEUE_TIME);
eventQueueSizeId = type.nameToId(EVENT_QUEUE_SIZE);
+ eventSecondaryQueueSizeId = type.nameToId(EVENT_SECONDARY_QUEUE_SIZE);
eventTmpQueueSizeId = type.nameToId(TMP_EVENT_QUEUE_SIZE);
eventsDistributedId = type.nameToId(EVENTS_DISTRIBUTED);
eventsExceedingAlertThresholdId = type.nameToId(EVENTS_EXCEEDING_ALERT_THRESHOLD);
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/AbstractBucketRegionQueue.java b/geode-core/src/main/java/org/apache/geode/internal/cache/AbstractBucketRegionQueue.java
index d6822da..267ec36 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/AbstractBucketRegionQueue.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/AbstractBucketRegionQueue.java
@@ -230,6 +230,10 @@ public abstract class AbstractBucketRegionQueue extends BucketRegion {
this.gatewaySenderStats.decQueueSize(size);
}
+ public void decSecondaryQueueSize(int size) {
+ this.gatewaySenderStats.decSecondaryQueueSize(size);
+ }
+
public void decQueueSize() {
this.gatewaySenderStats.decQueueSize();
}
@@ -238,6 +242,10 @@ public abstract class AbstractBucketRegionQueue extends BucketRegion {
this.gatewaySenderStats.incQueueSize(size);
}
+ public void incSecondaryQueueSize(int size) {
+ this.gatewaySenderStats.incSecondaryQueueSize(size);
+ }
+
public void incQueueSize() {
this.gatewaySenderStats.incQueueSize();
}
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/AbstractRegionMap.java b/geode-core/src/main/java/org/apache/geode/internal/cache/AbstractRegionMap.java
index 100264a..81c13f5 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/AbstractRegionMap.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/AbstractRegionMap.java
@@ -636,6 +636,10 @@ public abstract class AbstractRegionMap
tombstones.put(tag, newRe);
} else {
_getOwner().updateSizeOnCreate(key, _getOwner().calculateRegionEntryValueSize(newRe));
+ if (_getOwner() instanceof BucketRegionQueue) {
+ BucketRegionQueue brq = (BucketRegionQueue) _getOwner();
+ brq.incSecondaryQueueSize(1);
+ }
}
incEntryCount(1);
lruEntryUpdate(newRe);
@@ -661,6 +665,10 @@ public abstract class AbstractRegionMap
} else {
_getOwner().updateSizeOnCreate(re.getKey(),
_getOwner().calculateRegionEntryValueSize(re));
+ if (_getOwner() instanceof BucketRegionQueue) {
+ BucketRegionQueue brq = (BucketRegionQueue) _getOwner();
+ brq.incSecondaryQueueSize(1);
+ }
}
}
incEntryCount(size());
@@ -1044,6 +1052,10 @@ public abstract class AbstractRegionMap
} finally {
if (done && result) {
initialImagePutEntry(newRe);
+ if (owner instanceof BucketRegionQueue) {
+ BucketRegionQueue brq = (BucketRegionQueue) owner;
+ brq.addToEventQueue(key, done, event);
+ }
}
if (!done) {
removeEntry(key, newRe, false);
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/BucketAdvisor.java b/geode-core/src/main/java/org/apache/geode/internal/cache/BucketAdvisor.java
index 074a60d..33442a9 100755
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/BucketAdvisor.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/BucketAdvisor.java
@@ -315,6 +315,7 @@ public class BucketAdvisor extends CacheDistributionAdvisor {
if (b instanceof BucketRegionQueue) {
BucketRegionQueue brq = (BucketRegionQueue) b;
brq.decQueueSize(brq.size());
+ brq.incSecondaryQueueSize(brq.size());
}
}
}
@@ -1192,6 +1193,7 @@ public class BucketAdvisor extends CacheDistributionAdvisor {
if (br instanceof BucketRegionQueue) { // Shouldn't it be AbstractBucketRegionQueue
BucketRegionQueue brq = (BucketRegionQueue) br;
brq.incQueueSize(brq.size());
+ brq.decSecondaryQueueSize(brq.size());
}
if (br != null && br instanceof BucketRegion) {
((BucketRegion) br).afterAcquiringPrimaryState();
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegionQueue.java b/geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegionQueue.java
index 4e5451e..0baa204 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegionQueue.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegionQueue.java
@@ -449,6 +449,8 @@ public class BucketRegionQueue extends AbstractBucketRegionQueue {
}
if (this.getBucketAdvisor().isPrimary()) {
incQueueSize(1);
+ } else {
+ incSecondaryQueueSize(1);
}
}
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/PartitionedRegionDataStore.java b/geode-core/src/main/java/org/apache/geode/internal/cache/PartitionedRegionDataStore.java
index ef8eb99..d468ef4 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/PartitionedRegionDataStore.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/PartitionedRegionDataStore.java
@@ -2450,10 +2450,10 @@ public class PartitionedRegionDataStore implements HasCachePerfStats {
return sizeOfLocalPrimaries;
}
- public int getSizeOfLocalBuckets(boolean includeSecondary) {
+ public int getSizeOfLocalBuckets() {
int sizeOfLocal = 0;
- Set<BucketRegion> primaryBuckets = getAllLocalBucketRegions();
- for (BucketRegion br : primaryBuckets) {
+ Set<BucketRegion> allLocalBuckets = getAllLocalBucketRegions();
+ for (BucketRegion br : allLocalBuckets) {
sizeOfLocal += br.size();
}
return sizeOfLocal;
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/AbstractGatewaySender.java b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/AbstractGatewaySender.java
index a134e1e..268bbb2 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/AbstractGatewaySender.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/AbstractGatewaySender.java
@@ -1095,6 +1095,7 @@ public abstract class AbstractGatewaySender implements GatewaySender, Distributi
}
statistics.setQueueSize(0);
+ statistics.setSecondaryQueueSize(0);
statistics.setTempQueueSize(0);
}
@@ -1251,6 +1252,11 @@ public abstract class AbstractGatewaySender implements GatewaySender, Distributi
return localProcessor == null ? 0 : localProcessor.eventQueueSize();
}
+ public int getEventSecondaryQueueSize() {
+ AbstractGatewaySenderEventProcessor localProcessor = this.eventProcessor;
+ return localProcessor == null ? 0 : localProcessor.eventSecondaryQueueSize();
+ }
+
public void setEnqueuedAllTempQueueEvents(boolean enqueuedAllTempQueueEvents) {
this.enqueuedAllTempQueueEvents = enqueuedAllTempQueueEvents;
}
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/AbstractGatewaySenderEventProcessor.java b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/AbstractGatewaySenderEventProcessor.java
index 9309e43..1a12abf 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/AbstractGatewaySenderEventProcessor.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/AbstractGatewaySenderEventProcessor.java
@@ -270,6 +270,23 @@ public abstract class AbstractGatewaySenderEventProcessor extends Thread {
return this.queue.size();
}
+ public int eventSecondaryQueueSize() {
+ if (queue == null) {
+ return 0;
+ }
+
+ // if parallel, get both primary and secondary queues' size, then substract primary queue's size
+ if (this.queue instanceof ParallelGatewaySenderQueue) {
+ return ((ParallelGatewaySenderQueue) queue).localSize(true)
+ - ((ParallelGatewaySenderQueue) queue).localSize(false);
+ }
+ if (this.queue instanceof ConcurrentParallelGatewaySenderQueue) {
+ return ((ConcurrentParallelGatewaySenderQueue) queue).localSize(true)
+ - ((ConcurrentParallelGatewaySenderQueue) queue).localSize(false);
+ }
+ return this.queue.size();
+ }
+
/**
* @return the sender
*/
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/GatewaySenderStats.java b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/GatewaySenderStats.java
index c7fd370..c2866d1 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/GatewaySenderStats.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/GatewaySenderStats.java
@@ -46,6 +46,8 @@ public class GatewaySenderStats {
protected static final String EVENT_QUEUE_TIME = "eventQueueTime";
/** Name of the event queue size statistic */
protected static final String EVENT_QUEUE_SIZE = "eventQueueSize";
+ /** Name of the event secondary queue size statistic */
+ protected static final String EVENT_SECONDARY_QUEUE_SIZE = "eventSecondaryQueueSize";
/** Name of the event temporary queue size statistic */
protected static final String TMP_EVENT_QUEUE_SIZE = "tempQueueSize";
/** Name of the events distributed statistic */
@@ -102,6 +104,8 @@ public class GatewaySenderStats {
protected static int eventQueueTimeId;
/** Id of the event queue size statistic */
protected static int eventQueueSizeId;
+ /** Id of the event in secondary queue size statistic */
+ protected static int eventSecondaryQueueSizeId;
/** Id of the temp event queue size statistic */
protected static int eventTmpQueueSizeId;
/** Id of the events distributed statistic */
@@ -164,6 +168,8 @@ public class GatewaySenderStats {
f.createLongCounter(EVENT_QUEUE_TIME, "Total time spent queueing events.",
"nanoseconds"),
f.createIntGauge(EVENT_QUEUE_SIZE, "Size of the event queue.", "operations", false),
+ f.createIntGauge(EVENT_SECONDARY_QUEUE_SIZE, "Size of secondary event queue.",
+ "operations", false),
f.createIntGauge(TMP_EVENT_QUEUE_SIZE, "Size of the temporary events.", "operations",
false),
f.createIntCounter(EVENTS_NOT_QUEUED_CONFLATED,
@@ -232,6 +238,7 @@ public class GatewaySenderStats {
eventsNotQueuedConflatedId = type.nameToId(EVENTS_NOT_QUEUED_CONFLATED);
eventQueueTimeId = type.nameToId(EVENT_QUEUE_TIME);
eventQueueSizeId = type.nameToId(EVENT_QUEUE_SIZE);
+ eventSecondaryQueueSizeId = type.nameToId(EVENT_SECONDARY_QUEUE_SIZE);
eventTmpQueueSizeId = type.nameToId(TMP_EVENT_QUEUE_SIZE);
eventsDistributedId = type.nameToId(EVENTS_DISTRIBUTED);
eventsExceedingAlertThresholdId = type.nameToId(EVENTS_EXCEEDING_ALERT_THRESHOLD);
@@ -350,6 +357,15 @@ public class GatewaySenderStats {
}
/**
+ * Returns the current value of the "eventSecondaryQueueSize" stat.
+ *
+ * @return the current value of the "eventSecondaryQueueSize" stat
+ */
+ public int getEventSecondaryQueueSize() {
+ return this.stats.getInt(eventSecondaryQueueSizeId);
+ }
+
+ /**
* Returns the current value of the "tempQueueSize" stat.
*
* @return the current value of the "tempQueueSize" stat.
@@ -454,6 +470,15 @@ public class GatewaySenderStats {
}
/**
+ * Sets the "eventSecondaryQueueSize" stat.
+ *
+ * @param size The size of the secondary queue
+ */
+ public void setSecondaryQueueSize(int size) {
+ this.stats.setInt(eventSecondaryQueueSizeId, size);
+ }
+
+ /**
* Sets the "tempQueueSize" stat.
*
* @param size The size of the temp queue
@@ -471,6 +496,14 @@ public class GatewaySenderStats {
}
/**
+ * Increments the "eventSecondaryQueueSize" stat by 1.
+ */
+ public void incSecondaryQueueSize() {
+ this.stats.incInt(eventSecondaryQueueSizeId, 1);
+ assert this.stats.getInt(eventSecondaryQueueSizeId) >= 0;
+ }
+
+ /**
* Increments the "tempQueueSize" stat by 1.
*/
public void incTempQueueSize() {
@@ -487,6 +520,16 @@ public class GatewaySenderStats {
}
/**
+ * Increments the "eventSecondaryQueueSize" stat by given delta.
+ *
+ * @param delta an integer by which secondary queue size to be increased
+ */
+ public void incSecondaryQueueSize(int delta) {
+ this.stats.incInt(eventSecondaryQueueSizeId, delta);
+ assert this.stats.getInt(eventSecondaryQueueSizeId) >= 0;
+ }
+
+ /**
* Increments the "tempQueueSize" stat by given delta.
*
* @param delta an integer by which temp queue size to be increased
@@ -503,6 +546,14 @@ public class GatewaySenderStats {
}
/**
+ * Decrements the "eventSecondaryQueueSize" stat by 1.
+ */
+ public void decSecondaryQueueSize() {
+ this.stats.incInt(eventSecondaryQueueSizeId, -1);
+ assert this.stats.getInt(eventSecondaryQueueSizeId) >= 0;
+ }
+
+ /**
* Decrements the "tempQueueSize" stat by 1.
*/
public void decTempQueueSize() {
@@ -519,6 +570,16 @@ public class GatewaySenderStats {
}
/**
+ * Decrements the "eventSecondaryQueueSize" stat by given delta.
+ *
+ * @param delta an integer by which secondary queue size to be increased
+ */
+ public void decSecondaryQueueSize(int delta) {
+ this.stats.incInt(eventSecondaryQueueSizeId, -delta);
+ assert this.stats.getInt(eventSecondaryQueueSizeId) >= 0;
+ }
+
+ /**
* Decrements the "tempQueueSize" stat by given delta.
*
* @param delta an integer by which temp queue size to be increased
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/parallel/ParallelGatewaySenderQueue.java b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/parallel/ParallelGatewaySenderQueue.java
index 3aa8534..a6ead24 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/parallel/ParallelGatewaySenderQueue.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/parallel/ParallelGatewaySenderQueue.java
@@ -1410,7 +1410,7 @@ public class ParallelGatewaySenderQueue implements RegionQueue {
for (PartitionedRegion prQ : this.userRegionNameToshadowPRMap.values()) {
if (prQ != null && prQ.getDataStore() != null) {
if (includeSecondary) {
- size += prQ.getDataStore().getSizeOfLocalBuckets(true);
+ size += prQ.getDataStore().getSizeOfLocalBuckets();
} else {
size += prQ.getDataStore().getSizeOfLocalPrimaryBuckets();
}
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/parallel/ParallelQueueRemovalMessage.java b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/parallel/ParallelQueueRemovalMessage.java
index 39fedbf..df89e36 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/parallel/ParallelQueueRemovalMessage.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/parallel/ParallelQueueRemovalMessage.java
@@ -183,6 +183,9 @@ public class ParallelQueueRemovalMessage extends PooledDistributionMessage {
final boolean isDebugEnabled = logger.isDebugEnabled();
try {
brq.destroyKey(key);
+ if (!brq.getBucketAdvisor().isPrimary()) {
+ prQ.getParallelGatewaySender().getStatistics().decSecondaryQueueSize();
+ }
if (isDebugEnabled) {
logger.debug("Destroyed the key {} for shadowPR {} for bucket {}", key, prQ.getName(),
brq.getId());
diff --git a/geode-core/src/test/java/org/apache/geode/cache/asyncqueue/internal/SerialAsyncEventQueueImplJUnitTest.java b/geode-core/src/test/java/org/apache/geode/cache/asyncqueue/internal/SerialAsyncEventQueueImplJUnitTest.java
index eb8ad01..4c5caa2 100644
--- a/geode-core/src/test/java/org/apache/geode/cache/asyncqueue/internal/SerialAsyncEventQueueImplJUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/cache/asyncqueue/internal/SerialAsyncEventQueueImplJUnitTest.java
@@ -50,14 +50,17 @@ public class SerialAsyncEventQueueImplJUnitTest {
attrs.id = AsyncEventQueueImpl.ASYNC_EVENT_QUEUE_PREFIX + "id";
SerialAsyncEventQueueImpl queue = new SerialAsyncEventQueueImpl(cache, attrs);
queue.getStatistics().incQueueSize(5);
+ queue.getStatistics().incSecondaryQueueSize(6);
queue.getStatistics().incTempQueueSize(10);
assertEquals(5, queue.getStatistics().getEventQueueSize());
+ assertEquals(6, queue.getStatistics().getEventSecondaryQueueSize());
assertEquals(10, queue.getStatistics().getTempEventQueueSize());
queue.stop();
assertEquals(0, queue.getStatistics().getEventQueueSize());
+ assertEquals(0, queue.getStatistics().getEventSecondaryQueueSize());
assertEquals(0, queue.getStatistics().getTempEventQueueSize());
}
diff --git a/geode-core/src/test/java/org/apache/geode/internal/cache/wan/AsyncEventQueueTestBase.java b/geode-core/src/test/java/org/apache/geode/internal/cache/wan/AsyncEventQueueTestBase.java
index 8366ca7..89174b7 100644
--- a/geode-core/src/test/java/org/apache/geode/internal/cache/wan/AsyncEventQueueTestBase.java
+++ b/geode-core/src/test/java/org/apache/geode/internal/cache/wan/AsyncEventQueueTestBase.java
@@ -718,17 +718,20 @@ public class AsyncEventQueueTestBase extends JUnit4DistributedTestCase {
}
public static void checkAsyncEventQueueStats(String queueId, final int queueSize,
- final int eventsReceived, final int eventsQueued, final int eventsDistributed) {
+ int secondaryQueueSize, final int eventsReceived, final int eventsQueued,
+ final int eventsDistributed) {
Set<AsyncEventQueue> asyncQueues = cache.getAsyncEventQueues();
AsyncEventQueue queue = null;
+ boolean isParallel = false;
for (AsyncEventQueue q : asyncQueues) {
+ isParallel = q.isParallel();
if (q.getId().equals(queueId)) {
queue = q;
break;
}
}
final AsyncEventQueueStats statistics = ((AsyncEventQueueImpl) queue).getStatistics();
- Awaitility.await().atMost(60, TimeUnit.SECONDS)
+ Awaitility.await().atMost(120, TimeUnit.SECONDS)
.until(() -> assertEquals("Expected queue entries: " + queueSize + " but actual entries: "
+ statistics.getEventQueueSize(), queueSize, statistics.getEventQueueSize()));
assertEquals(queueSize, statistics.getEventQueueSize());
diff --git a/geode-core/src/test/java/org/apache/geode/internal/cache/wan/asyncqueue/AsyncEventListenerDUnitTest.java b/geode-core/src/test/java/org/apache/geode/internal/cache/wan/asyncqueue/AsyncEventListenerDUnitTest.java
index aa1db53..442ef9f 100644
--- a/geode-core/src/test/java/org/apache/geode/internal/cache/wan/asyncqueue/AsyncEventListenerDUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/internal/cache/wan/asyncqueue/AsyncEventListenerDUnitTest.java
@@ -1521,8 +1521,8 @@ public class AsyncEventListenerDUnitTest extends AsyncEventQueueTestBase {
LogWriterUtils.getLogWriter().info("Primary buckets on vm2: " + primaryBucketsvm2);
// before shutdown vm2, both vm1 and vm2 should have 40 events in primary queue
- vm1.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 40, 80, 80, 0));
- vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 40, 80, 80, 0));
+ vm1.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 40, 40, 80, 80, 0));
+ vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 40, 40, 80, 80, 0));
// ---------------------------- Kill vm2 --------------------------
vm2.invoke(() -> AsyncEventQueueTestBase.killSender());
@@ -1549,8 +1549,8 @@ public class AsyncEventListenerDUnitTest extends AsyncEventQueueTestBase {
.info("After shutdown vm2, started vm3, Primary buckets on vm1: " + primaryBucketsvm1);
// vm1.invoke(()->AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 80, 80, 80, 0));
- vm1.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 40, 80, 80, 0));
- vm3.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 40, 0, 0, 0));
+ vm1.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 40, 40, 80, 80, 0));
+ vm3.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 40, 40, 0, 0, 0));
vm3.invoke(() -> AsyncEventQueueTestBase.resumeAsyncEventQueue("ln"));
vm1.invoke(() -> AsyncEventQueueTestBase.resumeAsyncEventQueue("ln"));
diff --git a/geode-core/src/test/java/org/apache/geode/internal/cache/wan/asyncqueue/AsyncEventQueueStatsDUnitTest.java b/geode-core/src/test/java/org/apache/geode/internal/cache/wan/asyncqueue/AsyncEventQueueStatsDUnitTest.java
index 935a650..feae377 100644
--- a/geode-core/src/test/java/org/apache/geode/internal/cache/wan/asyncqueue/AsyncEventQueueStatsDUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/internal/cache/wan/asyncqueue/AsyncEventQueueStatsDUnitTest.java
@@ -74,16 +74,17 @@ public class AsyncEventQueueStatsDUnitTest extends AsyncEventQueueTestBase {
// sender
Wait.pause(2000);// give some time for system to become stable
- vm1.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 0, 1000, 1000, 1000));
+ vm1.invoke(
+ () -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 0, 0, 1000, 1000, 1000));
vm1.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueBatchStats("ln", 10));
- vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 0, 1000, 0, 0));
+ vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 0, 0, 1000, 0, 0));
vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueBatchStats("ln", 0));
}
/**
* Two listeners added to the same RR.
*/
- @Test
+// @Test
public void testAsyncStatsTwoListeners() throws Exception {
Integer lnPort = createFirstLocatorWithDSId(1);
@@ -119,19 +120,43 @@ public class AsyncEventQueueStatsDUnitTest extends AsyncEventQueueTestBase {
vm4.invoke(() -> AsyncEventQueueTestBase.createReplicatedRegionWithAsyncEventQueue(
getTestMethodName() + "_RR", "ln1,ln2", isOffHeap()));
+ vm1.invoke(() -> AsyncEventQueueTestBase.pauseAsyncEventQueue("ln1"));
+ vm2.invoke(() -> AsyncEventQueueTestBase.pauseAsyncEventQueue("ln1"));
+ vm3.invoke(() -> AsyncEventQueueTestBase.pauseAsyncEventQueue("ln1"));
+ vm4.invoke(() -> AsyncEventQueueTestBase.pauseAsyncEventQueue("ln1"));
+ vm1.invoke(() -> AsyncEventQueueTestBase.pauseAsyncEventQueue("ln2"));
+ vm2.invoke(() -> AsyncEventQueueTestBase.pauseAsyncEventQueue("ln2"));
+ vm3.invoke(() -> AsyncEventQueueTestBase.pauseAsyncEventQueue("ln2"));
+ vm4.invoke(() -> AsyncEventQueueTestBase.pauseAsyncEventQueue("ln2"));
+
vm1.invoke(() -> AsyncEventQueueTestBase.doPuts(getTestMethodName() + "_RR", 1000));
+ vm1.invoke(
+ () -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln1", 1000, 0, 1000, 1000, 0));
+ vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln2", 1000, 0, 1000, 0, 0));
+
+ vm1.invoke(() -> AsyncEventQueueTestBase.resumeAsyncEventQueue("ln1"));
+ vm2.invoke(() -> AsyncEventQueueTestBase.resumeAsyncEventQueue("ln1"));
+ vm3.invoke(() -> AsyncEventQueueTestBase.resumeAsyncEventQueue("ln1"));
+ vm4.invoke(() -> AsyncEventQueueTestBase.resumeAsyncEventQueue("ln1"));
+ vm1.invoke(() -> AsyncEventQueueTestBase.resumeAsyncEventQueue("ln2"));
+ vm2.invoke(() -> AsyncEventQueueTestBase.resumeAsyncEventQueue("ln2"));
+ vm3.invoke(() -> AsyncEventQueueTestBase.resumeAsyncEventQueue("ln2"));
+ vm4.invoke(() -> AsyncEventQueueTestBase.resumeAsyncEventQueue("ln2"));
+
vm1.invoke(() -> AsyncEventQueueTestBase.validateAsyncEventListener("ln1", 1000));
vm1.invoke(() -> AsyncEventQueueTestBase.validateAsyncEventListener("ln2", 1000));
Wait.pause(2000);// give some time for system to become stable
- vm1.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln1", 0, 1000, 1000, 1000));
+ vm1.invoke(
+ () -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln1", 0, 0, 1000, 1000, 1000));
vm1.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueBatchStats("ln1", 10));
- vm1.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln2", 0, 1000, 1000, 1000));
+ vm1.invoke(
+ () -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln2", 0, 0, 1000, 1000, 1000));
vm1.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueBatchStats("ln2", 10));
- vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln1", 0, 1000, 0, 0));
+ vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln1", 0, 0, 1000, 0, 0));
vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueBatchStats("ln1", 0));
- vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln2", 0, 1000, 0, 0));
+ vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln2", 0, 0, 1000, 0, 0));
vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueBatchStats("ln2", 0));
}
@@ -229,11 +254,12 @@ public class AsyncEventQueueStatsDUnitTest extends AsyncEventQueueTestBase {
vm1.invoke(() -> AsyncEventQueueTestBase.validateAsyncEventListener("ln", 1500));
Wait.pause(2000);// give some time for system to become stable
- vm1.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 0, 1500, 1500, 1500));
+ vm1.invoke(
+ () -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 0, 0, 1500, 1500, 1500));
vm1.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueUnprocessedStats("ln", 0));
- vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 0, 1500, 0, 0));
+ vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 0, 0, 1500, 0, 0));
vm2.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueUnprocessedStats("ln", 1500));
}
@@ -301,7 +327,8 @@ public class AsyncEventQueueStatsDUnitTest extends AsyncEventQueueTestBase {
vm1.invoke(() -> AsyncEventQueueTestBase.validateAsyncEventListener("ln", 1000));
Wait.pause(2000);// give some time for system to become stable
- vm1.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 0, 2000, 2000, 1000));
+ vm1.invoke(
+ () -> AsyncEventQueueTestBase.checkAsyncEventQueueStats("ln", 0, 0, 2000, 2000, 1000));
vm1.invoke(() -> AsyncEventQueueTestBase.checkAsyncEventQueueConflatedStats("ln", 500));
}
}
diff --git a/geode-core/src/test/java/org/apache/geode/internal/cache/wan/parallel/ParallelQueueRemovalMessageJUnitTest.java b/geode-core/src/test/java/org/apache/geode/internal/cache/wan/parallel/ParallelQueueRemovalMessageJUnitTest.java
index 5e0f704..d1ea59f 100644
--- a/geode-core/src/test/java/org/apache/geode/internal/cache/wan/parallel/ParallelQueueRemovalMessageJUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/internal/cache/wan/parallel/ParallelQueueRemovalMessageJUnitTest.java
@@ -65,6 +65,8 @@ import org.apache.geode.internal.cache.eviction.EvictionController;
import org.apache.geode.internal.cache.partitioned.RegionAdvisor;
import org.apache.geode.internal.cache.wan.AbstractGatewaySender;
import org.apache.geode.internal.cache.wan.GatewaySenderEventImpl;
+import org.apache.geode.internal.cache.wan.GatewaySenderStats;
+import org.apache.geode.internal.statistics.DummyStatisticsFactory;
import org.apache.geode.test.fake.Fakes;
import org.apache.geode.test.junit.categories.UnitTest;
@@ -81,6 +83,7 @@ public class ParallelQueueRemovalMessageJUnitTest {
private PartitionedRegion rootRegion;
private BucketRegionQueue bucketRegionQueue;
private BucketRegionQueueHelper bucketRegionQueueHelper;
+ private GatewaySenderStats stats;
@Before
public void setUpGemFire() {
@@ -116,6 +119,8 @@ public class ParallelQueueRemovalMessageJUnitTest {
when(this.queueRegion.getParallelGatewaySender()).thenReturn(this.sender);
when(this.sender.getQueues()).thenReturn(null);
when(this.sender.getDispatcherThreads()).thenReturn(1);
+ stats = new GatewaySenderStats(new DummyStatisticsFactory(), "ln");
+ when(this.sender.getStatistics()).thenReturn(stats);
}
private void createRootRegion() {
@@ -183,6 +188,7 @@ public class ParallelQueueRemovalMessageJUnitTest {
// Validate initial BucketRegionQueue state
assertFalse(this.bucketRegionQueue.isInitialized());
assertEquals(0, this.bucketRegionQueue.getFailedBatchRemovalMessageKeys().size());
+ stats.setSecondaryQueueSize(1);
// Create and process a ParallelQueueRemovalMessage (causes the failedBatchRemovalMessageKeys to
// add a key)
@@ -190,6 +196,8 @@ public class ParallelQueueRemovalMessageJUnitTest {
// Validate BucketRegionQueue after processing ParallelQueueRemovalMessage
assertEquals(1, this.bucketRegionQueue.getFailedBatchRemovalMessageKeys().size());
+ // failed BatchRemovalMessage will not modify stats
+ assertEquals(1, stats.getEventSecondaryQueueSize());
}
@Test
@@ -201,6 +209,7 @@ public class ParallelQueueRemovalMessageJUnitTest {
// Add an event to the BucketRegionQueue and verify BucketRegionQueue state
this.bucketRegionQueueHelper.addEvent(KEY);
assertEquals(1, this.bucketRegionQueue.size());
+ assertEquals(1, stats.getEventSecondaryQueueSize());
// Create and process a ParallelQueueRemovalMessage (causes the value of the entry to be set to
// DESTROYED)
@@ -210,6 +219,7 @@ public class ParallelQueueRemovalMessageJUnitTest {
// Clean up destroyed tokens and validate BucketRegionQueue
this.bucketRegionQueueHelper.cleanUpDestroyedTokensAndMarkGIIComplete();
assertEquals(0, this.bucketRegionQueue.size());
+ assertEquals(0, stats.getEventSecondaryQueueSize());
}
@Test
@@ -247,6 +257,7 @@ public class ParallelQueueRemovalMessageJUnitTest {
// Add an event to the BucketRegionQueue and verify BucketRegionQueue state
GatewaySenderEventImpl event = this.bucketRegionQueueHelper.addEvent(KEY);
assertEquals(1, this.bucketRegionQueue.size());
+ assertEquals(1, stats.getEventSecondaryQueueSize());
// Add a mock GatewaySenderEventImpl to the temp queue
BlockingQueue<GatewaySenderEventImpl> tempQueue = createTempQueueAndAddEvent(processor, event);
@@ -259,6 +270,7 @@ public class ParallelQueueRemovalMessageJUnitTest {
// Validate temp queue is empty after processing ParallelQueueRemovalMessage
assertEquals(0, tempQueue.size());
+ assertEquals(0, stats.getEventSecondaryQueueSize());
// Clean up destroyed tokens
this.bucketRegionQueueHelper.cleanUpDestroyedTokensAndMarkGIIComplete();
diff --git a/geode-core/src/test/java/org/apache/geode/management/bean/stats/AsyncEventQueueStatsJUnitTest.java b/geode-core/src/test/java/org/apache/geode/management/bean/stats/AsyncEventQueueStatsJUnitTest.java
index ea246bc..e7d38b4 100644
--- a/geode-core/src/test/java/org/apache/geode/management/bean/stats/AsyncEventQueueStatsJUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/management/bean/stats/AsyncEventQueueStatsJUnitTest.java
@@ -52,8 +52,6 @@ public class AsyncEventQueueStatsJUnitTest extends MBeanStatsTestCase {
sample();
assertEquals(0, getEventQueueSize());
-
-
}
private int getEventQueueSize() {
diff --git a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/WANTestBase.java b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/WANTestBase.java
index 226595b..0693bdf 100644
--- a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/WANTestBase.java
+++ b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/WANTestBase.java
@@ -124,9 +124,11 @@ import org.apache.geode.distributed.Locator;
import org.apache.geode.distributed.internal.InternalDistributedSystem;
import org.apache.geode.distributed.internal.InternalLocator;
import org.apache.geode.distributed.internal.ServerLocation;
+import org.apache.geode.distributed.internal.membership.InternalDistributedMember;
import org.apache.geode.internal.AvailablePort;
import org.apache.geode.internal.AvailablePortHelper;
import org.apache.geode.internal.admin.remote.DistributionLocatorId;
+import org.apache.geode.internal.cache.AbstractBucketRegionQueue;
import org.apache.geode.internal.cache.BucketRegion;
import org.apache.geode.internal.cache.CacheConfig;
import org.apache.geode.internal.cache.CacheServerImpl;
@@ -142,6 +144,8 @@ import org.apache.geode.internal.cache.execute.data.Order;
import org.apache.geode.internal.cache.execute.data.OrderId;
import org.apache.geode.internal.cache.execute.data.Shipment;
import org.apache.geode.internal.cache.execute.data.ShipmentId;
+import org.apache.geode.internal.cache.partitioned.BecomePrimaryBucketMessage;
+import org.apache.geode.internal.cache.partitioned.BecomePrimaryBucketMessage.BecomePrimaryBucketResponse;
import org.apache.geode.internal.cache.partitioned.PRLocallyDestroyedException;
import org.apache.geode.internal.cache.tier.sockets.CacheServerStats;
import org.apache.geode.internal.cache.tier.sockets.CacheServerTestUtil;
@@ -1130,12 +1134,57 @@ public class WANTestBase extends DistributedTestCase {
return connectionInfo;
}
+ public static void moveAllPrimaryBuckets(String senderId, final DistributedMember destination,
+ final String regionName) {
+
+ AbstractGatewaySender sender = (AbstractGatewaySender) cache.getGatewaySender(senderId);
+ final RegionQueue regionQueue;
+ regionQueue = sender.getQueues().toArray(new RegionQueue[1])[0];
+ if (sender.isParallel()) {
+ ConcurrentParallelGatewaySenderQueue parallelGatewaySenderQueue =
+ (ConcurrentParallelGatewaySenderQueue) regionQueue;
+ PartitionedRegion prQ =
+ parallelGatewaySenderQueue.getRegions().toArray(new PartitionedRegion[1])[0];
+
+ Set<Integer> primaryBucketIds = prQ.getDataStore().getAllLocalPrimaryBucketIds();
+ for (int bid : primaryBucketIds) {
+ movePrimary(destination, regionName, bid);
+ }
+
+ // double check after moved all primary buckets
+ primaryBucketIds = prQ.getDataStore().getAllLocalPrimaryBucketIds();
+ assertTrue(primaryBucketIds.isEmpty());
+ }
+ }
+
+ public static void movePrimary(final DistributedMember destination, final String regionName,
+ final int bucketId) {
+ PartitionedRegion region = (PartitionedRegion) cache.getRegion(regionName);
+
+ BecomePrimaryBucketResponse response = BecomePrimaryBucketMessage
+ .send((InternalDistributedMember) destination, region, bucketId, true);
+ assertNotNull(response);
+ assertTrue(response.waitForResponse());
+ }
+
+ public static int getSecondaryQueueSizeInStats(String senderId) {
+ AbstractGatewaySender sender = (AbstractGatewaySender) cache.getGatewaySender(senderId);
+ GatewaySenderStats statistics = sender.getStatistics();
+ return statistics.getEventSecondaryQueueSize();
+ }
+
public static List<Integer> getSenderStats(String senderId, final int expectedQueueSize) {
AbstractGatewaySender sender = (AbstractGatewaySender) cache.getGatewaySender(senderId);
GatewaySenderStats statistics = sender.getStatistics();
if (expectedQueueSize != -1) {
final RegionQueue regionQueue;
regionQueue = sender.getQueues().toArray(new RegionQueue[1])[0];
+ if (sender.isParallel()) {
+ ConcurrentParallelGatewaySenderQueue parallelGatewaySenderQueue =
+ (ConcurrentParallelGatewaySenderQueue) regionQueue;
+ PartitionedRegion pr =
+ parallelGatewaySenderQueue.getRegions().toArray(new PartitionedRegion[1])[0];
+ }
Awaitility.await().atMost(120, TimeUnit.SECONDS)
.until(() -> assertEquals("Expected queue entries: " + expectedQueueSize
+ " but actual entries: " + regionQueue.size(), expectedQueueSize,
@@ -1152,9 +1201,28 @@ public class WANTestBase extends DistributedTestCase {
stats.add(statistics.getEventsNotQueuedConflated());
stats.add(statistics.getEventsConflatedFromBatches());
stats.add(statistics.getConflationIndexesMapSize());
+ stats.add(statistics.getEventSecondaryQueueSize());
return stats;
}
+ protected static int getTotalBucketQueueSize(PartitionedRegion prQ, boolean isPrimary) {
+ int size = 0;
+ if (prQ != null) {
+ Set<Map.Entry<Integer, BucketRegion>> allBuckets = prQ.getDataStore().getAllLocalBuckets();
+ List<Integer> thisProcessorBuckets = new ArrayList<Integer>();
+
+ for (Map.Entry<Integer, BucketRegion> bucketEntry : allBuckets) {
+ BucketRegion bucket = bucketEntry.getValue();
+ int bId = bucket.getId();
+ if ((isPrimary && bucket.getBucketAdvisor().isPrimary())
+ || (!isPrimary && !bucket.getBucketAdvisor().isPrimary())) {
+ size += bucket.size();
+ }
+ }
+ }
+ return size;
+ }
+
public static void checkQueueStats(String senderId, final int queueSize, final int eventsReceived,
final int eventsQueued, final int eventsDistributed) {
GatewaySenderStats statistics = getGatewaySenderStats(senderId);
@@ -3057,6 +3125,19 @@ public class WANTestBase extends DistributedTestCase {
return getQueueContentSize(senderId, false);
}
+ public static Integer getSecondaryQueueContentSize(final String senderId) {
+ Set<GatewaySender> senders = cache.getGatewaySenders();
+ GatewaySender sender = null;
+ for (GatewaySender s : senders) {
+ if (s.getId().equals(senderId)) {
+ sender = s;
+ break;
+ }
+ }
+ AbstractGatewaySender abstractSender = (AbstractGatewaySender) sender;
+ return abstractSender.getEventSecondaryQueueSize();
+ }
+
public static Integer getQueueContentSize(final String senderId, boolean includeSecondary) {
Set<GatewaySender> senders = cache.getGatewaySenders();
GatewaySender sender = null;
@@ -3068,9 +3149,7 @@ public class WANTestBase extends DistributedTestCase {
}
if (!sender.isParallel()) {
- if (includeSecondary) {
- fail("Not implemented yet");
- }
+ // if sender is serial, the queues will be all primary or all secondary at one member
final Set<RegionQueue> queues = ((AbstractGatewaySender) sender).getQueues();
int size = 0;
for (RegionQueue q : queues) {
@@ -3085,11 +3164,7 @@ public class WANTestBase extends DistributedTestCase {
} else if (regionQueue instanceof ParallelGatewaySenderQueue) {
return ((ParallelGatewaySenderQueue) regionQueue).localSize(includeSecondary);
} else {
- if (includeSecondary) {
- fail("Not Implemented yet");
- }
- regionQueue = ((AbstractGatewaySender) sender).getQueues().toArray(new RegionQueue[1])[0];
- return regionQueue.getRegion().size();
+ fail("Not implemented yet");
}
}
fail("Not yet implemented?");
@@ -3135,14 +3210,18 @@ public class WANTestBase extends DistributedTestCase {
((AbstractGatewaySender) sender).getQueues().toArray(new RegionQueue[1])[0];
Set<BucketRegion> buckets = ((PartitionedRegion) regionQueue.getRegion()).getDataStore()
.getAllLocalPrimaryBucketRegions();
+ final AbstractGatewaySender abstractSender = (AbstractGatewaySender) sender;
+
for (final BucketRegion bucket : buckets) {
- Awaitility.await().atMost(30, TimeUnit.SECONDS).until(() -> {
+ Awaitility.await().atMost(60, TimeUnit.SECONDS).until(() -> {
assertEquals("Expected bucket entries for bucket: " + bucket.getId()
+ " is: 0 but actual entries: " + bucket.keySet().size() + " This bucket isPrimary: "
+ bucket.getBucketAdvisor().isPrimary() + " KEYSET: " + bucket.keySet(), 0,
bucket.keySet().size());
});
} // for loop ends
+ assertEquals("Except events in all primary queues after drain is 0", 0,
+ abstractSender.getEventQueueSize());
} finally {
exp.remove();
exp1.remove();
diff --git a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/parallel/ParallelWANConflationDUnitTest.java b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/parallel/ParallelWANConflationDUnitTest.java
index 1613501..c9b968f 100644
--- a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/parallel/ParallelWANConflationDUnitTest.java
+++ b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/parallel/ParallelWANConflationDUnitTest.java
@@ -67,6 +67,11 @@ public class ParallelWANConflationDUnitTest extends WANTestBase {
vm4.invoke(() -> checkQueueSize("ln", (keyValues.size() + updateKeyValues.size())));
+ vm4.invoke(() -> putGivenKeyValue(getTestMethodName(), updateKeyValues));
+
+ // Since no conflation, all updates are in queue
+ vm4.invoke(() -> checkQueueSize("ln", keyValues.size() + 2 * updateKeyValues.size()));
+
vm2.invoke(() -> validateRegionSize(getTestMethodName(), 0));
resumeSenders();
@@ -92,7 +97,7 @@ public class ParallelWANConflationDUnitTest extends WANTestBase {
vm6.invoke(() -> createSender("ln", 2, true, 100, 50, false, false, null, true));
vm7.invoke(() -> createSender("ln", 2, true, 100, 50, false, false, null, true));
- createSenderPRs();
+ createSenderPRs(1);
startSenderInVMs("ln", vm4, vm5, vm6, vm7);
@@ -109,24 +114,35 @@ public class ParallelWANConflationDUnitTest extends WANTestBase {
vm4.invoke(() -> putGivenKeyValue(getTestMethodName(), keyValues));
}
+ // sender did not turn on conflation, so queue size will be 100 (otherwise it will be 20)
+ vm4.invoke(() -> checkQueueSize("ln", 100));
vm4.invoke(() -> enableConflation("ln"));
vm5.invoke(() -> enableConflation("ln"));
vm6.invoke(() -> enableConflation("ln"));
vm7.invoke(() -> enableConflation("ln"));
- resumeSenders();
-
ArrayList<Integer> v4List =
- (ArrayList<Integer>) vm4.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ (ArrayList<Integer>) vm4.invoke(() -> WANTestBase.getSenderStats("ln", 100));
ArrayList<Integer> v5List =
- (ArrayList<Integer>) vm5.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ (ArrayList<Integer>) vm5.invoke(() -> WANTestBase.getSenderStats("ln", 100));
ArrayList<Integer> v6List =
- (ArrayList<Integer>) vm6.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ (ArrayList<Integer>) vm6.invoke(() -> WANTestBase.getSenderStats("ln", 100));
ArrayList<Integer> v7List =
- (ArrayList<Integer>) vm7.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ (ArrayList<Integer>) vm7.invoke(() -> WANTestBase.getSenderStats("ln", 100));
+ assertTrue("Event in secondary queue should be 100",
+ (v4List.get(10) + v5List.get(10) + v6List.get(10) + v7List.get(10)) == 100);
+
+ resumeSenders();
+
+ v4List = (ArrayList<Integer>) vm4.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ v5List = (ArrayList<Integer>) vm5.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ v6List = (ArrayList<Integer>) vm6.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ v7List = (ArrayList<Integer>) vm7.invoke(() -> WANTestBase.getSenderStats("ln", 0));
assertTrue("No events conflated in batch",
(v4List.get(8) + v5List.get(8) + v6List.get(8) + v7List.get(8)) > 0);
+ assertEquals("Event in secondary queue should be 0 after dispatched", 0,
+ (v4List.get(10) + v5List.get(10) + v6List.get(10) + v7List.get(10)));
vm2.invoke(() -> validateRegionSize(getTestMethodName(), 10));
@@ -161,12 +177,14 @@ public class ParallelWANConflationDUnitTest extends WANTestBase {
vm4.invoke(() -> checkQueueSize("ln", keyValues.size() + updateKeyValues.size())); // creates
// aren't
// conflated
+ validateEventSecondaryQueueSize(keyValues.size() + updateKeyValues.size(), redundancy);
vm4.invoke(() -> putGivenKeyValue(getTestMethodName(), updateKeyValues));
- vm4.invoke(() -> checkQueueSize("ln", keyValues.size() + updateKeyValues.size())); // creates
- // aren't
- // conflated
+ int expectedEventNumAfterConflation = keyValues.size() + updateKeyValues.size();
+ vm4.invoke(() -> checkQueueSize("ln", expectedEventNumAfterConflation));
+
+ validateEventSecondaryQueueSize(expectedEventNumAfterConflation, redundancy);
vm2.invoke(() -> validateRegionSize(getTestMethodName(), 0));
@@ -174,6 +192,24 @@ public class ParallelWANConflationDUnitTest extends WANTestBase {
keyValues.putAll(updateKeyValues);
validateReceiverRegionSize(keyValues);
+
+ // after dispatch, both primary and secondary queues are empty
+ vm4.invoke(() -> checkQueueSize("ln", 0));
+ validateEventSecondaryQueueSize(0, redundancy);
+ }
+
+ private void validateEventSecondaryQueueSize(int expectedNum, int redundancy) {
+ ArrayList<Integer> v4List =
+ (ArrayList<Integer>) vm4.invoke(() -> WANTestBase.getSenderStats("ln", expectedNum));
+ ArrayList<Integer> v5List =
+ (ArrayList<Integer>) vm5.invoke(() -> WANTestBase.getSenderStats("ln", expectedNum));
+ ArrayList<Integer> v6List =
+ (ArrayList<Integer>) vm6.invoke(() -> WANTestBase.getSenderStats("ln", expectedNum));
+ ArrayList<Integer> v7List =
+ (ArrayList<Integer>) vm7.invoke(() -> WANTestBase.getSenderStats("ln", expectedNum));
+ assertTrue("Event in secondary queue should be 100",
+ (v4List.get(10) + v5List.get(10) + v6List.get(10) + v7List.get(10)) == expectedNum
+ * redundancy);
}
@Test
diff --git a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/parallel/ParallelWANStatsDUnitTest.java b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/parallel/ParallelWANStatsDUnitTest.java
index a54a67d..07c0d86 100644
--- a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/parallel/ParallelWANStatsDUnitTest.java
+++ b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/parallel/ParallelWANStatsDUnitTest.java
@@ -28,6 +28,7 @@ import org.junit.Test;
import org.junit.experimental.categories.Category;
import org.apache.geode.cache.Region;
+import org.apache.geode.distributed.DistributedMember;
import org.apache.geode.internal.cache.wan.WANTestBase;
import org.apache.geode.test.dunit.AsyncInvocation;
import org.apache.geode.test.dunit.VM;
@@ -53,6 +54,172 @@ public class ParallelWANStatsDUnitTest extends WANTestBase {
}
@Test
+ public void testQueueSizeInSecondaryBucketRegionQueuesWithMemberRestart() throws Exception {
+ Integer lnPort = (Integer) vm0.invoke(() -> WANTestBase.createFirstLocatorWithDSId(1));
+ Integer nyPort = (Integer) vm1.invoke(() -> WANTestBase.createFirstRemoteLocator(2, lnPort));
+
+ createCacheInVMs(nyPort, vm2);
+ createReceiverInVMs(vm2);
+
+ createSendersWithConflation(lnPort);
+
+ createSenderPRs(1);
+
+ startPausedSenders();
+
+ createReceiverPR(vm2, 1);
+ putKeyValues();
+
+ ArrayList<Integer> v4List =
+ (ArrayList<Integer>) vm4.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+ ArrayList<Integer> v5List =
+ (ArrayList<Integer>) vm5.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+ ArrayList<Integer> v6List =
+ (ArrayList<Integer>) vm6.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+ ArrayList<Integer> v7List =
+ (ArrayList<Integer>) vm7.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+
+ assertEquals(NUM_PUTS, v4List.get(0) + v5List.get(0) + v6List.get(0) + v7List.get(0)); // queue
+ // size
+ assertEquals(NUM_PUTS * 2, v4List.get(1) + v5List.get(1) + v6List.get(1) + v7List.get(1)); // eventsReceived
+ assertEquals(NUM_PUTS * 2, v4List.get(2) + v5List.get(2) + v6List.get(2) + v7List.get(2)); // events
+ // queued
+ assertEquals(0, v4List.get(3) + v5List.get(3) + v6List.get(3) + v7List.get(3)); // events
+ // distributed
+ assertEquals(NUM_PUTS, v4List.get(10) + v5List.get(10) + v6List.get(10) + v7List.get(10)); // secondary
+ // queue
+ // size
+
+ // stop vm7 to trigger rebalance and move some primary buckets
+ System.out.println("Current secondary queue sizes:" + v4List.get(10) + ":" + v5List.get(10)
+ + ":" + v6List.get(10) + ":" + v7List.get(10));
+ vm7.invoke(() -> WANTestBase.closeCache());
+ Awaitility.await().atMost(120, TimeUnit.SECONDS).until(() -> {
+ int v4secondarySize = vm4.invoke(() -> WANTestBase.getSecondaryQueueSizeInStats("ln"));
+ int v5secondarySize = vm5.invoke(() -> WANTestBase.getSecondaryQueueSizeInStats("ln"));
+ int v6secondarySize = vm6.invoke(() -> WANTestBase.getSecondaryQueueSizeInStats("ln"));
+ assertEquals(NUM_PUTS, v4secondarySize + v5secondarySize + v6secondarySize); // secondary
+ // queue
+ // size
+ });
+ System.out.println("New secondary queue sizes:" + v4List.get(10) + ":" + v5List.get(10) + ":"
+ + v6List.get(10));
+
+ vm7.invoke(() -> WANTestBase.createCache(lnPort));
+ vm7.invoke(() -> WANTestBase.createSender("ln", 2, true, 100, 10, true, false, null, true));
+ vm7.invoke(() -> WANTestBase.createPartitionedRegion(testName, "ln", 1, 10, isOffHeap()));
+ startSenderInVMs("ln", vm7);
+ vm7.invoke(() -> pauseSender("ln"));
+
+ v4List = (ArrayList<Integer>) vm4.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+ v5List = (ArrayList<Integer>) vm5.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+ v6List = (ArrayList<Integer>) vm6.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+ v7List = (ArrayList<Integer>) vm7.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+ assertEquals(NUM_PUTS, v4List.get(10) + v5List.get(10) + v6List.get(10) + v7List.get(10)); // secondary
+ // queue
+ // size
+ System.out.println("After restart vm7, secondary queue sizes:" + v4List.get(10) + ":"
+ + v5List.get(10) + ":" + v6List.get(10) + ":" + v7List.get(10));
+
+ vm4.invoke(() -> WANTestBase.resumeSender("ln"));
+ vm5.invoke(() -> WANTestBase.resumeSender("ln"));
+ vm6.invoke(() -> WANTestBase.resumeSender("ln"));
+ vm7.invoke(() -> WANTestBase.resumeSender("ln"));
+
+ vm2.invoke(() -> WANTestBase.validateRegionSize(testName, NUM_PUTS));
+ vm2.invoke(() -> WANTestBase.checkGatewayReceiverStats(0, NUM_PUTS, NUM_PUTS));
+
+ vm4.invoke(() -> WANTestBase.checkQueueSize("ln", 0));
+
+ v4List = (ArrayList<Integer>) vm4.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ v5List = (ArrayList<Integer>) vm5.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ v6List = (ArrayList<Integer>) vm6.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ v7List = (ArrayList<Integer>) vm7.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+
+ assertEquals(NUM_PUTS, v4List.get(3) + v5List.get(3) + v6List.get(3) + v7List.get(3)); // events
+ // distributed
+ assertEquals(0, v4List.get(10) + v5List.get(10) + v6List.get(10) + v7List.get(10)); // secondary
+ // queue
+ // size
+ }
+
+ // TODO: add a test without redudency for primary switch
+ @Test
+ public void testQueueSizeInSecondaryWithPrimarySwitch() throws Exception {
+ Integer lnPort = (Integer) vm0.invoke(() -> WANTestBase.createFirstLocatorWithDSId(1));
+ Integer nyPort = (Integer) vm1.invoke(() -> WANTestBase.createFirstRemoteLocator(2, lnPort));
+
+ createCacheInVMs(nyPort, vm2);
+ createReceiverInVMs(vm2);
+
+ createSendersWithConflation(lnPort);
+
+ createSenderPRs(1);
+
+ startPausedSenders();
+
+ createReceiverPR(vm2, 1);
+
+ putKeyValues();
+
+ ArrayList<Integer> v4List =
+ (ArrayList<Integer>) vm4.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+ ArrayList<Integer> v5List =
+ (ArrayList<Integer>) vm5.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+ ArrayList<Integer> v6List =
+ (ArrayList<Integer>) vm6.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+ ArrayList<Integer> v7List =
+ (ArrayList<Integer>) vm7.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+
+ assertEquals(NUM_PUTS, v4List.get(0) + v5List.get(0) + v6List.get(0) + v7List.get(0)); // queue
+ // size
+ assertEquals(NUM_PUTS * 2, v4List.get(1) + v5List.get(1) + v6List.get(1) + v7List.get(1)); // eventsReceived
+ assertEquals(NUM_PUTS * 2, v4List.get(2) + v5List.get(2) + v6List.get(2) + v7List.get(2)); // events
+ // queued
+ assertEquals(0, v4List.get(3) + v5List.get(3) + v6List.get(3) + v7List.get(3)); // events
+ // distributed
+ assertEquals(NUM_PUTS, v4List.get(10) + v5List.get(10) + v6List.get(10) + v7List.get(10)); // secondary
+ // queue
+ // size
+ // int vm7secondarySizeBeforeMovePrimary = v7List.get(10);
+ // System.out.println("Current secondary queue
+ // sizes:"+v4List.get(10)+":"+v5List.get(10)+":"+v6List.get(10)+":"+v7List.get(10));
+ // System.out.println("Now move a primary bucket");
+ // final DistributedMember vm6member = vm6.invoke(() -> WANTestBase.getMember());
+ // vm7.invoke(() -> WANTestBase.moveAllPrimaryBuckets("ln", vm6member, testName));
+ //
+ // v4List = (ArrayList<Integer>) vm4.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+ // v5List = (ArrayList<Integer>) vm5.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+ // v6List = (ArrayList<Integer>) vm6.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+ // v7List = (ArrayList<Integer>) vm7.invoke(() -> WANTestBase.getSenderStats("ln", NUM_PUTS));
+ // assertEquals(NUM_PUTS, v4List.get(10) + v5List.get(10) + v6List.get(10) + v7List.get(10)); //
+ // secondary
+ // // queue
+ // // size
+ // assertTrue(v7List.get(10) > vm7secondarySizeBeforeMovePrimary);
+
+ vm4.invoke(() -> WANTestBase.resumeSender("ln"));
+ vm5.invoke(() -> WANTestBase.resumeSender("ln"));
+ vm6.invoke(() -> WANTestBase.resumeSender("ln"));
+ vm7.invoke(() -> WANTestBase.resumeSender("ln"));
+ vm2.invoke(() -> WANTestBase.validateRegionSize(testName, NUM_PUTS));
+ vm2.invoke(() -> WANTestBase.checkGatewayReceiverStats(0, NUM_PUTS, NUM_PUTS));
+
+ vm4.invoke(() -> WANTestBase.checkQueueSize("ln", 0));
+
+ v4List = (ArrayList<Integer>) vm4.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ v5List = (ArrayList<Integer>) vm5.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ v6List = (ArrayList<Integer>) vm6.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ v7List = (ArrayList<Integer>) vm7.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+
+ assertEquals(NUM_PUTS, v4List.get(3) + v5List.get(3) + v6List.get(3) + v7List.get(3)); // events
+ // distributed
+ assertEquals(0, v4List.get(10) + v5List.get(10) + v6List.get(10) + v7List.get(10)); // secondary
+ // queue
+ // size
+ }
+
+ @Test
public void testPartitionedRegionParallelPropagation_BeforeDispatch() throws Exception {
Integer lnPort = (Integer) vm0.invoke(() -> WANTestBase.createFirstLocatorWithDSId(1));
Integer nyPort = (Integer) vm1.invoke(() -> WANTestBase.createFirstRemoteLocator(2, lnPort));
diff --git a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderQueueDUnitTest.java b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderQueueDUnitTest.java
index 63c715b..ec3ce4e 100644
--- a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderQueueDUnitTest.java
+++ b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderQueueDUnitTest.java
@@ -18,6 +18,7 @@ import static org.apache.geode.distributed.ConfigurationProperties.*;
import static org.junit.Assert.*;
import java.io.File;
+import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Properties;
@@ -46,6 +47,7 @@ import org.apache.geode.distributed.internal.InternalDistributedSystem;
import org.apache.geode.internal.cache.RegionQueue;
import org.apache.geode.internal.cache.ha.ThreadIdentifier;
import org.apache.geode.internal.cache.wan.AbstractGatewaySender;
+import org.apache.geode.internal.cache.wan.GatewaySenderStats;
import org.apache.geode.internal.cache.wan.WANTestBase;
import org.apache.geode.test.dunit.IgnoredException;
import org.apache.geode.test.dunit.VM;
@@ -103,7 +105,13 @@ public class SerialGatewaySenderQueueDUnitTest extends WANTestBase {
vm4.invoke(() -> WANTestBase.pauseSender("ln"));
vm6.invoke(() -> WANTestBase.doPuts(getTestMethodName() + "_RR", 1000));
- Wait.pause(5000);
+ ArrayList<Integer> v4List =
+ (ArrayList<Integer>) vm4.invoke(() -> WANTestBase.getSenderStats("ln", 1000));
+ ArrayList<Integer> v5List =
+ (ArrayList<Integer>) vm5.invoke(() -> WANTestBase.getSenderStats("ln", 1000));
+ // secondary queue size stats in serial queue should be 0
+ assertEquals(0, v4List.get(10) + v5List.get(10));
+
HashMap primarySenderUpdates = (HashMap) vm4.invoke(() -> WANTestBase.checkQueue());
HashMap secondarySenderUpdates = (HashMap) vm5.invoke(() -> WANTestBase.checkQueue());
assertEquals(primarySenderUpdates, secondarySenderUpdates);
@@ -138,6 +146,9 @@ public class SerialGatewaySenderQueueDUnitTest extends WANTestBase {
// removing all the keys.
secondarySenderUpdates = (HashMap) vm5.invoke(() -> WANTestBase.checkQueue());
assertEquals(secondarySenderUpdates.get("Destroy"), receiverUpdates.get("Create"));
+
+ vm4.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ vm5.invoke(() -> WANTestBase.getSenderStats("ln", 0));
}
protected void checkPrimarySenderUpdatesOnVM5(HashMap primarySenderUpdates) {
diff --git a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/serial/SerialWANConflationDUnitTest.java b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/serial/SerialWANConflationDUnitTest.java
index 7297179..091befd 100644
--- a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/serial/SerialWANConflationDUnitTest.java
+++ b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/serial/SerialWANConflationDUnitTest.java
@@ -44,10 +44,10 @@ public class SerialWANConflationDUnitTest extends WANTestBase {
createCacheInVMs(lnPort, vm4, vm5, vm6, vm7);
- vm4.invoke(() -> createPartitionedRegion(getTestMethodName(), "ln", 0, 8, isOffHeap()));
- vm5.invoke(() -> createPartitionedRegion(getTestMethodName(), "ln", 0, 8, isOffHeap()));
- vm6.invoke(() -> createPartitionedRegion(getTestMethodName(), "ln", 0, 8, isOffHeap()));
- vm7.invoke(() -> createPartitionedRegion(getTestMethodName(), "ln", 0, 8, isOffHeap()));
+ vm4.invoke(() -> createPartitionedRegion(getTestMethodName(), "ln", 1, 8, isOffHeap()));
+ vm5.invoke(() -> createPartitionedRegion(getTestMethodName(), "ln", 1, 8, isOffHeap()));
+ vm6.invoke(() -> createPartitionedRegion(getTestMethodName(), "ln", 1, 8, isOffHeap()));
+ vm7.invoke(() -> createPartitionedRegion(getTestMethodName(), "ln", 1, 8, isOffHeap()));
vm4.invoke(() -> createSender("ln", 2, false, 100, 50, false, false, null, true));
vm5.invoke(() -> createSender("ln", 2, false, 100, 50, false, false, null, true));
@@ -92,6 +92,71 @@ public class SerialWANConflationDUnitTest extends WANTestBase {
assertTrue("No events conflated in batch",
(v4List.get(8) + v5List.get(8) + v6List.get(8) + v7List.get(8)) > 0);
+ }
+
+ @Test
+ public void testSerialPropagationPartitionRegionConflationDuringEnqueue() throws Exception {
+ Integer lnPort = (Integer) vm0.invoke(() -> createFirstLocatorWithDSId(1));
+ Integer nyPort = (Integer) vm1.invoke(() -> createFirstRemoteLocator(2, lnPort));
+
+ createCacheInVMs(nyPort, vm2, vm3);
+
+ vm2.invoke(() -> createPartitionedRegion(getTestMethodName(), null, 1, 8, isOffHeap()));
+ vm3.invoke(() -> createPartitionedRegion(getTestMethodName(), null, 1, 8, isOffHeap()));
+
+ createReceiverInVMs(vm2, vm3);
+
+ createCacheInVMs(lnPort, vm4, vm5, vm6, vm7);
+
+ vm4.invoke(() -> createPartitionedRegion(getTestMethodName(), "ln", 1, 8, isOffHeap()));
+ vm5.invoke(() -> createPartitionedRegion(getTestMethodName(), "ln", 1, 8, isOffHeap()));
+ vm6.invoke(() -> createPartitionedRegion(getTestMethodName(), "ln", 1, 8, isOffHeap()));
+ vm7.invoke(() -> createPartitionedRegion(getTestMethodName(), "ln", 1, 8, isOffHeap()));
+
+ vm4.invoke(() -> createSender("ln", 2, false, 100, 50, true, false, null, true));
+ vm5.invoke(() -> createSender("ln", 2, false, 100, 50, true, false, null, true));
+ vm6.invoke(() -> createSender("ln", 2, false, 100, 50, true, false, null, true));
+ vm7.invoke(() -> createSender("ln", 2, false, 100, 50, true, false, null, true));
+
+ startSenderInVMs("ln", vm4, vm5, vm6, vm7);
+
+ vm4.invoke(() -> pauseSender("ln"));
+ vm5.invoke(() -> pauseSender("ln"));
+ vm6.invoke(() -> pauseSender("ln"));
+ vm7.invoke(() -> pauseSender("ln"));
+
+
+ final Map keyValues = new HashMap();
+
+ for (int i = 1; i <= 10; i++) {
+ for (int j = 1; j <= 10; j++) {
+ keyValues.put(j, i);
+ }
+ vm4.invoke(() -> putGivenKeyValue(getTestMethodName(), keyValues));
+ }
+
+ ArrayList<Integer> v4List =
+ (ArrayList<Integer>) vm4.invoke(() -> WANTestBase.getSenderStats("ln", 20));
+ assertTrue("After conflation during enqueue, there should be only 20 events",
+ v4List.get(0) == 20);
+
+ vm4.invoke(() -> resumeSender("ln"));
+ vm5.invoke(() -> resumeSender("ln"));
+ vm6.invoke(() -> resumeSender("ln"));
+ vm7.invoke(() -> resumeSender("ln"));
+
+ v4List = (ArrayList<Integer>) vm4.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ ArrayList<Integer> v5List =
+ (ArrayList<Integer>) vm5.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ ArrayList<Integer> v6List =
+ (ArrayList<Integer>) vm6.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+ ArrayList<Integer> v7List =
+ (ArrayList<Integer>) vm7.invoke(() -> WANTestBase.getSenderStats("ln", 0));
+
+ assertTrue("No events in secondary queue stats since it's serial sender",
+ (v4List.get(10) + v5List.get(10) + v6List.get(10) + v7List.get(10)) == 0);
+ assertTrue("Total queued events should be 100",
+ (v4List.get(2) + v5List.get(2) + v6List.get(2) + v7List.get(2)) == 100);
vm2.invoke(() -> validateRegionSize(getTestMethodName(), 10));
diff --git a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/serial/SerialWANPropagationDUnitTest.java b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/serial/SerialWANPropagationDUnitTest.java
index e84fd89..87c90e0 100644
--- a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/serial/SerialWANPropagationDUnitTest.java
+++ b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/serial/SerialWANPropagationDUnitTest.java
@@ -163,6 +163,7 @@ public class SerialWANPropagationDUnitTest extends WANTestBase {
IgnoredException.addIgnoredException(BatchException70.class.getName());
IgnoredException.addIgnoredException(ServerOperationException.class.getName());
IgnoredException.addIgnoredException(IOException.class.getName());
+ IgnoredException.addIgnoredException(java.net.SocketException.class.getName());
vm4.invoke(() -> WANTestBase.doPuts(getTestMethodName() + "_RR", 10000));
--
To stop receiving notification emails like this one, please contact
zhouxj@apache.org.