You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Josh Elser <jo...@gmail.com> on 2014/11/19 05:44:26 UTC
Review Request 28214: Decouple MiniAccumuloCluster from integration
tests and introduce StandaloneAccumuloCluster
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/
-----------------------------------------------------------
Review request for accumulo.
Bugs: ACCUMULO-3167
https://issues.apache.org/jira/browse/ACCUMULO-3167
Repository: accumulo
Description
-------
We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
Diffs
-----
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
test/src/test/java/org/apache/accumulo/test/functional/BulkFileIT.java a7cf6bd
test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
Diff: https://reviews.apache.org/r/28214/diff/
Testing
-------
Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
Thanks,
Josh Elser
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
> On Nov. 19, 2014, 5 p.m., Josh Elser wrote:
> > test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java, line 77
> > <https://reviews.apache.org/r/28214/diff/3/?file=769778#file769778line77>
> >
> > This turns out to be a really common pattern that cropped up across the test code. Providing some mechanism to automatically do this in AccumuloClusterIT would be nice.
Will do in follow-on.
> On Nov. 19, 2014, 5 p.m., Josh Elser wrote:
> > test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java, line 32
> > <https://reviews.apache.org/r/28214/diff/3/?file=769775#file769775line32>
> >
> > Will need to add some more pieces here to run against an instance with SSL enabled.
Will do in follow-on.
> On Nov. 19, 2014, 5 p.m., Josh Elser wrote:
> > test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java, line 197
> > <https://reviews.apache.org/r/28214/diff/3/?file=769767#file769767line197>
> >
> > I wasn't really sure what should be done here. It helps keep the code more backwards compatible but I don't like adding implementation specifics to this class.
> >
> > It might be possible to push this down to the AccumuloCluster implementation. The worry for a standalone instance is that we'd want it returned to it's original state after the test. To do that, we'd also have to add a teardown hook to the AccumuloCluster implementation that this class could invoke.
Can address if better approach comes up later.
> On Nov. 19, 2014, 5 p.m., Josh Elser wrote:
> > minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java, line 290
> > <https://reviews.apache.org/r/28214/diff/3/?file=769757#file769757line290>
> >
> > This is rather brittle because it expects the tests to be running on a properly configured host for the instance that you're connecting to. Acceptable for a first pass?
Will address when it becomes an issue.
> On Nov. 19, 2014, 5 p.m., Josh Elser wrote:
> > minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java, line 46
> > <https://reviews.apache.org/r/28214/diff/3/?file=769752#file769752line46>
> >
> > Are the variants that accept a hostname even worthwhile? The caller would have to know where things are running which is hard to get at presently.
Can remove later if desired, not intended for client use anyways.
> On Nov. 19, 2014, 5 p.m., Josh Elser wrote:
> > minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java, line 31
> > <https://reviews.apache.org/r/28214/diff/3/?file=769752#file769752line31>
> >
> > Not sure if it's better to accept `String[] args` or `String... args`. MiniAccumuloClusterImpl's `exec` method did accept the varargs variant.
6 of one, half-dozen of another.
> On Nov. 19, 2014, 5 p.m., Josh Elser wrote:
> > test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java, line 122
> > <https://reviews.apache.org/r/28214/diff/3/?file=769843#file769843line122>
> >
> > This is a big downside of implementing ClusterControl how I did. It's easy to just run something and get the exit code, but it's difficult to spawn a background process and wait for it to return.
> >
> > Is it worth it to provide both of these methods?
I haven't hit a test that isn't satisfied by getting the return code and stdout or doing the above. Most aren't really forking off anyways.
- Josh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62156
-----------------------------------------------------------
On Nov. 21, 2014, 8:40 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 21, 2014, 8:40 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 8406570
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java bc9ab4a
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62156
-----------------------------------------------------------
minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java
<https://reviews.apache.org/r/28214/#comment104114>
Not sure if it's better to accept `String[] args` or `String... args`. MiniAccumuloClusterImpl's `exec` method did accept the varargs variant.
minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java
<https://reviews.apache.org/r/28214/#comment104116>
Are the variants that accept a hostname even worthwhile? The caller would have to know where things are running which is hard to get at presently.
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java
<https://reviews.apache.org/r/28214/#comment104115>
I think this would be sufficient to allow SSH to work in most environments. You can provide a specific key (any options) and the user to ssh as. Not sure if I'm missing anything else?
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java
<https://reviews.apache.org/r/28214/#comment104117>
This is rather brittle because it expects the tests to be running on a properly configured host for the instance that you're connecting to. Acceptable for a first pass?
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java
<https://reviews.apache.org/r/28214/#comment104118>
Oops.
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java
<https://reviews.apache.org/r/28214/#comment104119>
StandaloneAccumuloCluster obviates the need for the existingInstance support inside MAC. Don't need to kill it in the initial code, though.
minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java
<https://reviews.apache.org/r/28214/#comment104120>
Was an incorrectly named method.
test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java
<https://reviews.apache.org/r/28214/#comment104121>
Let's us pass down a specific prefix for the tables that are created which allow us to automatically clean up those tables in Before/After methods.
test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java
<https://reviews.apache.org/r/28214/#comment104122>
This is really gross and not properly implemented. I think to make this correctly work, we'd have to get at core-site.xml and/or hdfs-site.xml. We might be able to guess at this from Accumulo configuration instead of requiring it to be explicitly passed in.
Without the ability to get the filesystem for the cluster, we can't port over tests that exercise functionality like bulk import or import/export table.
test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java
<https://reviews.apache.org/r/28214/#comment104123>
I wasn't really sure what should be done here. It helps keep the code more backwards compatible but I don't like adding implementation specifics to this class.
It might be possible to push this down to the AccumuloCluster implementation. The worry for a standalone instance is that we'd want it returned to it's original state after the test. To do that, we'd also have to add a teardown hook to the AccumuloCluster implementation that this class could invoke.
test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java
<https://reviews.apache.org/r/28214/#comment104124>
Because the Before methods of the parent class are run before the children class, even if a test did not want to run against a MiniCluster, the MiniCluster would still be started before it could make any JUnit Assume call.
Providing a method that the implementation can override ensures that we don't waste time starting some cluster that we don't want to use.
test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java
<https://reviews.apache.org/r/28214/#comment104125>
Whitespace
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java
<https://reviews.apache.org/r/28214/#comment104126>
Will need to add some more pieces here to run against an instance with SSL enabled.
test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java
<https://reviews.apache.org/r/28214/#comment104127>
This turns out to be a really common pattern that cropped up across the test code. Providing some mechanism to automatically do this in AccumuloClusterIT would be nice.
test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java
<https://reviews.apache.org/r/28214/#comment104128>
Maybe this one wouldn't be so bad to run against a standalone cluster? I think I initially shyed away from it because of all the metadata and root table mucking.
test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java
<https://reviews.apache.org/r/28214/#comment104129>
I think AbstractMacIT should just inherit these methods from AccumuloIT for consistency.
test/src/test/java/org/apache/accumulo/test/functional/BulkFileIT.java
<https://reviews.apache.org/r/28214/#comment104134>
These changes need to be reverted. Had tried to port this one and ultimately gave up.
test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java
<https://reviews.apache.org/r/28214/#comment104135>
This is a big downside of implementing ClusterControl how I did. It's easy to just run something and get the exit code, but it's difficult to spawn a background process and wait for it to return.
Is it worth it to provide both of these methods?
test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java
<https://reviews.apache.org/r/28214/#comment104136>
Providing methods that accept a ServerType and check/wait for the ZooLocks in AccumuloClusterIT might be generally useful.
test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java
<https://reviews.apache.org/r/28214/#comment104137>
Outdated comment.
self-review...
- Josh Elser
On Nov. 19, 2014, 4:02 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 19, 2014, 4:02 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkFileIT.java a7cf6bd
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/
-----------------------------------------------------------
(Updated Nov. 24, 2014, 11:23 p.m.)
Review request for accumulo.
Changes
-------
Updated to what I believe will be the final changeset here. Fixing little things here and there are just adding up way too much -- things are pretty stable as long as I don't keep mucking with new tests. Running a suite of tests to make sure I didn't break anything new in the default `mvn verify`. Will definitely add some documentation before committing.
Bugs: ACCUMULO-3167
https://issues.apache.org/jira/browse/ACCUMULO-3167
Repository: accumulo
Description
-------
We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
Diffs (updated)
-----
minicluster/pom.xml 5ad531f
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/StandaloneAccumuloClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
test/src/test/java/org/apache/accumulo/test/BulkImportVolumeIT.java 79a2513
test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 8570858
test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
test/src/test/java/org/apache/accumulo/test/functional/BulkFileIT.java a7cf6bd
test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
test/src/test/java/org/apache/accumulo/test/functional/BulkSplitOptimizationIT.java 492e36f
test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
test/src/test/java/org/apache/accumulo/test/functional/CleanTmpIT.java 676f6d7
test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
test/src/test/java/org/apache/accumulo/test/functional/CompactionIT.java b659913
test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
test/src/test/java/org/apache/accumulo/test/functional/ExamplesIT.java 210e057
test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java bc9ab4a
test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
Diff: https://reviews.apache.org/r/28214/diff/
Testing
-------
Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
Thanks,
Josh Elser
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/
-----------------------------------------------------------
(Updated Nov. 21, 2014, 8:40 p.m.)
Review request for accumulo.
Changes
-------
Got some more problematic tests running. I think I'm close to 100% coverage now. Fixed up a number of issues myself and others have found.
Bugs: ACCUMULO-3167
https://issues.apache.org/jira/browse/ACCUMULO-3167
Repository: accumulo
Description
-------
We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
Diffs (updated)
-----
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 8406570
test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java bc9ab4a
test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
Diff: https://reviews.apache.org/r/28214/diff/
Testing
-------
Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
Thanks,
Josh Elser
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
> On Nov. 21, 2014, 3:19 a.m., kturner wrote:
> > test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java, line 98
> > <https://reviews.apache.org/r/28214/diff/5/?file=770707#file770707line98>
> >
> > SimpleMacIT [starts one MAC][1] per class. It seems like AccumuloClusterIT starts a MAC per method?
> >
> > [1]:https://github.com/apache/accumulo/blob/1.6.1/test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java#L60
Yes, this is a change. I made SharedMiniClusterIT to make ShellServerIT run a *lot* faster, I didn't see any other ITs that were affected to the same extent. I couldn't come up with a good way to implement this inside of AccumuloClusterIT so I punted on it for now. Suggestions welcome.
> On Nov. 21, 2014, 3:19 a.m., kturner wrote:
> > test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java, line 105
> > <https://reviews.apache.org/r/28214/diff/5/?file=770695#file770695line105>
> >
> > constant?
I had copy-pasted these from AbstractMacIT -- will lift to some constants. It would be nice to have some sole reference for properties that devs can provide to control build options.. TBD I think.
- Josh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62514
-----------------------------------------------------------
On Nov. 20, 2014, 3:27 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 20, 2014, 3:27 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by ke...@deenlo.com.
> On Nov. 21, 2014, 3:19 a.m., kturner wrote:
> > test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java, line 98
> > <https://reviews.apache.org/r/28214/diff/5/?file=770707#file770707line98>
> >
> > SimpleMacIT [starts one MAC][1] per class. It seems like AccumuloClusterIT starts a MAC per method?
> >
> > [1]:https://github.com/apache/accumulo/blob/1.6.1/test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java#L60
>
> Josh Elser wrote:
> Yes, this is a change. I made SharedMiniClusterIT to make ShellServerIT run a *lot* faster, I didn't see any other ITs that were affected to the same extent. I couldn't come up with a good way to implement this inside of AccumuloClusterIT so I punted on it for now. Suggestions welcome.
Ok. I saw SharedMiniClusterIT and was wondering why only ShellServerIT used it. Do you know what the difference in build times is w/ this change?
- kturner
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62514
-----------------------------------------------------------
On Nov. 21, 2014, 8:40 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 21, 2014, 8:40 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 8406570
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java bc9ab4a
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
> On Nov. 21, 2014, 3:19 a.m., kturner wrote:
> > test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java, line 98
> > <https://reviews.apache.org/r/28214/diff/5/?file=770707#file770707line98>
> >
> > SimpleMacIT [starts one MAC][1] per class. It seems like AccumuloClusterIT starts a MAC per method?
> >
> > [1]:https://github.com/apache/accumulo/blob/1.6.1/test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java#L60
>
> Josh Elser wrote:
> Yes, this is a change. I made SharedMiniClusterIT to make ShellServerIT run a *lot* faster, I didn't see any other ITs that were affected to the same extent. I couldn't come up with a good way to implement this inside of AccumuloClusterIT so I punted on it for now. Suggestions welcome.
>
> kturner wrote:
> Ok. I saw SharedMiniClusterIT and was wondering why only ShellServerIT used it. Do you know what the difference in build times is w/ this change?
>
> Josh Elser wrote:
> Ha, significant. Up to 230s instead of 30 when running with MAC :). Will see if I can come up with anything to address it.
>
> kturner wrote:
> > Ha, significant. Up to 230s instead of 30 when running with MAC :).
>
> Thats for an individual test. Do you know what the total diff for `time mvn verify` is w/ and w/o your changes?
>
> Josh Elser wrote:
> Oh, sorry, I thought you were only asking specifically about ConditionalWriterIT :) (with a standalone instance, it's 25s if you were curious :P). I'll run some comparisions against 1.6 presently, 1.6 with these changes out of the box (still using MAC) and then 1.6 with a standalone instance available tonight.
>
> Josh Elser wrote:
> Oddly, NamespacesIT has also been really problematic with ZK starting. My guess is that it this change is also affecting things for that test. Definitely something I need to look into until we resolve the ZK issue.
>
> kturner wrote:
> > with a standalone instance, it's 25s if you were curious
>
> Do you think we could start a Mini instance preintegration test and use that as a "standalone instance", if no standalone instance is configured?
> Do you think we could start a Mini instance preintegration test and use that as a "standalone instance", if no standalone instance is configured?
We would have to make some sort of new cluster to support this. The start/stop semantics (using start-server.sh) would not work to use a normal StandaloneCluster with MAC. This wouldn't be very difficult though.
> Do you know what the total diff for time mvn verify is w/ and w/o your changes?
Summary of baseline (before these changes), running against MAC with changes, and then using a Standalone single-node cluster (falling back to MAC when the test doesn't support the standalone test): http://people.apache.org/~elserj/accumulo/ACCUMULO-3167-comparison.html. The only thing I did special to the configuration of the StandaloneCluster was setting instance.zookeeper.timeout=15s instead of the default 30s.
- Josh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62514
-----------------------------------------------------------
On Nov. 21, 2014, 8:40 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 21, 2014, 8:40 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 8406570
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java bc9ab4a
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
> On Nov. 21, 2014, 3:19 a.m., kturner wrote:
> > test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java, line 98
> > <https://reviews.apache.org/r/28214/diff/5/?file=770707#file770707line98>
> >
> > SimpleMacIT [starts one MAC][1] per class. It seems like AccumuloClusterIT starts a MAC per method?
> >
> > [1]:https://github.com/apache/accumulo/blob/1.6.1/test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java#L60
>
> Josh Elser wrote:
> Yes, this is a change. I made SharedMiniClusterIT to make ShellServerIT run a *lot* faster, I didn't see any other ITs that were affected to the same extent. I couldn't come up with a good way to implement this inside of AccumuloClusterIT so I punted on it for now. Suggestions welcome.
>
> kturner wrote:
> Ok. I saw SharedMiniClusterIT and was wondering why only ShellServerIT used it. Do you know what the difference in build times is w/ this change?
>
> Josh Elser wrote:
> Ha, significant. Up to 230s instead of 30 when running with MAC :). Will see if I can come up with anything to address it.
>
> kturner wrote:
> > Ha, significant. Up to 230s instead of 30 when running with MAC :).
>
> Thats for an individual test. Do you know what the total diff for `time mvn verify` is w/ and w/o your changes?
>
> Josh Elser wrote:
> Oh, sorry, I thought you were only asking specifically about ConditionalWriterIT :) (with a standalone instance, it's 25s if you were curious :P). I'll run some comparisions against 1.6 presently, 1.6 with these changes out of the box (still using MAC) and then 1.6 with a standalone instance available tonight.
>
> Josh Elser wrote:
> Oddly, NamespacesIT has also been really problematic with ZK starting. My guess is that it this change is also affecting things for that test. Definitely something I need to look into until we resolve the ZK issue.
>
> kturner wrote:
> > with a standalone instance, it's 25s if you were curious
>
> Do you think we could start a Mini instance preintegration test and use that as a "standalone instance", if no standalone instance is configured?
>
> Josh Elser wrote:
> > Do you think we could start a Mini instance preintegration test and use that as a "standalone instance", if no standalone instance is configured?
>
> We would have to make some sort of new cluster to support this. The start/stop semantics (using start-server.sh) would not work to use a normal StandaloneCluster with MAC. This wouldn't be very difficult though.
>
> > Do you know what the total diff for time mvn verify is w/ and w/o your changes?
>
> Summary of baseline (before these changes), running against MAC with changes, and then using a Standalone single-node cluster (falling back to MAC when the test doesn't support the standalone test): http://people.apache.org/~elserj/accumulo/ACCUMULO-3167-comparison.html. The only thing I did special to the configuration of the StandaloneCluster was setting instance.zookeeper.timeout=15s instead of the default 30s.
>
> kturner wrote:
> Nice report. So w/ these changes `mvn verify` took ~11 more mins, and increase of ~11%. These are a nice set of changes, IMO nothing needs to be done about the 11% increase.
Thanks, Keith. I think I might be able to do something which makes those faster, but my plan is to try that after this behemoth gets committed :)
- Josh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62514
-----------------------------------------------------------
On Nov. 21, 2014, 8:40 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 21, 2014, 8:40 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 8406570
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java bc9ab4a
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
> On Nov. 21, 2014, 3:19 a.m., kturner wrote:
> > test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java, line 98
> > <https://reviews.apache.org/r/28214/diff/5/?file=770707#file770707line98>
> >
> > SimpleMacIT [starts one MAC][1] per class. It seems like AccumuloClusterIT starts a MAC per method?
> >
> > [1]:https://github.com/apache/accumulo/blob/1.6.1/test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java#L60
>
> Josh Elser wrote:
> Yes, this is a change. I made SharedMiniClusterIT to make ShellServerIT run a *lot* faster, I didn't see any other ITs that were affected to the same extent. I couldn't come up with a good way to implement this inside of AccumuloClusterIT so I punted on it for now. Suggestions welcome.
>
> kturner wrote:
> Ok. I saw SharedMiniClusterIT and was wondering why only ShellServerIT used it. Do you know what the difference in build times is w/ this change?
>
> Josh Elser wrote:
> Ha, significant. Up to 230s instead of 30 when running with MAC :). Will see if I can come up with anything to address it.
>
> kturner wrote:
> > Ha, significant. Up to 230s instead of 30 when running with MAC :).
>
> Thats for an individual test. Do you know what the total diff for `time mvn verify` is w/ and w/o your changes?
>
> Josh Elser wrote:
> Oh, sorry, I thought you were only asking specifically about ConditionalWriterIT :) (with a standalone instance, it's 25s if you were curious :P). I'll run some comparisions against 1.6 presently, 1.6 with these changes out of the box (still using MAC) and then 1.6 with a standalone instance available tonight.
Oddly, NamespacesIT has also been really problematic with ZK starting. My guess is that it this change is also affecting things for that test. Definitely something I need to look into until we resolve the ZK issue.
- Josh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62514
-----------------------------------------------------------
On Nov. 21, 2014, 8:40 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 21, 2014, 8:40 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 8406570
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java bc9ab4a
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by ke...@deenlo.com.
> On Nov. 21, 2014, 3:19 a.m., kturner wrote:
> > test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java, line 98
> > <https://reviews.apache.org/r/28214/diff/5/?file=770707#file770707line98>
> >
> > SimpleMacIT [starts one MAC][1] per class. It seems like AccumuloClusterIT starts a MAC per method?
> >
> > [1]:https://github.com/apache/accumulo/blob/1.6.1/test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java#L60
>
> Josh Elser wrote:
> Yes, this is a change. I made SharedMiniClusterIT to make ShellServerIT run a *lot* faster, I didn't see any other ITs that were affected to the same extent. I couldn't come up with a good way to implement this inside of AccumuloClusterIT so I punted on it for now. Suggestions welcome.
>
> kturner wrote:
> Ok. I saw SharedMiniClusterIT and was wondering why only ShellServerIT used it. Do you know what the difference in build times is w/ this change?
>
> Josh Elser wrote:
> Ha, significant. Up to 230s instead of 30 when running with MAC :). Will see if I can come up with anything to address it.
>
> kturner wrote:
> > Ha, significant. Up to 230s instead of 30 when running with MAC :).
>
> Thats for an individual test. Do you know what the total diff for `time mvn verify` is w/ and w/o your changes?
>
> Josh Elser wrote:
> Oh, sorry, I thought you were only asking specifically about ConditionalWriterIT :) (with a standalone instance, it's 25s if you were curious :P). I'll run some comparisions against 1.6 presently, 1.6 with these changes out of the box (still using MAC) and then 1.6 with a standalone instance available tonight.
>
> Josh Elser wrote:
> Oddly, NamespacesIT has also been really problematic with ZK starting. My guess is that it this change is also affecting things for that test. Definitely something I need to look into until we resolve the ZK issue.
>
> kturner wrote:
> > with a standalone instance, it's 25s if you were curious
>
> Do you think we could start a Mini instance preintegration test and use that as a "standalone instance", if no standalone instance is configured?
>
> Josh Elser wrote:
> > Do you think we could start a Mini instance preintegration test and use that as a "standalone instance", if no standalone instance is configured?
>
> We would have to make some sort of new cluster to support this. The start/stop semantics (using start-server.sh) would not work to use a normal StandaloneCluster with MAC. This wouldn't be very difficult though.
>
> > Do you know what the total diff for time mvn verify is w/ and w/o your changes?
>
> Summary of baseline (before these changes), running against MAC with changes, and then using a Standalone single-node cluster (falling back to MAC when the test doesn't support the standalone test): http://people.apache.org/~elserj/accumulo/ACCUMULO-3167-comparison.html. The only thing I did special to the configuration of the StandaloneCluster was setting instance.zookeeper.timeout=15s instead of the default 30s.
Nice report. So w/ these changes `mvn verify` took ~11 more mins, and increase of ~11%. These are a nice set of changes, IMO nothing needs to be done about the 11% increase.
- kturner
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62514
-----------------------------------------------------------
On Nov. 21, 2014, 8:40 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 21, 2014, 8:40 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 8406570
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java bc9ab4a
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by ke...@deenlo.com.
> On Nov. 21, 2014, 3:19 a.m., kturner wrote:
> > test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java, line 98
> > <https://reviews.apache.org/r/28214/diff/5/?file=770707#file770707line98>
> >
> > SimpleMacIT [starts one MAC][1] per class. It seems like AccumuloClusterIT starts a MAC per method?
> >
> > [1]:https://github.com/apache/accumulo/blob/1.6.1/test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java#L60
>
> Josh Elser wrote:
> Yes, this is a change. I made SharedMiniClusterIT to make ShellServerIT run a *lot* faster, I didn't see any other ITs that were affected to the same extent. I couldn't come up with a good way to implement this inside of AccumuloClusterIT so I punted on it for now. Suggestions welcome.
>
> kturner wrote:
> Ok. I saw SharedMiniClusterIT and was wondering why only ShellServerIT used it. Do you know what the difference in build times is w/ this change?
>
> Josh Elser wrote:
> Ha, significant. Up to 230s instead of 30 when running with MAC :). Will see if I can come up with anything to address it.
> Ha, significant. Up to 230s instead of 30 when running with MAC :).
Thats for an individual test. Do you know what the total diff for `time mvn verify` is w/ and w/o your changes?
- kturner
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62514
-----------------------------------------------------------
On Nov. 21, 2014, 8:40 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 21, 2014, 8:40 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 8406570
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java bc9ab4a
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
> On Nov. 21, 2014, 3:19 a.m., kturner wrote:
> > test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java, line 98
> > <https://reviews.apache.org/r/28214/diff/5/?file=770707#file770707line98>
> >
> > SimpleMacIT [starts one MAC][1] per class. It seems like AccumuloClusterIT starts a MAC per method?
> >
> > [1]:https://github.com/apache/accumulo/blob/1.6.1/test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java#L60
>
> Josh Elser wrote:
> Yes, this is a change. I made SharedMiniClusterIT to make ShellServerIT run a *lot* faster, I didn't see any other ITs that were affected to the same extent. I couldn't come up with a good way to implement this inside of AccumuloClusterIT so I punted on it for now. Suggestions welcome.
>
> kturner wrote:
> Ok. I saw SharedMiniClusterIT and was wondering why only ShellServerIT used it. Do you know what the difference in build times is w/ this change?
>
> Josh Elser wrote:
> Ha, significant. Up to 230s instead of 30 when running with MAC :). Will see if I can come up with anything to address it.
>
> kturner wrote:
> > Ha, significant. Up to 230s instead of 30 when running with MAC :).
>
> Thats for an individual test. Do you know what the total diff for `time mvn verify` is w/ and w/o your changes?
Oh, sorry, I thought you were only asking specifically about ConditionalWriterIT :) (with a standalone instance, it's 25s if you were curious :P). I'll run some comparisions against 1.6 presently, 1.6 with these changes out of the box (still using MAC) and then 1.6 with a standalone instance available tonight.
- Josh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62514
-----------------------------------------------------------
On Nov. 21, 2014, 8:40 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 21, 2014, 8:40 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 8406570
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java bc9ab4a
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
> On Nov. 21, 2014, 3:19 a.m., kturner wrote:
> > test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java, line 98
> > <https://reviews.apache.org/r/28214/diff/5/?file=770707#file770707line98>
> >
> > SimpleMacIT [starts one MAC][1] per class. It seems like AccumuloClusterIT starts a MAC per method?
> >
> > [1]:https://github.com/apache/accumulo/blob/1.6.1/test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java#L60
>
> Josh Elser wrote:
> Yes, this is a change. I made SharedMiniClusterIT to make ShellServerIT run a *lot* faster, I didn't see any other ITs that were affected to the same extent. I couldn't come up with a good way to implement this inside of AccumuloClusterIT so I punted on it for now. Suggestions welcome.
>
> kturner wrote:
> Ok. I saw SharedMiniClusterIT and was wondering why only ShellServerIT used it. Do you know what the difference in build times is w/ this change?
Ha, significant. Up to 230s instead of 30 when running with MAC :). Will see if I can come up with anything to address it.
- Josh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62514
-----------------------------------------------------------
On Nov. 21, 2014, 8:40 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 21, 2014, 8:40 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 8406570
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java bc9ab4a
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by ke...@deenlo.com.
> On Nov. 21, 2014, 3:19 a.m., kturner wrote:
> > test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java, line 98
> > <https://reviews.apache.org/r/28214/diff/5/?file=770707#file770707line98>
> >
> > SimpleMacIT [starts one MAC][1] per class. It seems like AccumuloClusterIT starts a MAC per method?
> >
> > [1]:https://github.com/apache/accumulo/blob/1.6.1/test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java#L60
>
> Josh Elser wrote:
> Yes, this is a change. I made SharedMiniClusterIT to make ShellServerIT run a *lot* faster, I didn't see any other ITs that were affected to the same extent. I couldn't come up with a good way to implement this inside of AccumuloClusterIT so I punted on it for now. Suggestions welcome.
>
> kturner wrote:
> Ok. I saw SharedMiniClusterIT and was wondering why only ShellServerIT used it. Do you know what the difference in build times is w/ this change?
>
> Josh Elser wrote:
> Ha, significant. Up to 230s instead of 30 when running with MAC :). Will see if I can come up with anything to address it.
>
> kturner wrote:
> > Ha, significant. Up to 230s instead of 30 when running with MAC :).
>
> Thats for an individual test. Do you know what the total diff for `time mvn verify` is w/ and w/o your changes?
>
> Josh Elser wrote:
> Oh, sorry, I thought you were only asking specifically about ConditionalWriterIT :) (with a standalone instance, it's 25s if you were curious :P). I'll run some comparisions against 1.6 presently, 1.6 with these changes out of the box (still using MAC) and then 1.6 with a standalone instance available tonight.
>
> Josh Elser wrote:
> Oddly, NamespacesIT has also been really problematic with ZK starting. My guess is that it this change is also affecting things for that test. Definitely something I need to look into until we resolve the ZK issue.
> with a standalone instance, it's 25s if you were curious
Do you think we could start a Mini instance preintegration test and use that as a "standalone instance", if no standalone instance is configured?
- kturner
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62514
-----------------------------------------------------------
On Nov. 21, 2014, 8:40 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 21, 2014, 8:40 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 8406570
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java bc9ab4a
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by ke...@deenlo.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62514
-----------------------------------------------------------
minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java
<https://reviews.apache.org/r/28214/#comment104543>
was thinking these classes could go in the test module
test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java
<https://reviews.apache.org/r/28214/#comment104542>
constant?
test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java
<https://reviews.apache.org/r/28214/#comment104554>
SimpleMacIT [starts one MAC][1] per class. It seems like AccumuloClusterIT starts a MAC per method?
[1]:https://github.com/apache/accumulo/blob/1.6.1/test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java#L60
- kturner
On Nov. 20, 2014, 3:27 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 20, 2014, 3:27 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
> On Nov. 21, 2014, 2 a.m., kturner wrote:
> > minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java, line 118
> > <https://reviews.apache.org/r/28214/diff/5/?file=770681#file770681line118>
> >
> > The scripts may fall back to the masters files if the gc file is empty.
Good point! Forgot about that.
> On Nov. 21, 2014, 2 a.m., kturner wrote:
> > minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java, line 34
> > <https://reviews.apache.org/r/28214/diff/5/?file=770677#file770677line34>
> >
> > Took a quick look at ShellCommandExecutor, its marked unstable and private to hadoop using annotations.
Ah, I didn't take a look at that. RemoteShell was lifted from HBase, actually (at least the general premise and some code).
> On Nov. 21, 2014, 2 a.m., kturner wrote:
> > minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java, line 108
> > <https://reviews.apache.org/r/28214/diff/5/?file=770680#file770680line108>
> >
> > why not use control.startAll
Agreed. Will reduce code.
> On Nov. 21, 2014, 2 a.m., kturner wrote:
> > minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java, line 108
> > <https://reviews.apache.org/r/28214/diff/5/?file=770681#file770681line108>
> >
> > constants for these file names?
Will add.
> On Nov. 21, 2014, 2 a.m., kturner wrote:
> > minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java, line 61
> > <https://reviews.apache.org/r/28214/diff/5/?file=770676#file770676line61>
> >
> > Seems this method and suspend and resume are not called.
Yeah, there's only one place in the code which actually uses SIGSTOP/SIGCONT and it does it by hand. I think I added these methods initially because HBase had them. Not sure if it's worth leaving them in unused for now.
- Josh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62476
-----------------------------------------------------------
On Nov. 20, 2014, 3:27 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 20, 2014, 3:27 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
> On Nov. 21, 2014, 2 a.m., kturner wrote:
> > minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java, line 34
> > <https://reviews.apache.org/r/28214/diff/5/?file=770677#file770677line34>
> >
> > Took a quick look at ShellCommandExecutor, its marked unstable and private to hadoop using annotations.
>
> Josh Elser wrote:
> Ah, I didn't take a look at that. RemoteShell was lifted from HBase, actually (at least the general premise and some code).
Since this is internal to the user writing the test, I think I'm ok for the time being. If this turns out to be a major pain point, we can fork and make our own copy of ShellCommandExecutor.
- Josh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62476
-----------------------------------------------------------
On Nov. 21, 2014, 8:40 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 21, 2014, 8:40 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 8406570
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java bc9ab4a
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by ke...@deenlo.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62476
-----------------------------------------------------------
minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java
<https://reviews.apache.org/r/28214/#comment104516>
Seems this method and suspend and resume are not called.
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java
<https://reviews.apache.org/r/28214/#comment104517>
Took a quick look at ShellCommandExecutor, its marked unstable and private to hadoop using annotations.
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java
<https://reviews.apache.org/r/28214/#comment104527>
why not use control.startAll
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java
<https://reviews.apache.org/r/28214/#comment104524>
constants for these file names?
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java
<https://reviews.apache.org/r/28214/#comment104525>
The scripts may fall back to the masters files if the gc file is empty.
- kturner
On Nov. 20, 2014, 3:27 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 20, 2014, 3:27 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62570
-----------------------------------------------------------
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java
<https://reviews.apache.org/r/28214/#comment104633>
Need to check return code here and for other methods (used to throw an exception on error)
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java
<https://reviews.apache.org/r/28214/#comment104634>
accumuloHome might be a symlink to some other directory (for me, I just '/usr/local/lib/accumulo' but the actual directory is '/usr/local/lib64/accumulo-1.6.2-SNAPSHOT'). Need to ensure the grep is finding what's expected or use something else in the process arg list.
- Josh Elser
On Nov. 20, 2014, 3:27 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 20, 2014, 3:27 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
> If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/
-----------------------------------------------------------
(Updated Nov. 20, 2014, 3:27 p.m.)
Review request for accumulo.
Changes
-------
Forgot to actually document how it can be tested.
Bugs: ACCUMULO-3167
https://issues.apache.org/jira/browse/ACCUMULO-3167
Repository: accumulo
Description
-------
We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
Diffs
-----
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
Diff: https://reviews.apache.org/r/28214/diff/
Testing (updated)
-------
Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
If you want to try it out, you configure it using properties on the maven command line. For example, -Daccumulo.it.cluster.type=STANDALONE -Daccumulo.it.cluster.standalone.principal=root -Daccumulo.it.cluster.standalone.password=password -Daccumulo.it.cluster.standalone.instance.name=accumulo -Daccumulo.it.cluster.standalone.zookeepers=localhost. This would use a "real" instance with the expected connection information. Alternatively, you can make a properties file on the local filesystem and specify that file directly using -Daccumulo.it.properties=/path/to/file.properties. Properties on the command line will override those specified in the property if you provide the same in more than one place.
Thanks,
Josh Elser
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/
-----------------------------------------------------------
(Updated Nov. 20, 2014, 12:46 a.m.)
Review request for accumulo.
Changes
-------
Implements Keith's recommendations. Fixed Accumulo3047IT and InterruptibleScannersIT.
Bugs: ACCUMULO-3167
https://issues.apache.org/jira/browse/ACCUMULO-3167
Repository: accumulo
Description
-------
We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
Diffs (updated)
-----
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 55548e3
test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
Diff: https://reviews.apache.org/r/28214/diff/
Testing
-------
Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
Thanks,
Josh Elser
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
> On Nov. 19, 2014, 9:21 p.m., kturner wrote:
> > test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java, line 73
> > <https://reviews.apache.org/r/28214/diff/4/?file=769887#file769887line73>
> >
> > Could you do `Assume.assumeTrue(canRunTest(type))` ?
Yup, that would be more concise.
> On Nov. 19, 2014, 9:21 p.m., kturner wrote:
> > test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java, line 149
> > <https://reviews.apache.org/r/28214/diff/4/?file=769887#file769887line149>
> >
> > the enum could have an isDynamic method.
Ah, that would help. I used to have an additional level of Cluster hierarchy (Managed and Unmanaged, instances that can be "controlled" and those that cannot) which was ultimately removed when I added in the ClusterControl interface. Pushing this into the enum makes sense.
- Josh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62230
-----------------------------------------------------------
On Nov. 19, 2014, 5:01 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 19, 2014, 5:01 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
On Nov. 19, 2014, 9:21 p.m., Josh Elser wrote:
> > Still looking at this, just publishing a few things I noticed. Accumulo3030IT was deleted, why? Also I saw another IT was deleted, but can not remember which one.
Two tests were deleted in the current patch (that I recall). Accumulo3030IT was replaced with InterruptibleScannersIT and there was an orphaned test that didn't match "*IT" which means it was never actually being run during the normal build.
- Josh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62230
-----------------------------------------------------------
On Nov. 19, 2014, 5:01 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 19, 2014, 5:01 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by ke...@deenlo.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/#review62230
-----------------------------------------------------------
test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java
<https://reviews.apache.org/r/28214/#comment104240>
Could you do `Assume.assumeTrue(canRunTest(type))` ?
test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java
<https://reviews.apache.org/r/28214/#comment104243>
the enum could have an isDynamic method.
Still looking at this, just publishing a few things I noticed. Accumulo3030IT was deleted, why? Also I saw another IT was deleted, but can not remember which one.
- kturner
On Nov. 19, 2014, 5:01 p.m., Josh Elser wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28214/
> -----------------------------------------------------------
>
> (Updated Nov. 19, 2014, 5:01 p.m.)
>
>
> Review request for accumulo.
>
>
> Bugs: ACCUMULO-3167
> https://issues.apache.org/jira/browse/ACCUMULO-3167
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
>
> Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
>
> I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
>
>
> Diffs
> -----
>
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
> minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
> minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
> minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
> minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
> minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
> minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
> minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
> test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
> test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
> test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
> test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
> test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
> test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
> test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
> test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
> test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
> test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
> test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
> test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
> test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
> test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
> test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
> test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
> test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
> test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
> test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
> test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
> test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
> test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
> test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
> test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
> test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
> test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
> test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
> test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
> test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
> test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
> test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
> test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
> test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
> test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
> test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
> test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
> test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
> test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
> test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
> test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
> test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
> test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
> test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
> test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
> test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
> test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
> test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
> test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
> test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
> test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
> test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
> test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
> test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
> test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
> test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
> test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
> test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
> test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
> test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
> test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
> test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
> test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
> test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
> test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
> test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
> test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
> test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
> test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
> test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
> test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
> test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
> test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
> test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
> test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
> test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
> test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
> test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
> test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
> test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
> test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
> test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
> test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
> test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
>
> Diff: https://reviews.apache.org/r/28214/diff/
>
>
> Testing
> -------
>
> Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
>
>
> Thanks,
>
> Josh Elser
>
>
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/
-----------------------------------------------------------
(Updated Nov. 19, 2014, 5:01 p.m.)
Review request for accumulo.
Changes
-------
Fixes trivial issues.
Bugs: ACCUMULO-3167
https://issues.apache.org/jira/browse/ACCUMULO-3167
Repository: accumulo
Description
-------
We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
Diffs (updated)
-----
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
Diff: https://reviews.apache.org/r/28214/diff/
Testing
-------
Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
Thanks,
Josh Elser
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/
-----------------------------------------------------------
(Updated Nov. 19, 2014, 4:02 p.m.)
Review request for accumulo.
Changes
-------
Missed some more dead code cleanup
Bugs: ACCUMULO-3167
https://issues.apache.org/jira/browse/ACCUMULO-3167
Repository: accumulo
Description
-------
We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
Diffs (updated)
-----
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
test/src/test/java/org/apache/accumulo/test/functional/BulkFileIT.java a7cf6bd
test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
Diff: https://reviews.apache.org/r/28214/diff/
Testing
-------
Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
Thanks,
Josh Elser
Re: Review Request 28214: Decouple MiniAccumuloCluster from
integration tests and introduce StandaloneAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28214/
-----------------------------------------------------------
(Updated Nov. 19, 2014, 5:24 a.m.)
Review request for accumulo.
Changes
-------
Broke compilation removing some unused things at the last minute.
Bugs: ACCUMULO-3167
https://issues.apache.org/jira/browse/ACCUMULO-3167
Repository: accumulo
Description
-------
We have a large number of good tests which are only capable of being run against a MiniAccumuloCluster. This is undesirable because it limits our tests to the scope of what MiniAccumuloCluster provides, or implements as a standalone cluster does. An accurate test environment would be a true deployment with distributed HDFS and ZooKeper instances that back a distributed Accumulo instance. This patch introduces a StandaloneAccumuloCluster, in addition to a few other new interfaces (ClusterControl) which help support the necessary functionality. The StandaloneAccumuloCluster is the "MiniAccumuloCluster" counterpart to a distributed cluster.
Given the StandaloneAccumuloCluster, many of the integration tests need to be rewritten in such a way that support both a MiniAccumuloCluster and the Standalone cluster. While being a painful set of changes, this does help generalize some of the tests and conform them to some best practices to simplify things.
I also nuked some initial interfaces which I initially stubbed out because they turned out not being useful.
Diffs (updated)
-----
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloCluster.java c982de0
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloClusters.java 50cb9db
minicluster/src/main/java/org/apache/accumulo/cluster/AccumuloConfig.java 0df2348
minicluster/src/main/java/org/apache/accumulo/cluster/ClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShell.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/RemoteShellOptions.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/package-info.java f1b649d
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneAccumuloCluster.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/cluster/standalone/StandaloneClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java 3e8c5a0
minicluster/src/main/java/org/apache/accumulo/minicluster/ServerType.java 3590a20
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterControl.java PRE-CREATION
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 7283c19
minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 2d7103e
minicluster/src/test/java/org/apache/accumulo/cluster/AccumuloClustersTest.java e368240
minicluster/src/test/java/org/apache/accumulo/minicluster/MiniAccumuloClusterStartStopTest.java 2031b11
minicluster/src/test/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImplTest.java b19d289
test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java 16f0b3f
test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/AccumuloIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterConfigurationCallback.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloClusterPropertyConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/harness/conf/AccumuloStandaloneClusterConfiguration.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/Accumulo3010IT.java 791b1d5
test/src/test/java/org/apache/accumulo/test/Accumulo3030IT.java 3512e4a
test/src/test/java/org/apache/accumulo/test/Accumulo3047IT.java 70e1c30
test/src/test/java/org/apache/accumulo/test/AuditMessageIT.java 49b5d70
test/src/test/java/org/apache/accumulo/test/BatchWriterIT.java ca72e7a
test/src/test/java/org/apache/accumulo/test/CleanWalIT.java d0bfe3f
test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java 516cd46
test/src/test/java/org/apache/accumulo/test/ConfigurableMajorCompactionIT.java 899b41b
test/src/test/java/org/apache/accumulo/test/DeleteRowsIT.java ff67e89
test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java ec72281
test/src/test/java/org/apache/accumulo/test/ImportExportIT.java a48ed9d
test/src/test/java/org/apache/accumulo/test/InterruptibleScannersIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/KeyValueEqualityTest.java 1302b23
test/src/test/java/org/apache/accumulo/test/MetaConstraintRetryIT.java b3c3640
test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java 50a1446
test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java 484c048
test/src/test/java/org/apache/accumulo/test/NamespacesIT.java 8188deb
test/src/test/java/org/apache/accumulo/test/NoMutationRecoveryIT.java 87ad1a3
test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 4457e70
test/src/test/java/org/apache/accumulo/test/SplitRecoveryIT.java 96d3a1a
test/src/test/java/org/apache/accumulo/test/TableConfigurationUpdateIT.java 0d9a211
test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java bb12279
test/src/test/java/org/apache/accumulo/test/VolumeIT.java d5c940d
test/src/test/java/org/apache/accumulo/test/functional/AbstractMacIT.java 22e46ff
test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java ad84960
test/src/test/java/org/apache/accumulo/test/functional/AddSplitIT.java 05de342
test/src/test/java/org/apache/accumulo/test/functional/BadIteratorMincIT.java 9c4492e
test/src/test/java/org/apache/accumulo/test/functional/BalanceInPresenceOfOfflineTableIT.java 887aee4
test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 688a326
test/src/test/java/org/apache/accumulo/test/functional/BatchWriterFlushIT.java 465936e
test/src/test/java/org/apache/accumulo/test/functional/BigRootTabletIT.java b021c3a
test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java 9c0edaa
test/src/test/java/org/apache/accumulo/test/functional/BinaryStressIT.java a60c2d5
test/src/test/java/org/apache/accumulo/test/functional/BloomFilterIT.java 8f6b830
test/src/test/java/org/apache/accumulo/test/functional/BulkFileIT.java a7cf6bd
test/src/test/java/org/apache/accumulo/test/functional/BulkIT.java 831dcd4
test/src/test/java/org/apache/accumulo/test/functional/ChaoticBalancerIT.java 8afb3d2
test/src/test/java/org/apache/accumulo/test/functional/ClassLoaderIT.java d71819e
test/src/test/java/org/apache/accumulo/test/functional/CleanUpIT.java 79bbb90
test/src/test/java/org/apache/accumulo/test/functional/CloneTestIT.java 505dd5a
test/src/test/java/org/apache/accumulo/test/functional/CombinerIT.java 69f9134
test/src/test/java/org/apache/accumulo/test/functional/ConcurrencyIT.java 92bd714
test/src/test/java/org/apache/accumulo/test/functional/ConfigurableMacIT.java 9185e1b
test/src/test/java/org/apache/accumulo/test/functional/ConstraintIT.java 7e5944e
test/src/test/java/org/apache/accumulo/test/functional/CreateAndUseIT.java 5b5249b
test/src/test/java/org/apache/accumulo/test/functional/CreateManyScannersIT.java aed38e8
test/src/test/java/org/apache/accumulo/test/functional/CredentialsIT.java 8e2e1e0
test/src/test/java/org/apache/accumulo/test/functional/DeleteEverythingIT.java 0578ef4
test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java 3510fbd
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsIT.java 4b7d664
test/src/test/java/org/apache/accumulo/test/functional/DeleteRowsSplitIT.java d35ba9f
test/src/test/java/org/apache/accumulo/test/functional/DeleteTableDuringSplitIT.java a0bff64
test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java 87497b9
test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java 4d75a16
test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java d77d060
test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java 6aec7cd
test/src/test/java/org/apache/accumulo/test/functional/MasterAssignmentIT.java 46f6b23
test/src/test/java/org/apache/accumulo/test/functional/MasterFailoverIT.java 218d65e
test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java 2649890
test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java c264dfe
test/src/test/java/org/apache/accumulo/test/functional/MetadataIT.java bd0282d
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFiles.java 6b8d9b3
test/src/test/java/org/apache/accumulo/test/functional/MetadataMaxFilesIT.java PRE-CREATION
test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java d51dcbb
test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java 60b1908
test/src/test/java/org/apache/accumulo/test/functional/RecoveryWithEmptyRFileIT.java 814dd85
test/src/test/java/org/apache/accumulo/test/functional/RenameIT.java 8cbe84f
test/src/test/java/org/apache/accumulo/test/functional/RestartIT.java e4b9c5a
test/src/test/java/org/apache/accumulo/test/functional/RestartStressIT.java d8c2804
test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java 4dbd912
test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java 189a55c
test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java 90b881c
test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java 3547b68
test/src/test/java/org/apache/accumulo/test/functional/ScannerIT.java cbd1290
test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java d765b16
test/src/test/java/org/apache/accumulo/test/functional/SimpleMacIT.java a4e6647
test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java 0b63d01
test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java 6203523
test/src/test/java/org/apache/accumulo/test/functional/SslIT.java a14795c
test/src/test/java/org/apache/accumulo/test/functional/StartIT.java 82278af
test/src/test/java/org/apache/accumulo/test/functional/TableIT.java 832ec60
test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java fccc79f
test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java 4dc72e0
test/src/test/java/org/apache/accumulo/test/functional/VisibilityIT.java f2460cf
test/src/test/java/org/apache/accumulo/test/functional/WriteAheadLogIT.java af6eca5
test/src/test/java/org/apache/accumulo/test/functional/WriteLotsIT.java 214fc2f
Diff: https://reviews.apache.org/r/28214/diff/
Testing
-------
Haven't had a 100% IT pass rate yet (last run was about 95% pass rate), but I wanted to get the code up and have some eyes on it sooner than later.
Thanks,
Josh Elser