You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Josh Elser <jo...@gmail.com> on 2014/05/29 04:53:06 UTC

Review Request 22003: Replication: Client-facing changes

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/
-----------------------------------------------------------

Review request for accumulo.


Bugs: ACCUMULO-378
    https://issues.apache.org/jira/browse/ACCUMULO-378


Repository: accumulo


Description
-------

"Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.


Diffs
-----

  core/pom.xml 2c3f648 
  core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
  core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
  core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
  core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
  core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
  core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
  core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
  core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
  core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
  core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
  core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
  core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
  core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
  core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
  core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
  core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
  core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
  core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
  core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
  core/src/main/protobuf/replication.proto PRE-CREATION 
  core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
  core/src/main/scripts/generate-thrift.sh 5a5d69f 
  core/src/main/thrift/data.thrift ae6f439 
  core/src/main/thrift/replication.thrift PRE-CREATION 
  core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
  core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
  core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
  core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
  core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
  core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
  docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
  docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
  docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
  docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
  docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
  minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
  server/monitor/pom.xml 411812c 
  server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
  server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
  server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 

Diff: https://reviews.apache.org/r/22003/diff/


Testing
-------


Thanks,

Josh Elser


Re: Review Request 22003: Replication: Client-facing changes

Posted by Josh Elser <jo...@gmail.com>.

> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java, line 46
> > <https://reviews.apache.org/r/22003/diff/1/?file=598220#file598220line46>
> >
> >     If this really needs to be a utility class, it needs a private constructor to avoid instantiation. But, I would *so much rather* it be a regular class to allow mocking / testing / dependency injection.
> 
> Josh Elser wrote:
>     The "cruft" of this class was inherited by copying it from MasterClient. I can understand where you're coming from, but I will say that most of the code in here isn't the "critical" thing to directly test. It is covered by the other tests which use this to actually connect to a remote service.
>     
>     I could convert it to a regular class with instance methods, or some factory class which returns instances of these connections, but honestly I don't see that buying us much anything in respect to code coverage or practical tests.
> 
> Bill Havanki wrote:
>     That's fine with me. Keeping it consistent with other similar classes is a good goal. This is something we can re-address later if it does become problematic, but if you don't see it being useful to do now, that's good enough for me.
> 
> Josh Elser wrote:
>     I'll open up a general ticket to revisit both of these classes and see what we can extract into a more testable, common base.

Created ACCUMULO-2867


- Josh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44646
-----------------------------------------------------------


On May 29, 2014, 2:52 a.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 29, 2014, 2:52 a.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Re: Review Request 22003: Replication: Client-facing changes

Posted by Josh Elser <jo...@gmail.com>.

> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java, line 35
> > <https://reviews.apache.org/r/22003/diff/1/?file=598217#file598217line35>
> >
> >     The usual convention is to use third-person present tense instead of imperative, e.g., "defines" instead of "define". See http://www.oracle.com/technetwork/java/javase/documentation/index-137868.html#styleguide.
> >     
> >     Also, I think javadoc should have a first sentence that ends with a period / some other mark.

Fine by me!


> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java, line 55
> > <https://reviews.apache.org/r/22003/diff/1/?file=598220#file598220line55>
> >
> >     Any reason to avoid checkNotNull here?

Nah, was just a copy-paste I didn't fix from MasterClient.


> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java, line 46
> > <https://reviews.apache.org/r/22003/diff/1/?file=598220#file598220line46>
> >
> >     If this really needs to be a utility class, it needs a private constructor to avoid instantiation. But, I would *so much rather* it be a regular class to allow mocking / testing / dependency injection.

The "cruft" of this class was inherited by copying it from MasterClient. I can understand where you're coming from, but I will say that most of the code in here isn't the "critical" thing to directly test. It is covered by the other tests which use this to actually connect to a remote service.

I could convert it to a regular class with instance methods, or some factory class which returns instances of these connections, but honestly I don't see that buying us much anything in respect to code coverage or practical tests.


> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java, line 94
> > <https://reviews.apache.org/r/22003/diff/1/?file=598220#file598220line94>
> >
> >     Log the exception too

This was actually intentional to avoid spam (e.g. a peer is offline). I think this might be less of a worry now that the caller does the sleep with a backoff.


> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java, line 111
> > <https://reviews.apache.org/r/22003/diff/1/?file=598220#file598220line111>
> >
> >     Avoid throwing an ordinary RuntimeException

Not as "deadly" when talking to a peer's Master (as opposed to our Master). Just removing it completely, and letting it cycle normally.


> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java, line 113
> > <https://reviews.apache.org/r/22003/diff/1/?file=598220#file598220line113>
> >
> >     The retry mechanism isn't in this method, so it shouldn't log that here.

More copy-paste. Left real description in place, lifted retry message to caller.


> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java, line 152
> > <https://reviews.apache.org/r/22003/diff/1/?file=598221#file598221line152>
> >
> >     Make the number of query threads configurable.

4 threads should be fine as a default, but created ACCUMULO-2856 to make these thread pool sizes configurable if something comes up that I didn't think of.


> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java, line 158
> > <https://reviews.apache.org/r/22003/diff/1/?file=598221#file598221line158>
> >
> >     Possible / beneficial to use the same batch scanner for each loop iteration?

BatchScanners can't be re-used.


> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java, line 95
> > <https://reviews.apache.org/r/22003/diff/1/?file=598224#file598224line95>
> >
> >     Could adding a table ID to mock tables be a separate, smaller commit / ticket?

Ugh, I forgot about this one. I don't remember what I was actually invoking that created the underlying exception. I think it was TableOperations.tableIdMap not working with MockInstance. I can lift it out, but there is more included that just these changes (MockTableOperations, at least). Filed ACCUMULO-2857


> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java, line 36
> > <https://reviews.apache.org/r/22003/diff/1/?file=598228#file598228line36>
> >
> >     Would it be useful to define an over-arching sort of ReplicationException for this?

I don't think so, because then we would have to bake the logic of what exactly to do when that exception is seen into the TabletServer. This forces the implementation to know what its doing and how to recover from Exceptions, rather than let that bubble up into the core Accumulo logic.


> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java, line 50
> > <https://reviews.apache.org/r/22003/diff/1/?file=598229#file598229line50>
> >
> >     Just check clz? Or use o instanceof ReplicaSystem

Is there actually a difference between what's in the code and what `instanceof ReplicaSystem` would actually do? Could I have a class that implements ReplicaSystem that isn't an instanceof it? I don't know, tbh.


> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java, line 59
> > <https://reviews.apache.org/r/22003/diff/1/?file=598229#file598229line59>
> >
> >     ew, throwing RuntimeException

Changed to IllegalArgumentException (assuming your issue was with the untyped RTE)


> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java, line 75
> > <https://reviews.apache.org/r/22003/diff/1/?file=598229#file598229line75>
> >
> >     This could return a value which the get() method will pull an empty-string configuration from. Is that OK?

This isn't pulling data from a Configuration object, merely providing "serialization" that would go into the value of a Property. I think this is fine as-is.


- Josh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44646
-----------------------------------------------------------


On May 29, 2014, 2:52 a.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 29, 2014, 2:52 a.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Re: Review Request 22003: Replication: Client-facing changes

Posted by Bill Havanki <bh...@clouderagovt.com>.

> On June 3, 2014, 2:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java, line 46
> > <https://reviews.apache.org/r/22003/diff/1/?file=598220#file598220line46>
> >
> >     If this really needs to be a utility class, it needs a private constructor to avoid instantiation. But, I would *so much rather* it be a regular class to allow mocking / testing / dependency injection.
> 
> Josh Elser wrote:
>     The "cruft" of this class was inherited by copying it from MasterClient. I can understand where you're coming from, but I will say that most of the code in here isn't the "critical" thing to directly test. It is covered by the other tests which use this to actually connect to a remote service.
>     
>     I could convert it to a regular class with instance methods, or some factory class which returns instances of these connections, but honestly I don't see that buying us much anything in respect to code coverage or practical tests.

That's fine with me. Keeping it consistent with other similar classes is a good goal. This is something we can re-address later if it does become problematic, but if you don't see it being useful to do now, that's good enough for me.


> On June 3, 2014, 2:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java, line 94
> > <https://reviews.apache.org/r/22003/diff/1/?file=598220#file598220line94>
> >
> >     Log the exception too
> 
> Josh Elser wrote:
>     This was actually intentional to avoid spam (e.g. a peer is offline). I think this might be less of a worry now that the caller does the sleep with a backoff.

OK


> On June 3, 2014, 2:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java, line 59
> > <https://reviews.apache.org/r/22003/diff/1/?file=598229#file598229line59>
> >
> >     ew, throwing RuntimeException
> 
> Josh Elser wrote:
>     Changed to IllegalArgumentException (assuming your issue was with the untyped RTE)

It was, thanks :)


> On June 3, 2014, 2:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java, line 36
> > <https://reviews.apache.org/r/22003/diff/1/?file=598228#file598228line36>
> >
> >     Would it be useful to define an over-arching sort of ReplicationException for this?
> 
> Josh Elser wrote:
>     I don't think so, because then we would have to bake the logic of what exactly to do when that exception is seen into the TabletServer. This forces the implementation to know what its doing and how to recover from Exceptions, rather than let that bubble up into the core Accumulo logic.

OK, that's a good design constraint. Along the same lines, if some odd thing happens that can't be handled within the replication logic, tossing a RuntimeException would be appropriate.


> On June 3, 2014, 2:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java, line 50
> > <https://reviews.apache.org/r/22003/diff/1/?file=598229#file598229line50>
> >
> >     Just check clz? Or use o instanceof ReplicaSystem
> 
> Josh Elser wrote:
>     Is there actually a difference between what's in the code and what `instanceof ReplicaSystem` would actually do? Could I have a class that implements ReplicaSystem that isn't an instanceof it? I don't know, tbh.

Sorry, should have explained my thought process more. You could just do isAssignableFrom(clz) right away, without making an instance, and detect the case where clz isn't ReplicaSystem.class or a subclass. Then you wouldn't have to call Class.newInstance on that class type when it ends up useless anyway.


> On June 3, 2014, 2:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java, line 75
> > <https://reviews.apache.org/r/22003/diff/1/?file=598229#file598229line75>
> >
> >     This could return a value which the get() method will pull an empty-string configuration from. Is that OK?
> 
> Josh Elser wrote:
>     This isn't pulling data from a Configuration object, merely providing "serialization" that would go into the value of a Property. I think this is fine as-is.

OK


- Bill


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44646
-----------------------------------------------------------


On May 28, 2014, 10:52 p.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 28, 2014, 10:52 p.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Re: Review Request 22003: Replication: Client-facing changes

Posted by Josh Elser <jo...@gmail.com>.

> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java, line 46
> > <https://reviews.apache.org/r/22003/diff/1/?file=598220#file598220line46>
> >
> >     If this really needs to be a utility class, it needs a private constructor to avoid instantiation. But, I would *so much rather* it be a regular class to allow mocking / testing / dependency injection.
> 
> Josh Elser wrote:
>     The "cruft" of this class was inherited by copying it from MasterClient. I can understand where you're coming from, but I will say that most of the code in here isn't the "critical" thing to directly test. It is covered by the other tests which use this to actually connect to a remote service.
>     
>     I could convert it to a regular class with instance methods, or some factory class which returns instances of these connections, but honestly I don't see that buying us much anything in respect to code coverage or practical tests.
> 
> Bill Havanki wrote:
>     That's fine with me. Keeping it consistent with other similar classes is a good goal. This is something we can re-address later if it does become problematic, but if you don't see it being useful to do now, that's good enough for me.

I'll open up a general ticket to revisit both of these classes and see what we can extract into a more testable, common base.


> On June 3, 2014, 6:11 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java, line 50
> > <https://reviews.apache.org/r/22003/diff/1/?file=598229#file598229line50>
> >
> >     Just check clz? Or use o instanceof ReplicaSystem
> 
> Josh Elser wrote:
>     Is there actually a difference between what's in the code and what `instanceof ReplicaSystem` would actually do? Could I have a class that implements ReplicaSystem that isn't an instanceof it? I don't know, tbh.
> 
> Bill Havanki wrote:
>     Sorry, should have explained my thought process more. You could just do isAssignableFrom(clz) right away, without making an instance, and detect the case where clz isn't ReplicaSystem.class or a subclass. Then you wouldn't have to call Class.newInstance on that class type when it ends up useless anyway.

Ahhh, ok, I got ya.


- Josh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44646
-----------------------------------------------------------


On May 29, 2014, 2:52 a.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 29, 2014, 2:52 a.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Re: Review Request 22003: Replication: Client-facing changes

Posted by Bill Havanki <bh...@clouderagovt.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44646
-----------------------------------------------------------



core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java
<https://reviews.apache.org/r/22003/#comment79059>

    The usual convention is to use third-person present tense instead of imperative, e.g., "defines" instead of "define". See http://www.oracle.com/technetwork/java/javase/documentation/index-137868.html#styleguide.
    
    Also, I think javadoc should have a first sentence that ends with a period / some other mark.



core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java
<https://reviews.apache.org/r/22003/#comment79060>

    Javadoc needs update.



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java
<https://reviews.apache.org/r/22003/#comment79062>

    If this really needs to be a utility class, it needs a private constructor to avoid instantiation. But, I would *so much rather* it be a regular class to allow mocking / testing / dependency injection.



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java
<https://reviews.apache.org/r/22003/#comment79064>

    Any reason to avoid checkNotNull here?



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java
<https://reviews.apache.org/r/22003/#comment79065>

    UtilWaitThread.sleep eats interrupts, so I don't like it. :) Can this do a regular sleep and handle interruption gracefully, perhaps by just throwing an AccumuloException?
    
    (I won't comment on any other uses of the method.)



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java
<https://reviews.apache.org/r/22003/#comment79066>

    Log the exception too



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java
<https://reviews.apache.org/r/22003/#comment79067>

    Avoid throwing an ordinary RuntimeException



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java
<https://reviews.apache.org/r/22003/#comment79068>

    The retry mechanism isn't in this method, so it shouldn't log that here.



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java
<https://reviews.apache.org/r/22003/#comment79071>

    Consider adding an overall timeout and/or maximum number of attempts.



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java
<https://reviews.apache.org/r/22003/#comment79070>

    This catches RuntimeExceptions too. Isn't there some awesome Java 7 way to do this? Can't remember.



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java
<https://reviews.apache.org/r/22003/#comment79072>

    Consider adding an overall timeout and/or maximum number of attempts.



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java
<https://reviews.apache.org/r/22003/#comment79074>

    Make these fields final. Null check too?



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java
<https://reviews.apache.org/r/22003/#comment79077>

    Looks like we could use an asynchronous / callback mechanism for this sort of thing.



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java
<https://reviews.apache.org/r/22003/#comment79079>

    Make the number of query threads configurable.



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java
<https://reviews.apache.org/r/22003/#comment79081>

    The singleton set here could be computed once outside the loop.



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java
<https://reviews.apache.org/r/22003/#comment79082>

    Possible / beneficial to use the same batch scanner for each loop iteration?



core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java
<https://reviews.apache.org/r/22003/#comment79083>

    Refactor this with the same section in drain().



core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java
<https://reviews.apache.org/r/22003/#comment79084>

    Could adding a table ID to mock tables be a separate, smaller commit / ticket?



core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java
<https://reviews.apache.org/r/22003/#comment79085>

    Add a constructor for (peer, message, cause)



core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java
<https://reviews.apache.org/r/22003/#comment79086>

    Add a constructor for (peer, message, cause)



core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java
<https://reviews.apache.org/r/22003/#comment79087>

    Missing @param for helper



core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java
<https://reviews.apache.org/r/22003/#comment79088>

    Would it be useful to define an over-arching sort of ReplicationException for this?



core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java
<https://reviews.apache.org/r/22003/#comment79089>

    sp: quorum



core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java
<https://reviews.apache.org/r/22003/#comment79090>

    Needs a private constructor



core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java
<https://reviews.apache.org/r/22003/#comment79092>

    Just check clz? Or use o instanceof ReplicaSystem



core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java
<https://reviews.apache.org/r/22003/#comment79091>

    ew, throwing RuntimeException



core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java
<https://reviews.apache.org/r/22003/#comment79093>

    This could return a value which the get() method will pull an empty-string configuration from. Is that OK?


- Bill Havanki


On May 28, 2014, 10:52 p.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 28, 2014, 10:52 p.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Re: Review Request 22003: Replication: Client-facing changes

Posted by Josh Elser <jo...@gmail.com>.

> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/conf/Property.java, line 433
> > <https://reviews.apache.org/r/22003/diff/1/?file=598231#file598231line433>
> >
> >     Target or targets? enum and key should match in number.

I did it this way because it is a prefix. and thus there would be many "table.replication.target." prefixes, but each key-value is singular and represents a single target. Do you still think this is confusing?


- Josh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44749
-----------------------------------------------------------


On May 29, 2014, 2:52 a.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 29, 2014, 2:52 a.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Re: Review Request 22003: Replication: Client-facing changes

Posted by Christopher Tubbs <ct...@apache.org>.

> On June 4, 2014, 4:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java, lines 34-47
> > <https://reviews.apache.org/r/22003/diff/1/?file=598217#file598217line34>
> >
> >     What is the difference between a ReplicaSystem parameter and the class name of a ReplicaSystem? Will the class name be instantiated on behalf of the user?
> >     
> >     Also, is String the best type for the class name? Or would Class<? extends ReplicaSystem> be better?
> 
> Josh Elser wrote:
>     No difference in the implementation, just trying to be flexible in what variants are available. Class<? extends ReplicaSystem> would also make sense -- I'll add that.

I think there would be a slight difference in implementation... because you have to instantiate the ReplicaSystem if you're not passed an instance. The javadoc should explain how that instantiation works (no-arg constructor and init(...) method, or only no-arg constructor, or something else)


> On June 4, 2014, 4:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java, lines 56-70
> > <https://reviews.apache.org/r/22003/diff/1/?file=598217#file598217line56>
> >
> >     Are there other terms that are better than "drain"? "sync", for example, is a bit more descriptive, and has standard analogies in filesystems.
> 
> Josh Elser wrote:
>     I'd be worried of implying that naming the method "sync" might imply that it's actually doing something. I liked drain because it's more descriptive that it's just waiting (e.g. for water to drain in a sink). It's not doing anything active to *make* the replication happen. I can likely make better javadoc here -- I had been waffling with how these methods should work for a while, and they didn't get as much love as they should've once I finally accepted them.

I like the drain analogy, but from an API perspective, the user doesn't really need to know here whether the implementation flushes queued replication tasks or just waits for existing tasks to complete, or some other implementation variant (and that could actually change over time), and the semantics appear to be strikingly similar to coreutils' /bin/sync

If all you want to provide is a wait loop, perhaps another option is to just provide an "boolean isFullyReplicated();" method or similar so users can write their own wait code. Alternatively, something very explicit might work rather than an analogy, like "waitForFullReplication()" or similar.

With good javadocs, drain works, but I think it could be more clear.

(Also, the overloaded method with the additional files parameter needs to be explained a bit more.)


> On June 4, 2014, 4:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/conf/Property.java, line 433
> > <https://reviews.apache.org/r/22003/diff/1/?file=598231#file598231line433>
> >
> >     Target or targets? enum and key should match in number.
> 
> Josh Elser wrote:
>     I did it this way because it is a prefix. and thus there would be many "table.replication.target." prefixes, but each key-value is singular and represents a single target. Do you still think this is confusing?

By convention, all our other property prefixes look like: <SINGULAR_THING_THAT_IS_PREFIXED>_PREFIX

The reason for this is because it doesn't prefix the set. Rather, it prefixes each singular element of the set. Further, the enum and the key for the enum are really identifiers for the same thing (in a slightly different form), so it's very confusing for them to have different number. Grammatically speaking, I think the convention is better.


> On June 4, 2014, 4:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java, line 108
> > <https://reviews.apache.org/r/22003/diff/1/?file=598232#file598232line108>
> >
> >     Maybe in a future version. Would be nice.
> 
> Josh Elser wrote:
>     It's a possibility -- I don't know how to make that work yet. We can make a ticket if you feel it would be prudent :)

A ticket would be nice, for future work.


> On June 4, 2014, 4:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java, lines 56-57
> > <https://reviews.apache.org/r/22003/diff/1/?file=598239#file598239line56>
> >
> >     Are these necessary to add to the public API?
> 
> Josh Elser wrote:
>     Ah, I had to look, but the reason I did this is so that I could actually test that the replication table had the correct configuration set. If I'm doing it, I don't see why users might not also want to do the same. By having it be a public variable, we're not advertising that the value itself is fixed, just that the variable will be there? Is that still too much?

The problem with that is that some consumers of the API will make assumptions about how we use these fields internally (for example, to serialize to a configuration). This has happened in the past with the MapReduce configuration and constants in the Input/OutputFormats.

Putting them in the public API is fine, I suppose, as long as we understand that doing so implies that our internal usage of them will be stable, and that may restrict us from restructuring the internals of Combiner (for example, for configuration serialization) without breaking users expectations.

Scenario:
1. User uses constant to manually generate per-table configuration for their combiner.
2. We change the combiner serialization such that it no longer has need of this constant.
3. User's generated per-table config is now broken and they can't figure out why, because they are using a public Constant (and they don't realize Combiner now ignores it).

Keeping it protected reduces these kinds of risks by restricting problems to only users who write their own Combiners.

Granted, it's a small risk, and it's unlikely Combiner is going to change its use of these constants much, if ever. I'm just thinking that if we don't *need* to make it more visible, we should be careful about doing so. To support a test, I suspect you could simply create an anonymous inner Combiner to grab the value of the constant, but perhaps that's too clunky?

I don't feel strongly about this... just trying to be cautious, based on past experience with exposing Constants in the core Constants.java file and in the MapReduce.


- Christopher


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44749
-----------------------------------------------------------


On May 28, 2014, 10:52 p.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 28, 2014, 10:52 p.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Re: Review Request 22003: Replication: Client-facing changes

Posted by Christopher Tubbs <ct...@apache.org>.

> On June 4, 2014, 4:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java, lines 34-47
> > <https://reviews.apache.org/r/22003/diff/1/?file=598217#file598217line34>
> >
> >     What is the difference between a ReplicaSystem parameter and the class name of a ReplicaSystem? Will the class name be instantiated on behalf of the user?
> >     
> >     Also, is String the best type for the class name? Or would Class<? extends ReplicaSystem> be better?
> 
> Josh Elser wrote:
>     No difference in the implementation, just trying to be flexible in what variants are available. Class<? extends ReplicaSystem> would also make sense -- I'll add that.
> 
> Christopher Tubbs wrote:
>     I think there would be a slight difference in implementation... because you have to instantiate the ReplicaSystem if you're not passed an instance. The javadoc should explain how that instantiation works (no-arg constructor and init(...) method, or only no-arg constructor, or something else)
> 
> Josh Elser wrote:
>     No, that is wrong. You should look at the implementation (it should be included in this review). The TabletServer needs to know the class name of a ReplicaSystem that it is to use to replicate data to a peer. This method does no instantiation of anything. A public no-args constructor is required on the ReplicaSystem, so I can add documentation there.

Ah, I see. It's just being passed as configuration. In that case, I think the method that accepts an instantiated ReplicaSystem object as a param is confusing.


> On June 4, 2014, 4:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java, lines 56-57
> > <https://reviews.apache.org/r/22003/diff/1/?file=598239#file598239line56>
> >
> >     Are these necessary to add to the public API?
> 
> Josh Elser wrote:
>     Ah, I had to look, but the reason I did this is so that I could actually test that the replication table had the correct configuration set. If I'm doing it, I don't see why users might not also want to do the same. By having it be a public variable, we're not advertising that the value itself is fixed, just that the variable will be there? Is that still too much?
> 
> Christopher Tubbs wrote:
>     The problem with that is that some consumers of the API will make assumptions about how we use these fields internally (for example, to serialize to a configuration). This has happened in the past with the MapReduce configuration and constants in the Input/OutputFormats.
>     
>     Putting them in the public API is fine, I suppose, as long as we understand that doing so implies that our internal usage of them will be stable, and that may restrict us from restructuring the internals of Combiner (for example, for configuration serialization) without breaking users expectations.
>     
>     Scenario:
>     1. User uses constant to manually generate per-table configuration for their combiner.
>     2. We change the combiner serialization such that it no longer has need of this constant.
>     3. User's generated per-table config is now broken and they can't figure out why, because they are using a public Constant (and they don't realize Combiner now ignores it).
>     
>     Keeping it protected reduces these kinds of risks by restricting problems to only users who write their own Combiners.
>     
>     Granted, it's a small risk, and it's unlikely Combiner is going to change its use of these constants much, if ever. I'm just thinking that if we don't *need* to make it more visible, we should be careful about doing so. To support a test, I suspect you could simply create an anonymous inner Combiner to grab the value of the constant, but perhaps that's too clunky?
>     
>     I don't feel strongly about this... just trying to be cautious, based on past experience with exposing Constants in the core Constants.java file and in the MapReduce.
> 
> Josh Elser wrote:
>     Sure, that's a valid risk; however, if these were ever changed, our tests would also break (at least the test that I wrote which uses these constants). Do you have a possible solution to the issue that I have where I can't validate the configuration of a table that uses a Combiner? I guess I could try to Mock out the code which creates and configures the replication table and just expect that the already-public methods on Combiner get invoked.

Well, you could use the clunky way I described to get the value of the constant from an anonymous subclass, in your test. But... it sounds like it's not really worth it.


- Christopher


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44749
-----------------------------------------------------------


On May 28, 2014, 10:52 p.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 28, 2014, 10:52 p.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Re: Review Request 22003: Replication: Client-facing changes

Posted by Josh Elser <jo...@gmail.com>.

> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > This is a partial review. I didn't examine all the implementation stuff. One question I had was whether or not this adds a new dependency on protocol buffers? If so, where is the pom.xml change for that? Another patch?

Yes, I use protocol buffers. No, it does not require any dependencies. Like thrift, it just generates new Java files which are included, but there is no runtime library required to use them. They are self-contained. Hadoop has a maven plugin that could automate the re-generation of them when the definition needs to be updated, but I didn't bother trying to wire that up since, last I checked, it wasn't in central (and would be a bigger pain to force devs to install over just installing protoc)


> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java, lines 34-47
> > <https://reviews.apache.org/r/22003/diff/1/?file=598217#file598217line34>
> >
> >     What is the difference between a ReplicaSystem parameter and the class name of a ReplicaSystem? Will the class name be instantiated on behalf of the user?
> >     
> >     Also, is String the best type for the class name? Or would Class<? extends ReplicaSystem> be better?

No difference in the implementation, just trying to be flexible in what variants are available. Class<? extends ReplicaSystem> would also make sense -- I'll add that.


> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java, lines 56-70
> > <https://reviews.apache.org/r/22003/diff/1/?file=598217#file598217line56>
> >
> >     Are there other terms that are better than "drain"? "sync", for example, is a bit more descriptive, and has standard analogies in filesystems.

I'd be worried of implying that naming the method "sync" might imply that it's actually doing something. I liked drain because it's more descriptive that it's just waiting (e.g. for water to drain in a sink). It's not doing anything active to *make* the replication happen. I can likely make better javadoc here -- I had been waffling with how these methods should work for a while, and they didn't get as much love as they should've once I finally accepted them.


> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java, line 77
> > <https://reviews.apache.org/r/22003/diff/1/?file=598217#file598217line77>
> >
> >     What does "referenced files" mean? Referenced how? This needs javadoc update to clarify, and the answer may yield a better method name. At the very least, a getReferencedFile would be helpful to identify it clearly as a getter without side-effects.

Noted -- same back-story about drain applies here. I can make better docs, maybe a better name too.


> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java, line 108
> > <https://reviews.apache.org/r/22003/diff/1/?file=598232#file598232line108>
> >
> >     Maybe in a future version. Would be nice.

It's a possibility -- I don't know how to make that work yet. We can make a ticket if you feel it would be prudent :)


> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/data/Mutation.java, line 919
> > <https://reviews.apache.org/r/22003/diff/1/?file=598233#file598233line919>
> >
> >     Please apply project formatter to strip trailing whitespace on blank lines. (commenting here, but applies to entire patch set)

Yes, I intended to reformat everything before I commit. It's a waste of time for me to incrementally re-apply the formatter after every commit I make (assuming I goof somewhere every time).


> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java, lines 617-646
> > <https://reviews.apache.org/r/22003/diff/1/?file=598236#file598236line617>
> >
> >     Next time, please exclude generated thrift from review. :)

Hahaha, psh. Sorry, I didn't think about excluding them when I was copy/pasting these diffs.


> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java, lines 56-57
> > <https://reviews.apache.org/r/22003/diff/1/?file=598239#file598239line56>
> >
> >     Are these necessary to add to the public API?

Ah, I had to look, but the reason I did this is so that I could actually test that the replication table had the correct configuration set. If I'm doing it, I don't see why users might not also want to do the same. By having it be a public variable, we're not advertising that the value itself is fixed, just that the variable will be there? Is that still too much?


- Josh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44749
-----------------------------------------------------------


On May 29, 2014, 2:52 a.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 29, 2014, 2:52 a.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Re: Review Request 22003: Replication: Client-facing changes

Posted by Josh Elser <jo...@gmail.com>.

> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java, lines 34-47
> > <https://reviews.apache.org/r/22003/diff/1/?file=598217#file598217line34>
> >
> >     What is the difference between a ReplicaSystem parameter and the class name of a ReplicaSystem? Will the class name be instantiated on behalf of the user?
> >     
> >     Also, is String the best type for the class name? Or would Class<? extends ReplicaSystem> be better?
> 
> Josh Elser wrote:
>     No difference in the implementation, just trying to be flexible in what variants are available. Class<? extends ReplicaSystem> would also make sense -- I'll add that.
> 
> Christopher Tubbs wrote:
>     I think there would be a slight difference in implementation... because you have to instantiate the ReplicaSystem if you're not passed an instance. The javadoc should explain how that instantiation works (no-arg constructor and init(...) method, or only no-arg constructor, or something else)

No, that is wrong. You should look at the implementation (it should be included in this review). The TabletServer needs to know the class name of a ReplicaSystem that it is to use to replicate data to a peer. This method does no instantiation of anything. A public no-args constructor is required on the ReplicaSystem, so I can add documentation there.


> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java, lines 56-70
> > <https://reviews.apache.org/r/22003/diff/1/?file=598217#file598217line56>
> >
> >     Are there other terms that are better than "drain"? "sync", for example, is a bit more descriptive, and has standard analogies in filesystems.
> 
> Josh Elser wrote:
>     I'd be worried of implying that naming the method "sync" might imply that it's actually doing something. I liked drain because it's more descriptive that it's just waiting (e.g. for water to drain in a sink). It's not doing anything active to *make* the replication happen. I can likely make better javadoc here -- I had been waffling with how these methods should work for a while, and they didn't get as much love as they should've once I finally accepted them.
> 
> Christopher Tubbs wrote:
>     I like the drain analogy, but from an API perspective, the user doesn't really need to know here whether the implementation flushes queued replication tasks or just waits for existing tasks to complete, or some other implementation variant (and that could actually change over time), and the semantics appear to be strikingly similar to coreutils' /bin/sync
>     
>     If all you want to provide is a wait loop, perhaps another option is to just provide an "boolean isFullyReplicated();" method or similar so users can write their own wait code. Alternatively, something very explicit might work rather than an analogy, like "waitForFullReplication()" or similar.
>     
>     With good javadocs, drain works, but I think it could be more clear.
>     
>     (Also, the overloaded method with the additional files parameter needs to be explained a bit more.)

I improved the documentation on these methods already. I'm apt to let this ride for now and see if it's still misleading or confusing. I can add the isFullyReplicated method as I can see the merits of having that method as well and it's practically written for me already with the implementation of drain.


> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/conf/Property.java, line 433
> > <https://reviews.apache.org/r/22003/diff/1/?file=598231#file598231line433>
> >
> >     Target or targets? enum and key should match in number.
> 
> Josh Elser wrote:
>     I did it this way because it is a prefix. and thus there would be many "table.replication.target." prefixes, but each key-value is singular and represents a single target. Do you still think this is confusing?
> 
> Christopher Tubbs wrote:
>     By convention, all our other property prefixes look like: <SINGULAR_THING_THAT_IS_PREFIXED>_PREFIX
>     
>     The reason for this is because it doesn't prefix the set. Rather, it prefixes each singular element of the set. Further, the enum and the key for the enum are really identifiers for the same thing (in a slightly different form), so it's very confusing for them to have different number. Grammatically speaking, I think the convention is better.

Ok, I'll remove the plural from the property key.


> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java, line 108
> > <https://reviews.apache.org/r/22003/diff/1/?file=598232#file598232line108>
> >
> >     Maybe in a future version. Would be nice.
> 
> Josh Elser wrote:
>     It's a possibility -- I don't know how to make that work yet. We can make a ticket if you feel it would be prudent :)
> 
> Christopher Tubbs wrote:
>     A ticket would be nice, for future work.

ACCUMULO-2860


> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java, lines 56-57
> > <https://reviews.apache.org/r/22003/diff/1/?file=598239#file598239line56>
> >
> >     Are these necessary to add to the public API?
> 
> Josh Elser wrote:
>     Ah, I had to look, but the reason I did this is so that I could actually test that the replication table had the correct configuration set. If I'm doing it, I don't see why users might not also want to do the same. By having it be a public variable, we're not advertising that the value itself is fixed, just that the variable will be there? Is that still too much?
> 
> Christopher Tubbs wrote:
>     The problem with that is that some consumers of the API will make assumptions about how we use these fields internally (for example, to serialize to a configuration). This has happened in the past with the MapReduce configuration and constants in the Input/OutputFormats.
>     
>     Putting them in the public API is fine, I suppose, as long as we understand that doing so implies that our internal usage of them will be stable, and that may restrict us from restructuring the internals of Combiner (for example, for configuration serialization) without breaking users expectations.
>     
>     Scenario:
>     1. User uses constant to manually generate per-table configuration for their combiner.
>     2. We change the combiner serialization such that it no longer has need of this constant.
>     3. User's generated per-table config is now broken and they can't figure out why, because they are using a public Constant (and they don't realize Combiner now ignores it).
>     
>     Keeping it protected reduces these kinds of risks by restricting problems to only users who write their own Combiners.
>     
>     Granted, it's a small risk, and it's unlikely Combiner is going to change its use of these constants much, if ever. I'm just thinking that if we don't *need* to make it more visible, we should be careful about doing so. To support a test, I suspect you could simply create an anonymous inner Combiner to grab the value of the constant, but perhaps that's too clunky?
>     
>     I don't feel strongly about this... just trying to be cautious, based on past experience with exposing Constants in the core Constants.java file and in the MapReduce.

Sure, that's a valid risk; however, if these were ever changed, our tests would also break (at least the test that I wrote which uses these constants). Do you have a possible solution to the issue that I have where I can't validate the configuration of a table that uses a Combiner? I guess I could try to Mock out the code which creates and configures the replication table and just expect that the already-public methods on Combiner get invoked.


- Josh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44749
-----------------------------------------------------------


On May 29, 2014, 2:52 a.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 29, 2014, 2:52 a.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Re: Review Request 22003: Replication: Client-facing changes

Posted by Josh Elser <jo...@gmail.com>.

> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java, lines 34-47
> > <https://reviews.apache.org/r/22003/diff/1/?file=598217#file598217line34>
> >
> >     What is the difference between a ReplicaSystem parameter and the class name of a ReplicaSystem? Will the class name be instantiated on behalf of the user?
> >     
> >     Also, is String the best type for the class name? Or would Class<? extends ReplicaSystem> be better?
> 
> Josh Elser wrote:
>     No difference in the implementation, just trying to be flexible in what variants are available. Class<? extends ReplicaSystem> would also make sense -- I'll add that.
> 
> Christopher Tubbs wrote:
>     I think there would be a slight difference in implementation... because you have to instantiate the ReplicaSystem if you're not passed an instance. The javadoc should explain how that instantiation works (no-arg constructor and init(...) method, or only no-arg constructor, or something else)
> 
> Josh Elser wrote:
>     No, that is wrong. You should look at the implementation (it should be included in this review). The TabletServer needs to know the class name of a ReplicaSystem that it is to use to replicate data to a peer. This method does no instantiation of anything. A public no-args constructor is required on the ReplicaSystem, so I can add documentation there.
> 
> Christopher Tubbs wrote:
>     Ah, I see. It's just being passed as configuration. In that case, I think the method that accepts an instantiated ReplicaSystem object as a param is confusing.

Yeah, that's valid. I probably already had an instance of the ReplicaSystem implementation because I was writing a unit test. In practice, there's no reason for a client to instantiate it.


> On June 4, 2014, 8:59 p.m., Christopher Tubbs wrote:
> > core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java, lines 56-57
> > <https://reviews.apache.org/r/22003/diff/1/?file=598239#file598239line56>
> >
> >     Are these necessary to add to the public API?
> 
> Josh Elser wrote:
>     Ah, I had to look, but the reason I did this is so that I could actually test that the replication table had the correct configuration set. If I'm doing it, I don't see why users might not also want to do the same. By having it be a public variable, we're not advertising that the value itself is fixed, just that the variable will be there? Is that still too much?
> 
> Christopher Tubbs wrote:
>     The problem with that is that some consumers of the API will make assumptions about how we use these fields internally (for example, to serialize to a configuration). This has happened in the past with the MapReduce configuration and constants in the Input/OutputFormats.
>     
>     Putting them in the public API is fine, I suppose, as long as we understand that doing so implies that our internal usage of them will be stable, and that may restrict us from restructuring the internals of Combiner (for example, for configuration serialization) without breaking users expectations.
>     
>     Scenario:
>     1. User uses constant to manually generate per-table configuration for their combiner.
>     2. We change the combiner serialization such that it no longer has need of this constant.
>     3. User's generated per-table config is now broken and they can't figure out why, because they are using a public Constant (and they don't realize Combiner now ignores it).
>     
>     Keeping it protected reduces these kinds of risks by restricting problems to only users who write their own Combiners.
>     
>     Granted, it's a small risk, and it's unlikely Combiner is going to change its use of these constants much, if ever. I'm just thinking that if we don't *need* to make it more visible, we should be careful about doing so. To support a test, I suspect you could simply create an anonymous inner Combiner to grab the value of the constant, but perhaps that's too clunky?
>     
>     I don't feel strongly about this... just trying to be cautious, based on past experience with exposing Constants in the core Constants.java file and in the MapReduce.
> 
> Josh Elser wrote:
>     Sure, that's a valid risk; however, if these were ever changed, our tests would also break (at least the test that I wrote which uses these constants). Do you have a possible solution to the issue that I have where I can't validate the configuration of a table that uses a Combiner? I guess I could try to Mock out the code which creates and configures the replication table and just expect that the already-public methods on Combiner get invoked.
> 
> Christopher Tubbs wrote:
>     Well, you could use the clunky way I described to get the value of the constant from an anonymous subclass, in your test. But... it sounds like it's not really worth it.

Haha, I'll see if I can mock this to get around opening up these variables, otherwise, it seems like this isn't a big concern.


- Josh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44749
-----------------------------------------------------------


On May 29, 2014, 2:52 a.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 29, 2014, 2:52 a.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Re: Review Request 22003: Replication: Client-facing changes

Posted by Christopher Tubbs <ct...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44749
-----------------------------------------------------------


This is a partial review. I didn't examine all the implementation stuff. One question I had was whether or not this adds a new dependency on protocol buffers? If so, where is the pom.xml change for that? Another patch?


core/src/main/java/org/apache/accumulo/core/Constants.java
<https://reviews.apache.org/r/22003/#comment79278>

    Please don't add new global constants if at all possible. Constants should be scoped to the thing they are for. (eg. Replication.ZKPATH, Replication.WORKQUEUEPATH, etc.)



core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java
<https://reviews.apache.org/r/22003/#comment79281>

    What is the difference between a ReplicaSystem parameter and the class name of a ReplicaSystem? Will the class name be instantiated on behalf of the user?
    
    Also, is String the best type for the class name? Or would Class<? extends ReplicaSystem> be better?



core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java
<https://reviews.apache.org/r/22003/#comment79280>

    Are there other terms that are better than "drain"? "sync", for example, is a bit more descriptive, and has standard analogies in filesystems.



core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java
<https://reviews.apache.org/r/22003/#comment79282>

    What does "referenced files" mean? Referenced how? This needs javadoc update to clarify, and the answer may yield a better method name. At the very least, a getReferencedFile would be helpful to identify it clearly as a getter without side-effects.



core/src/main/java/org/apache/accumulo/core/conf/Property.java
<https://reviews.apache.org/r/22003/#comment79283>

    Target or targets? enum and key should match in number.



core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java
<https://reviews.apache.org/r/22003/#comment79284>

    Maybe in a future version. Would be nice.



core/src/main/java/org/apache/accumulo/core/data/Mutation.java
<https://reviews.apache.org/r/22003/#comment79285>

    Please apply project formatter to strip trailing whitespace on blank lines. (commenting here, but applies to entire patch set)



core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java
<https://reviews.apache.org/r/22003/#comment79287>

    Next time, please exclude generated thrift from review. :)



core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
<https://reviews.apache.org/r/22003/#comment79288>

    Are these necessary to add to the public API?


- Christopher Tubbs


On May 28, 2014, 10:52 p.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 28, 2014, 10:52 p.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Re: Review Request 22003: Replication: Client-facing changes

Posted by Josh Elser <jo...@gmail.com>.

> On June 3, 2014, 7:22 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java, line 31
> > <https://reviews.apache.org/r/22003/diff/1/?file=598230#file598230line31>
> >
> >     Private constructor or convert to regular, non-utility class (preferred by yours truly). Also, the class doesn't really represent the replication table, so renaming it might be good. I do *not* suggest StuffYouCanDoWithReplicationTable.

You'll find a class in the server review (blech - this is why I do not like splitting up this review) that extends this class. I *really* only need it in accumulo-core for holding the name of the replication table. I could lift this in Constants or something, but I was following suit as to how RootTable and MetadataTable operate.


> On June 3, 2014, 7:22 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/data/Mutation.java, line 787
> > <https://reviews.apache.org/r/22003/diff/1/?file=598233#file598233line787>
> >
> >     This might NPE if replicationSources is null. It might be good to simply assure it can never be.

Good call. Unless ServerMutation (or some other Mutation class comes alongs and) does something silly, this should never NPE, but it's an easy fix.


> On June 3, 2014, 7:22 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/data/Mutation.java, line 929
> > <https://reviews.apache.org/r/22003/diff/1/?file=598233#file598233line929>
> >
> >     replicationSources might be null?

No, it should not ever be null with the implementations as they currently stand; however, like mentioned above, it's an easy extra check to make.


- Josh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44655
-----------------------------------------------------------


On May 29, 2014, 2:52 a.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 29, 2014, 2:52 a.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Re: Review Request 22003: Replication: Client-facing changes

Posted by Bill Havanki <bh...@clouderagovt.com>.

> On June 3, 2014, 3:22 p.m., Bill Havanki wrote:
> > core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java, line 31
> > <https://reviews.apache.org/r/22003/diff/1/?file=598230#file598230line31>
> >
> >     Private constructor or convert to regular, non-utility class (preferred by yours truly). Also, the class doesn't really represent the replication table, so renaming it might be good. I do *not* suggest StuffYouCanDoWithReplicationTable.
> 
> Josh Elser wrote:
>     You'll find a class in the server review (blech - this is why I do not like splitting up this review) that extends this class. I *really* only need it in accumulo-core for holding the name of the replication table. I could lift this in Constants or something, but I was following suit as to how RootTable and MetadataTable operate.

I appreciate the consistency with RootTable and MetadataTable. I'll leave it as your call whether to go with lifting the name into core/Constants. It could be worth exploring to see if those other classes could use the same treatment.


- Bill


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44655
-----------------------------------------------------------


On May 28, 2014, 10:52 p.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 28, 2014, 10:52 p.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Re: Review Request 22003: Replication: Client-facing changes

Posted by Bill Havanki <bh...@clouderagovt.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22003/#review44655
-----------------------------------------------------------



core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java
<https://reviews.apache.org/r/22003/#comment79097>

    Private constructor or convert to regular, non-utility class (preferred by yours truly). Also, the class doesn't really represent the replication table, so renaming it might be good. I do *not* suggest StuffYouCanDoWithReplicationTable.



core/src/main/java/org/apache/accumulo/core/conf/Property.java
<https://reviews.apache.org/r/22003/#comment79098>

    sp: replication



core/src/main/java/org/apache/accumulo/core/data/Mutation.java
<https://reviews.apache.org/r/22003/#comment79099>

    This might NPE if replicationSources is null. It might be good to simply assure it can never be.



core/src/main/java/org/apache/accumulo/core/data/Mutation.java
<https://reviews.apache.org/r/22003/#comment79100>

    replicationSources might be null?


- Bill Havanki


On May 28, 2014, 10:52 p.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22003/
> -----------------------------------------------------------
> 
> (Updated May 28, 2014, 10:52 p.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> "Public-facing" changes. Thrift IDLs, protobufs, client API additions, monitor additions.
> 
> 
> Diffs
> -----
> 
>   core/pom.xml 2c3f648 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 7d602bb 
>   core/src/main/java/org/apache/accumulo/core/client/Connector.java 4a2acff 
>   core/src/main/java/org/apache/accumulo/core/client/admin/ReplicationOperations.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java 0df35f6 
>   core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java 1fa8f12 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationClient.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ReplicationOperationsImpl.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockAccumulo.java 2c26ecc 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockConnector.java 996198c 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTable.java cb50761 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperations.java a44a027 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerExistsException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/PeerNotFoundException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystem.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicaSystemFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/replication/ReplicationTable.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 1200fd1 
>   core/src/main/java/org/apache/accumulo/core/data/ConditionalMutation.java e43968c 
>   core/src/main/java/org/apache/accumulo/core/data/Mutation.java 42fa143 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/MultiScanResult.java f0b8a87 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/ScanResult.java 6472587 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TConditionalMutation.java 3f9b3f7 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/TMutation.java f698892 
>   core/src/main/java/org/apache/accumulo/core/data/thrift/UpdateErrors.java 59ce5cb 
>   core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ceb4411 
>   core/src/main/java/org/apache/accumulo/core/metadata/schema/MetadataSchema.java 06eae23 
>   core/src/main/java/org/apache/accumulo/core/protobuf/ProtobufUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/AccumuloReplicationReplayer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/PrintReplicationRecords.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicaSystemHelper.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationSchema.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/ReplicationTarget.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusFormatter.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/StatusUtil.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/proto/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/KeyValues.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/RemoteReplicationException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/Replication.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinator.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorErrorCode.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationCoordinatorException.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/ReplicationServicer.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/replication/thrift/WalEdits.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/schema/Section.java PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/ReplicationFailedException.java PRE-CREATION 
>   core/src/main/protobuf/replication.proto PRE-CREATION 
>   core/src/main/scripts/generate-protobuf.sh PRE-CREATION 
>   core/src/main/scripts/generate-thrift.sh 5a5d69f 
>   core/src/main/thrift/data.thrift ae6f439 
>   core/src/main/thrift/replication.thrift PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationConfigurationUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationOperationsImplTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationSchemaTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/ReplicationTargetTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/StatusUtilTest.java PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/replication/proto/StatusTest.java PRE-CREATION 
>   docs/src/main/asciidoc/accumulo_user_manual.asciidoc fec40ca 
>   docs/src/main/asciidoc/chapters/replication.txt PRE-CREATION 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.gv PRE-CREATION 
>   docs/src/main/resources/state/replicationstatus.png PRE-CREATION 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java 3258991 
>   server/monitor/pom.xml 411812c 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java e6617d0 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java 86a84b9 
>   server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/ReplicationServlet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22003/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>