You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2022/02/08 23:29:32 UTC

[GitHub] [accumulo] milleruntime opened a new issue #2473: Refactor FATE

milleruntime opened a new issue #2473:
URL: https://github.com/apache/accumulo/issues/2473


   I was looking at the warnings we suppress in `ZooStore` and they all seem to stem from the serialization of the generics. I was wondering if it would be better to refactor out the generic types so we don't have to arbitrarily serialize objects and blindly cast them. These would be a bit of work but it might not be too bad. I think if we create an interface like `FateEnvironment` to pass to the operations we wouldn't have to pass the Manager around.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [accumulo] milleruntime commented on issue #2473: Refactor FATE

Posted by GitBox <gi...@apache.org>.
milleruntime commented on issue #2473:
URL: https://github.com/apache/accumulo/issues/2473#issuecomment-1039473784


   Another improvement that could be included in this change:
   Convert FateTxId to a stronger type to prevent coding errors like forgetting to call the method to display the value in HEX.
   The code would probably be 10x better if we had a FateTxId type that we used instead of long. Its toString method could consistently format for logging maybe. That could possible encapsulate or extend UUID, but I think we would benefit form a more specific type than UUID or long.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [accumulo] milleruntime commented on issue #2473: Refactor FATE

Posted by GitBox <gi...@apache.org>.
milleruntime commented on issue #2473:
URL: https://github.com/apache/accumulo/issues/2473#issuecomment-1049143368


   For serialization of FATE repos, we could use JSON. There is already this in the FateCommand:
   https://github.com/apache/accumulo/blob/0fbf9cda0d1e16d67bf714970cb22dfa2907e50f/shell/src/main/java/org/apache/accumulo/shell/commands/fate/FateCommand.java#L85-L90


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [accumulo] milleruntime commented on issue #2473: Refactor FATE

Posted by GitBox <gi...@apache.org>.
milleruntime commented on issue #2473:
URL: https://github.com/apache/accumulo/issues/2473#issuecomment-1033212168


   One idea I had was to create an Environment interface to pass between FATE and the Manager. The tricky part is that some of the FATE code is in core (the interface, framework and some implementation) while most of it is in the same package as the Manger. I thought we could do this:
   <pre>
   public interface RepoEnv {
     public ServerContext getContext();
   }
   </pre>
   But that won't work because `ServerContext` is in server and Fate is in core.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [accumulo] EdColeman commented on issue #2473: Refactor FATE

Posted by GitBox <gi...@apache.org>.
EdColeman commented on issue #2473:
URL: https://github.com/apache/accumulo/issues/2473#issuecomment-1033170922


   Maybe this should consider https://github.com/apache/accumulo/issues/1249


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [accumulo] ctubbsii commented on issue #2473: Refactor FATE

Posted by GitBox <gi...@apache.org>.
ctubbsii commented on issue #2473:
URL: https://github.com/apache/accumulo/issues/2473#issuecomment-1033230922


   On serialization, Java 17 has the new record keyword. It should be safer to serialize records than general Java objects, as we've done in the past. I have not yet used records, and we aren't using Java 17 yet... but could for 3.0 development.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [accumulo] ctubbsii commented on issue #2473: Refactor FATE

Posted by GitBox <gi...@apache.org>.
ctubbsii commented on issue #2473:
URL: https://github.com/apache/accumulo/issues/2473#issuecomment-1054604913


   > @ctubbsii @keith-turner I have been experimenting with JSON serialization in FATE. There are already some classes to serialize the data for displaying with the Fate command. What do you guys think of trying to change the serialization to JSON in 2.1?
   
   I think that would probably be too big and too disruptive for 2.1 as we're trying to wrap up major features for 2.1. But, I'm not opposed to the idea, generally. However, I still think it's worth looking into newer Java serialization for record objects in 3.0.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [accumulo] milleruntime commented on issue #2473: Refactor FATE

Posted by GitBox <gi...@apache.org>.
milleruntime commented on issue #2473:
URL: https://github.com/apache/accumulo/issues/2473#issuecomment-1033177876


   If this was for 3.0, we could probably drop the old Bulk Import. That would help clean up a lot of code.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [accumulo] milleruntime commented on issue #2473: Refactor FATE

Posted by GitBox <gi...@apache.org>.
milleruntime commented on issue #2473:
URL: https://github.com/apache/accumulo/issues/2473#issuecomment-1039473784


   Another improvement that could be included in this change:
   Convert FateTxId to a stronger type to prevent coding errors like forgetting to call the method to display the value in HEX.
   The code would probably be 10x better if we had a FateTxId type that we used instead of long. Its toString method could consistently format for logging maybe. That could possible encapsulate or extend UUID, but I think we would benefit form a more specific type than UUID or long.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [accumulo] keith-turner commented on issue #2473: Refactor FATE

Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #2473:
URL: https://github.com/apache/accumulo/issues/2473#issuecomment-1033835874


   > It should be safer to serialize records than general Java objects
   
   Looking around for what we could do now to make things safer, found the following.  Maybe we could use the new ObjectInputFilter added in Java 9 to explicitly limit what classes FATE can deserialize.
   
   https://www.oracle.com/java/technologies/javase/seccodeguide.html#8
   https://docs.oracle.com/javase/9/docs/api/java/io/ObjectInputFilter.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [accumulo] milleruntime commented on issue #2473: Refactor FATE

Posted by GitBox <gi...@apache.org>.
milleruntime commented on issue #2473:
URL: https://github.com/apache/accumulo/issues/2473#issuecomment-1054328458


   @ctubbsii @keith-turner I have been experimenting with JSON serialization in FATE. There are already some classes to serialize the data for displaying with the Fate command. What do you guys think of trying to change the serialization to JSON in 2.1?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [accumulo] milleruntime commented on issue #2473: Refactor FATE

Posted by GitBox <gi...@apache.org>.
milleruntime commented on issue #2473:
URL: https://github.com/apache/accumulo/issues/2473#issuecomment-1054606615


   > > @ctubbsii @keith-turner I have been experimenting with JSON serialization in FATE. There are already some classes to serialize the data for displaying with the Fate command. What do you guys think of trying to change the serialization to JSON in 2.1?
   > 
   > I think that would probably be too big and too disruptive for 2.1
   
   After some experimentation, that was my feeling as well. The generics combined with the complexity of the different interfaces really make it difficult to make any small changes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org