Contribution to apache gossip

[Ankur Agrawal <an...@gmail.com>]

Hi

Not sure if my previous email one week ago did actually get delivered. So I
apologize for resenting in advance.

My name is Ankur. I am a student, and I am keenly interested in, and want
to contribute to this project for implementation of Gossip protocol.

I have some job experience in designing instance migration services for a
popular public cloud infrastructure, and distributed data backup and
recovery services. I think this project will help me learn a lot, and I can
also contribute while learning.

I saw that GOSSIP-70 (https://issues.apache.org/jira/browse/GOSSIP-70) is
still open. It was mentioned as a project in GSoC 2017 too. If this is
indeed open, and you guys think I can start with this, it will be great.
Please note that I am not participating in GSoC and I want to contribute on
a individual capacity.

If there are other open issues where you are looking for contribution too,
do let me know, and I can get started with that.

Also, I am wondering if there is any interest in implementation of Gossip
in other languages like python or C++.

I look forward to hear from you guys.

Thanks
Ankur

Re: Contribution to apache gossip

[Ankur Agrawal <an...@gmail.com>]

Hi

I have made some progress with the design and implementation for tickets
related to GOSSIP-70. I looked at all the 5 sub tickets for GOSSIP-70,
because thinking about them together helped me with clearly understanding
the requirement for each ticket. I am writing the details for each ticket
below as per my understanding, so that we are at same page before I start
writing the code.

*GOSSIP-94: Create latitude and longitude based targeting*
I have a basic implementation for this ready (this was simple enough).
However since this ticket builds over GOSSIP-93, I have a question about
this. The generateTargets method of OperationTargets class takes
List<LocalMember>
living  as one of the arguments. Latitude and longitude based targeting
will make sense only if LocalMember class' properties map has key-values
latitude and longitude. So I thing a good solution might look like following

1. Create a subclass of LocalMember, called LocalMemberWithCoordinates
(pardon lack of a better name for now)
2. LocalMemberWithCoordinates has a constructor that takes co-ordinates as
one of the parameters
3. The function generateTargets takes List<? extends LocalMember> living
and List<? extends LocalMember> living as arguments (instead of
List<LocalMember> living and List<LocalMember> dead)

Please let me know if this makes sense.

*GOSSIP-95: Create a result merger*
The ResultMerger needs to work on List<Response>. It will be called with
this list of response objects by the co-ordinator, once the co-ordinator
hears back from all the executors. The response class that we have doesn't
have any structure. We will need to create a subclass, (lets call it
RWResponse for now). This subclass will have key value pair representing
the data returned by nodes for the request with key.


*GOSSIP-96: Create a multi-node request co-ordinator*
The request co-ordinator will issue requests to multiple nodes with futures
and block for n-results.
1. We can create ReadRequestMessages and WriteRequestMessages classes, as
subclasses of Message. These messages can be sent using gossipCore.send
method.
2. For the receiver side, we need to implement ReadRequestHandler and
WriteRequestHandler. Since the concrete implementation for this will be
given by the application using ApacheGossip, we can provide a abstract
implementation for now. The concrete implementations provided by the
application can inherit from the abstract implementation.
3. The Response objects created from the ticket GOSSIP-95 can be used to
return a response.


*GOSSIP-97: Implement model and message handler*
This is already covered above.

*GOSSIP-70: Build a GeoMetric/GridQuorum model*
Create a read and write function as mentioned by you earlier, with the
following signatures:
void write( String key, SharedGossipMessage message, Operationtargets c,
Consistency con)
Object read(String key, OperationTargets c, Consistency con)

Write Example
1. One getting a write request, we can use the generateTargets of
"OperationTargets c" object to get the list of members we want to send
request to.
2. Send a request to the co-ordinator with the list of target members,
message and key.
3. Apply the consistency model on the received responses.
4. Merge and send the results to the user.

Thanks
Ankur


On Fri, Aug 18, 2017 at 10:00 AM, Ankur Agrawal <an...@gmail.com>
wrote:

> Hi Edward
>
> Yes I saw that. I have made some initial outline and notes about the
> implementation that I was planning to share with you, but unfortunately I
> got sidetracked by some assignments and exams.
> I am planning to start working on it from Wednesday again. I have free
> time this week starting Wednesday when I can make good progress.
>
> Please let me know if this timeline looks ok to you.
>
> Thanks
> Ankur
>
> On Fri, Aug 18, 2017 at 8:41 AM Edward Capriolo <ed...@gmail.com>
> wrote:
>
>> You may have noticed that I broke out the task into an umbrella task. I
>> submitted a PR with one possibly implementation of implementing a "geo"
>> node chooser and created some tickets to add others. I also hinted at some
>> ways of doing the other components.
>>
>> On Thu, Aug 10, 2017 at 10:48 PM, Edward Capriolo <ed...@gmail.com>
>> wrote:
>>
>>>
>>> On Thu, Aug 10, 2017 at 4:27 AM, Ankur Agrawal <
>>> ankuragrawal004@gmail.com> wrote:
>>>
>>>> Hi Edward
>>>>
>>>> Thanks for sharing your notes.
>>>>
>>>> Most of what you have said makes sense to me as far as functionality is
>>>> concerned. I need to take a stab at understanding the code flow again, and
>>>> think through the entire flow, which might take a day or two. After that I
>>>> will come up with a more detailed design, and maybe more questions :)
>>>>
>>>> Thanks
>>>> Ankur
>>>>
>>>> On Wed, Aug 9, 2017 at 6:17 PM, Edward Capriolo <ed...@gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wed, Aug 9, 2017 at 6:32 PM, Ankur Agrawal <
>>>>> ankuragrawal004@gmail.com> wrote:
>>>>>
>>>>>> Hi Edward,
>>>>>>
>>>>>> Thanks for your response. I spent sometime understanding about the
>>>>>> Gossip protocol, and the apache-gossip codebase.
>>>>>>
>>>>>> About Gossip-70, I am trying to understand the requirement.
>>>>>>
>>>>>> Does this look like a good blueprint to you:
>>>>>>
>>>>>>    - Each node is initialized with a n-dimensional co-ordinate.
>>>>>>    - When a node creates its membership list, it gets the
>>>>>>    co-ordinates of its members.
>>>>>>    - Each node can draw a logical graph based on the co-ordinates of
>>>>>>    its members.
>>>>>>    - This topology can be used by the application that is using
>>>>>>    gossip, to forward its updates/messages and to decide quorum based on a
>>>>>>    formula.
>>>>>>
>>>>>> Do you think my understanding of basic requirements is correct?
>>>>>>
>>>>>> Thanks
>>>>>> Ankur
>>>>>>
>>>>>> On Mon, Jul 31, 2017 at 7:58 AM, Edward Capriolo <
>>>>>> edlinuxguru@gmail.com> wrote:
>>>>>>
>>>>>>> On Sun, Jul 30, 2017 at 10:59 PM, Ankur Agrawal <
>>>>>>> ankuragrawal004@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> > Hi
>>>>>>> >
>>>>>>> > Not sure if my previous email one week ago did actually get
>>>>>>> delivered. So I
>>>>>>> > apologize for resenting in advance.
>>>>>>> >
>>>>>>> > My name is Ankur. I am a student, and I am keenly interested in,
>>>>>>> and want
>>>>>>> > to contribute to this project for implementation of Gossip
>>>>>>> protocol.
>>>>>>> >
>>>>>>> > I have some job experience in designing instance migration
>>>>>>> services for a
>>>>>>> > popular public cloud infrastructure, and distributed data backup
>>>>>>> and
>>>>>>> > recovery services. I think this project will help me learn a lot,
>>>>>>> and I can
>>>>>>> > also contribute while learning.
>>>>>>> >
>>>>>>> > I saw that GOSSIP-70 (https://issues.apache.org/
>>>>>>> jira/browse/GOSSIP-70) is
>>>>>>> > still open. It was mentioned as a project in GSoC 2017 too. If
>>>>>>> this is
>>>>>>> > indeed open, and you guys think I can start with this, it will be
>>>>>>> great.
>>>>>>> > Please note that I am not participating in GSoC and I want to
>>>>>>> contribute on
>>>>>>> > a individual capacity.
>>>>>>> >
>>>>>>> > If there are other open issues where you are looking for
>>>>>>> contribution too,
>>>>>>> > do let me know, and I can get started with that.
>>>>>>> >
>>>>>>> > Also, I am wondering if there is any interest in implementation of
>>>>>>> Gossip
>>>>>>> > in other languages like python or C++.
>>>>>>> >
>>>>>>> > I look forward to hear from you guys.
>>>>>>> >
>>>>>>> > Thanks
>>>>>>> > Ankur
>>>>>>> >
>>>>>>>
>>>>>>> Thanks for your interest!
>>>>>>>
>>>>>>> "Also, I am wondering if there is any interest in implementation of
>>>>>>> Gossip
>>>>>>> in other languages like python or C++."
>>>>>>>
>>>>>>> Originally we planned to port Gossip to multiple languages but there
>>>>>>> has
>>>>>>> not been much interest in that. Currently the code base is evolving
>>>>>>> and
>>>>>>> porting would be challenging. I think that is a lot for us to bite
>>>>>>> off at
>>>>>>> this point.
>>>>>>>
>>>>>>> GOSSIP-70 is a fine place to start. We have not fleshed that ticket
>>>>>>> out
>>>>>>> enough. Starting a conversation in the ticket or on the mailing list
>>>>>>> would
>>>>>>> be a good thing to do before we start coding. I like the idea of
>>>>>>> using a
>>>>>>> formula to pick a quorum of nodes and doing an operation across
>>>>>>> them. This
>>>>>>> can be easily extracted but we need to figure out what the first
>>>>>>> step is
>>>>>>> and what a final target might be.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Edward
>>>>>>>
>>>>>>
>>>>> Your understanding is mostly in line with my thinking.  I will started
>>>>> some high level design but do not have the interfaces at a point such that I
>>>>> was ready for a commit. I will share my notes.
>>>>>
>>>>> >> Each node is initialized with a n-dimensional co-ordinate.
>>>>>
>>>>> That is one route but we can go more simple then that. We need an
>>>>> interface:
>>>>>
>>>>> interface chooser {
>>>>>    public List<LocalNode> generateEndpoints(string key,
>>>>> List<LocalNode> live, List<LocalNode> dead, LocalNode me);
>>>>> }
>>>>>
>>>>> The most simple implementation is we can hash the node's id into a
>>>>> line, or circular queue. We can then hash the key and chose nodes.
>>>>> Something like this: http://docs.basho.com/images/dynamo/figure2.png
>>>>>
>>>>> Of course this opens us up to other implementations. Nodes each have a
>>>>> Map of properties. We could easily make other implementations that are more
>>>>> advanced. A follow up would be datacenter specific quorum, another could me
>>>>> one that uses latitude and longitude and chooses the closest nodes.
>>>>>
>>>>> >> This topology can be used by the application that is using gossip,
>>>>> to forward its updates/messages and to decide quorum based on a formula.
>>>>>
>>>>> We already have support for controlling replication control
>>>>> https://issues.apache.org/jira/browse/GOSSIP-88 . These two features
>>>>> can be used together. A write request is sent to three nodes which reply
>>>>> instantly and replicate the data out lazily to other nodes. Or possibly you
>>>>> can actively read and write to two nodes only and chose to not replicate
>>>>> the data to any other nodes.
>>>>>
>>>>> To wrap that all up we can add a method to GossipManager that looks
>>>>> like this:
>>>>>
>>>>> void quorumWrite( String key, SharedGossipMessage message, Chooser c,
>>>>> Consistency con)
>>>>> Crdt quorumRead(String key, Chooser c, Consistency con)
>>>>>
>>>>> These methods could work something like this:
>>>>>
>>>>> https://github.com/edwardcapriolo/nibiru/blob/master/src/main/java/io/
>>>>> teknek/nibiru/coordinator/EventualCoordinator.java#L87
>>>>>
>>>>> Where we use executors to launch N simultanios requests and block for
>>>>> responses. We also need to build a new message object and event handlers
>>>>> for these type of requests.
>>>>>
>>>>> We obviously need to flesh some specifics out as we go on but that was
>>>>> what I was thinking implementation wise.
>>>>>
>>>>>
>>>>
>>> We can work together on this one. I will convert the Jira  into an
>>> umbrella task and make some subtasks for each of the component parts that
>>> need to be changed, I have a pretty solid idea of how about the majority of
>>> the feature would work, and if I mark out all the touch points you should
>>> have an easier time navigating the codebase.
>>>
>>>
>>>
>>

Re: Contribution to apache gossip

[Edward Capriolo <ed...@gmail.com>]

You may have noticed that I broke out the task into an umbrella task. I
submitted a PR with one possibly implementation of implementing a "geo"
node chooser and created some tickets to add others. I also hinted at some
ways of doing the other components.

On Thu, Aug 10, 2017 at 10:48 PM, Edward Capriolo <ed...@gmail.com>
wrote:

>
> On Thu, Aug 10, 2017 at 4:27 AM, Ankur Agrawal <an...@gmail.com>
> wrote:
>
>> Hi Edward
>>
>> Thanks for sharing your notes.
>>
>> Most of what you have said makes sense to me as far as functionality is
>> concerned. I need to take a stab at understanding the code flow again, and
>> think through the entire flow, which might take a day or two. After that I
>> will come up with a more detailed design, and maybe more questions :)
>>
>> Thanks
>> Ankur
>>
>> On Wed, Aug 9, 2017 at 6:17 PM, Edward Capriolo <ed...@gmail.com>
>> wrote:
>>
>>>
>>>
>>> On Wed, Aug 9, 2017 at 6:32 PM, Ankur Agrawal <ankuragrawal004@gmail.com
>>> > wrote:
>>>
>>>> Hi Edward,
>>>>
>>>> Thanks for your response. I spent sometime understanding about the
>>>> Gossip protocol, and the apache-gossip codebase.
>>>>
>>>> About Gossip-70, I am trying to understand the requirement.
>>>>
>>>> Does this look like a good blueprint to you:
>>>>
>>>>    - Each node is initialized with a n-dimensional co-ordinate.
>>>>    - When a node creates its membership list, it gets the co-ordinates
>>>>    of its members.
>>>>    - Each node can draw a logical graph based on the co-ordinates of
>>>>    its members.
>>>>    - This topology can be used by the application that is using
>>>>    gossip, to forward its updates/messages and to decide quorum based on a
>>>>    formula.
>>>>
>>>> Do you think my understanding of basic requirements is correct?
>>>>
>>>> Thanks
>>>> Ankur
>>>>
>>>> On Mon, Jul 31, 2017 at 7:58 AM, Edward Capriolo <edlinuxguru@gmail.com
>>>> > wrote:
>>>>
>>>>> On Sun, Jul 30, 2017 at 10:59 PM, Ankur Agrawal <
>>>>> ankuragrawal004@gmail.com>
>>>>> wrote:
>>>>>
>>>>> > Hi
>>>>> >
>>>>> > Not sure if my previous email one week ago did actually get
>>>>> delivered. So I
>>>>> > apologize for resenting in advance.
>>>>> >
>>>>> > My name is Ankur. I am a student, and I am keenly interested in, and
>>>>> want
>>>>> > to contribute to this project for implementation of Gossip protocol.
>>>>> >
>>>>> > I have some job experience in designing instance migration services
>>>>> for a
>>>>> > popular public cloud infrastructure, and distributed data backup and
>>>>> > recovery services. I think this project will help me learn a lot,
>>>>> and I can
>>>>> > also contribute while learning.
>>>>> >
>>>>> > I saw that GOSSIP-70 (https://issues.apache.org/jir
>>>>> a/browse/GOSSIP-70) is
>>>>> > still open. It was mentioned as a project in GSoC 2017 too. If this
>>>>> is
>>>>> > indeed open, and you guys think I can start with this, it will be
>>>>> great.
>>>>> > Please note that I am not participating in GSoC and I want to
>>>>> contribute on
>>>>> > a individual capacity.
>>>>> >
>>>>> > If there are other open issues where you are looking for
>>>>> contribution too,
>>>>> > do let me know, and I can get started with that.
>>>>> >
>>>>> > Also, I am wondering if there is any interest in implementation of
>>>>> Gossip
>>>>> > in other languages like python or C++.
>>>>> >
>>>>> > I look forward to hear from you guys.
>>>>> >
>>>>> > Thanks
>>>>> > Ankur
>>>>> >
>>>>>
>>>>> Thanks for your interest!
>>>>>
>>>>> "Also, I am wondering if there is any interest in implementation of
>>>>> Gossip
>>>>> in other languages like python or C++."
>>>>>
>>>>> Originally we planned to port Gossip to multiple languages but there
>>>>> has
>>>>> not been much interest in that. Currently the code base is evolving and
>>>>> porting would be challenging. I think that is a lot for us to bite off
>>>>> at
>>>>> this point.
>>>>>
>>>>> GOSSIP-70 is a fine place to start. We have not fleshed that ticket out
>>>>> enough. Starting a conversation in the ticket or on the mailing list
>>>>> would
>>>>> be a good thing to do before we start coding. I like the idea of using
>>>>> a
>>>>> formula to pick a quorum of nodes and doing an operation across them.
>>>>> This
>>>>> can be easily extracted but we need to figure out what the first step
>>>>> is
>>>>> and what a final target might be.
>>>>>
>>>>> Thanks,
>>>>> Edward
>>>>>
>>>>
>>> Your understanding is mostly in line with my thinking.  I will started
>>> some high level design but do not have the interfaces at a point such that I
>>> was ready for a commit. I will share my notes.
>>>
>>> >> Each node is initialized with a n-dimensional co-ordinate.
>>>
>>> That is one route but we can go more simple then that. We need an
>>> interface:
>>>
>>> interface chooser {
>>>    public List<LocalNode> generateEndpoints(string key, List<LocalNode>
>>> live, List<LocalNode> dead, LocalNode me);
>>> }
>>>
>>> The most simple implementation is we can hash the node's id into a line,
>>> or circular queue. We can then hash the key and chose nodes. Something like
>>> this: http://docs.basho.com/images/dynamo/figure2.png
>>>
>>> Of course this opens us up to other implementations. Nodes each have a
>>> Map of properties. We could easily make other implementations that are more
>>> advanced. A follow up would be datacenter specific quorum, another could me
>>> one that uses latitude and longitude and chooses the closest nodes.
>>>
>>> >> This topology can be used by the application that is using gossip, to
>>> forward its updates/messages and to decide quorum based on a formula.
>>>
>>> We already have support for controlling replication control
>>> https://issues.apache.org/jira/browse/GOSSIP-88 . These two features
>>> can be used together. A write request is sent to three nodes which reply
>>> instantly and replicate the data out lazily to other nodes. Or possibly you
>>> can actively read and write to two nodes only and chose to not replicate
>>> the data to any other nodes.
>>>
>>> To wrap that all up we can add a method to GossipManager that looks like
>>> this:
>>>
>>> void quorumWrite( String key, SharedGossipMessage message, Chooser c,
>>> Consistency con)
>>> Crdt quorumRead(String key, Chooser c, Consistency con)
>>>
>>> These methods could work something like this:
>>>
>>> https://github.com/edwardcapriolo/nibiru/blob/master/src/mai
>>> n/java/io/teknek/nibiru/coordinator/EventualCoordinator.java#L87
>>>
>>> Where we use executors to launch N simultanios requests and block for
>>> responses. We also need to build a new message object and event handlers
>>> for these type of requests.
>>>
>>> We obviously need to flesh some specifics out as we go on but that was
>>> what I was thinking implementation wise.
>>>
>>>
>>
> We can work together on this one. I will convert the Jira  into an
> umbrella task and make some subtasks for each of the component parts that
> need to be changed, I have a pretty solid idea of how about the majority of
> the feature would work, and if I mark out all the touch points you should
> have an easier time navigating the codebase.
>
>
>

Re: Contribution to apache gossip

[Edward Capriolo <ed...@gmail.com>]

On Sun, Jul 30, 2017 at 10:59 PM, Ankur Agrawal <an...@gmail.com>
wrote:

> Hi
>
> Not sure if my previous email one week ago did actually get delivered. So I
> apologize for resenting in advance.
>
> My name is Ankur. I am a student, and I am keenly interested in, and want
> to contribute to this project for implementation of Gossip protocol.
>
> I have some job experience in designing instance migration services for a
> popular public cloud infrastructure, and distributed data backup and
> recovery services. I think this project will help me learn a lot, and I can
> also contribute while learning.
>
> I saw that GOSSIP-70 (https://issues.apache.org/jira/browse/GOSSIP-70) is
> still open. It was mentioned as a project in GSoC 2017 too. If this is
> indeed open, and you guys think I can start with this, it will be great.
> Please note that I am not participating in GSoC and I want to contribute on
> a individual capacity.
>
> If there are other open issues where you are looking for contribution too,
> do let me know, and I can get started with that.
>
> Also, I am wondering if there is any interest in implementation of Gossip
> in other languages like python or C++.
>
> I look forward to hear from you guys.
>
> Thanks
> Ankur
>

Thanks for your interest!

"Also, I am wondering if there is any interest in implementation of Gossip
in other languages like python or C++."

Originally we planned to port Gossip to multiple languages but there has
not been much interest in that. Currently the code base is evolving and
porting would be challenging. I think that is a lot for us to bite off at
this point.

GOSSIP-70 is a fine place to start. We have not fleshed that ticket out
enough. Starting a conversation in the ticket or on the mailing list would
be a good thing to do before we start coding. I like the idea of using a
formula to pick a quorum of nodes and doing an operation across them. This
can be easily extracted but we need to figure out what the first step is
and what a final target might be.

Thanks,
Edward