You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by Hitesh Khamesra <hi...@yahoo.com.INVALID> on 2017/03/24 19:57:04 UTC
GEODE-2714 Proposal for new api on Function interface (Please read)
Please let us know your opinion on that.
Thanks.Hitesh
Re: GEODE-2714 Proposal for new api on Function interface (Please read)
Posted by Jacob Barrett <jb...@pivotal.io>.
Hey guys, take the conversation over to the ticket so context is maintained
with the ticket.
On Fri, Mar 24, 2017 at 3:16 PM Swapnil Bawaskar <sb...@pivotal.io>
wrote:
> For step 2 above, won't we end up recomputing the list of buckets?
>
> I agree, however, that we do have the problem of moving primary,
> especially in the middle of iterating over data. We already have an
> optimizeForWrite() method on Function that makes the function execute on
> the primary. Do we want to make locking part of this method, rather than
> introducing a new method?
>
> On Fri, Mar 24, 2017 at 2:10 PM Hitesh Khamesra <hi...@yahoo.com>
> wrote:
>
> > Hi Swapnil:
> >
> > Right, and then we execute that function on that node. Now assume during
> > that function execution time(first user thread) two things happens
> > simultaneously..
> >
> > 1. we move that bucket to other node.
> > 2. And other user thread execute function with same set of keys on other
> > node, which first thread is still executing.
> >
> > Basically we need to make sure while function execution we don't move the
> > primary bucket.
> >
> > Thanks.
> > Hitesh
> > ------------------------------
> > *From:* Swapnil Bawaskar <sb...@pivotal.io>
> > *To:* dev@geode.apache.org; Hitesh Khamesra <hi...@yahoo.com>
> > *Sent:* Friday, March 24, 2017 1:55 PM
> > *Subject:* Re: GEODE-2714 Proposal for new api on Function interface
> > (Please read)
> >
> > Here is the link: https://issues.apache.org/jira/browse/GEODE-2714
> >
> > Hi Hitesh,
> >
> > Before executing a function, we build a list of buckets that the function
> > needs to execute on and then we build the function context using only
> those
> > buckets. So, as long as the function is using the context to get the
> local
> > data set, it should not see the data twice, even if rebalance was in
> > progress.
> >
> > When you saw this behavior, was the function using the function context?
> >
> > Thanks!
> >
> >
> > On Fri, Mar 24, 2017 at 1:21 PM Jared Stewart <js...@pivotal.io>
> wrote:
> >
> > Can you give a link to the proposal?
> >
> > Thanks,
> > Jared
> > > On Mar 24, 2017, at 12:57 PM, Hitesh Khamesra
> > <hi...@yahoo.com.INVALID> wrote:
> > >
> > > Please let us know your opinion on that.
> > > Thanks.Hitesh
> >
> >
> >
> >
>
Re: GEODE-2714 Proposal for new api on Function interface (Please read)
Posted by Swapnil Bawaskar <sb...@pivotal.io>.
For step 2 above, won't we end up recomputing the list of buckets?
I agree, however, that we do have the problem of moving primary,
especially in the middle of iterating over data. We already have an
optimizeForWrite() method on Function that makes the function execute on
the primary. Do we want to make locking part of this method, rather than
introducing a new method?
On Fri, Mar 24, 2017 at 2:10 PM Hitesh Khamesra <hi...@yahoo.com> wrote:
> Hi Swapnil:
>
> Right, and then we execute that function on that node. Now assume during
> that function execution time(first user thread) two things happens
> simultaneously..
>
> 1. we move that bucket to other node.
> 2. And other user thread execute function with same set of keys on other
> node, which first thread is still executing.
>
> Basically we need to make sure while function execution we don't move the
> primary bucket.
>
> Thanks.
> Hitesh
> ------------------------------
> *From:* Swapnil Bawaskar <sb...@pivotal.io>
> *To:* dev@geode.apache.org; Hitesh Khamesra <hi...@yahoo.com>
> *Sent:* Friday, March 24, 2017 1:55 PM
> *Subject:* Re: GEODE-2714 Proposal for new api on Function interface
> (Please read)
>
> Here is the link: https://issues.apache.org/jira/browse/GEODE-2714
>
> Hi Hitesh,
>
> Before executing a function, we build a list of buckets that the function
> needs to execute on and then we build the function context using only those
> buckets. So, as long as the function is using the context to get the local
> data set, it should not see the data twice, even if rebalance was in
> progress.
>
> When you saw this behavior, was the function using the function context?
>
> Thanks!
>
>
> On Fri, Mar 24, 2017 at 1:21 PM Jared Stewart <js...@pivotal.io> wrote:
>
> Can you give a link to the proposal?
>
> Thanks,
> Jared
> > On Mar 24, 2017, at 12:57 PM, Hitesh Khamesra
> <hi...@yahoo.com.INVALID> wrote:
> >
> > Please let us know your opinion on that.
> > Thanks.Hitesh
>
>
>
>
Re: GEODE-2714 Proposal for new api on Function interface (Please
read)
Posted by Hitesh Khamesra <hi...@yahoo.com.INVALID>.
Hi Swapnil:
Right, and then we execute that function on that node. Now assume during that function execution time(first user thread) two things happens simultaneously..
1. we move that bucket to other node.
2. And other user thread execute function with same set of keys on other node, which first thread is still executing.
Basically we need to make sure while function execution we don't move the primary bucket.
Thanks.Hitesh
From: Swapnil Bawaskar <sb...@pivotal.io>
To: dev@geode.apache.org; Hitesh Khamesra <hi...@yahoo.com>
Sent: Friday, March 24, 2017 1:55 PM
Subject: Re: GEODE-2714 Proposal for new api on Function interface (Please read)
Here is the link: https://issues.apache.org/jira/browse/GEODE-2714
Hi Hitesh,
Before executing a function, we build a list of buckets that the function needs to execute on and then we build the function context using only those buckets. So, as long as the function is using the context to get the local data set, it should not see the data twice, even if rebalance was in progress.
When you saw this behavior, was the function using the function context?
Thanks!
On Fri, Mar 24, 2017 at 1:21 PM Jared Stewart <js...@pivotal.io> wrote:
Can you give a link to the proposal?
Thanks,
Jared
> On Mar 24, 2017, at 12:57 PM, Hitesh Khamesra <hi...@yahoo.com.INVALID> wrote:
>
> Please let us know your opinion on that.
> Thanks.Hitesh
Re: GEODE-2714 Proposal for new api on Function interface (Please read)
Posted by Swapnil Bawaskar <sb...@pivotal.io>.
Here is the link: https://issues.apache.org/jira/browse/GEODE-2714
Hi Hitesh,
Before executing a function, we build a list of buckets that the function
needs to execute on and then we build the function context using only those
buckets. So, as long as the function is using the context to get the local
data set, it should not see the data twice, even if rebalance was in
progress.
When you saw this behavior, was the function using the function context?
Thanks!
On Fri, Mar 24, 2017 at 1:21 PM Jared Stewart <js...@pivotal.io> wrote:
> Can you give a link to the proposal?
>
> Thanks,
> Jared
> > On Mar 24, 2017, at 12:57 PM, Hitesh Khamesra
> <hi...@yahoo.com.INVALID> wrote:
> >
> > Please let us know your opinion on that.
> > Thanks.Hitesh
>
>
Re: GEODE-2714 Proposal for new api on Function interface (Please
read)
Posted by Jared Stewart <js...@pivotal.io>.
Can you give a link to the proposal?
Thanks,
Jared
> On Mar 24, 2017, at 12:57 PM, Hitesh Khamesra <hi...@yahoo.com.INVALID> wrote:
>
> Please let us know your opinion on that.
> Thanks.Hitesh