You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@predictionio.apache.org by Pat Ferrel <pa...@occamsmachete.com> on 2016/08/02 14:47:02 UTC
Cleanup app
I’m trying to merge the AML fork and looked deeper into the cleanup app, which we turned into a optional feature of any Datasource.
In the cleanup app example it uses an RDD.collect, which is not scalable. It seems that PEvents does not really support delete so you have to get the `eventsToDelete` as an RDD then collect before you can actually delete and use LEvents to do it. Is this reasoning correct?
If so I think we will need get an RDD of `eventsToDelete` (done) then create a new method on PEvents that deletes them. Does this sound right?
Any advise on this would be helpful because I think it would block a merged release.
Re: Cleanup app
Posted by Pat Ferrel <pa...@occamsmachete.com>.
Thanks Alex. My pet feature :-)
On Aug 2, 2016, at 2:42 PM, Alex Merritt <em...@apache.org> wrote:
It does look like we do need a new delete method there.
I will take a look at this today.
On Tue, Aug 2, 2016 at 8:47 AM, Pat Ferrel <pa...@occamsmachete.com> wrote:
> I’m trying to merge the AML fork and looked deeper into the cleanup app,
> which we turned into a optional feature of any Datasource.
> In the cleanup app example it uses an RDD.collect, which is not scalable.
> It seems that PEvents does not really support delete so you have to get the
> `eventsToDelete` as an RDD then collect before you can actually delete and
> use LEvents to do it. Is this reasoning correct?
>
> If so I think we will need get an RDD of `eventsToDelete` (done) then
> create a new method on PEvents that deletes them. Does this sound right?
>
> Any advise on this would be helpful because I think it would block a
> merged release.
Re: Cleanup app
Posted by Alex Merritt <em...@apache.org>.
It does look like we do need a new delete method there.
I will take a look at this today.
On Tue, Aug 2, 2016 at 8:47 AM, Pat Ferrel <pa...@occamsmachete.com> wrote:
> I’m trying to merge the AML fork and looked deeper into the cleanup app,
> which we turned into a optional feature of any Datasource.
> In the cleanup app example it uses an RDD.collect, which is not scalable.
> It seems that PEvents does not really support delete so you have to get the
> `eventsToDelete` as an RDD then collect before you can actually delete and
> use LEvents to do it. Is this reasoning correct?
>
> If so I think we will need get an RDD of `eventsToDelete` (done) then
> create a new method on PEvents that deletes them. Does this sound right?
>
> Any advise on this would be helpful because I think it would block a
> merged release.