You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spot.apache.org by Alan Ross <al...@apache.org> on 2019/03/19 20:59:36 UTC
ODM Merge?
Hey team,
It's hard for people to find the ODM as it appears to be tied up in request
181. Can someone merge this? Not sure if we need to bring it for vote but I
support it being merged.
https://github.com/apache/incubator-spot/blob/SPOT-181_ODM/docs/open-data-model.md
Thanks, Alan
Re: ODM Merge?
Posted by Alan Ross <al...@apache.org>.
Awesome staff, thanks much!
On Thursday, March 21, 2019, Tadd Wood <ta...@digitalminion.com> wrote:
> Alan,
>
> I can help organize the open PRs. Right now the biggest barrier to
> merging in the ODM branch is bridging the gap between the ingest code and
> the ODM.
> @curtishoward did some great work in PR #144 using Envelope as the ingest
> framework for populating the ODM. I will reach out to see what work is
> left to finish up that PR so we can merge it in.
>
> Thank you,
> Tadd Wood
>
>
> > On Mar 21, 2019, at 4:02 PM, Alan Ross <al...@apache.org> wrote:
> >
> > thanks for the reply, Pierre-Luc.
> >
> > Any input on merging PRs? Is there a list of current open and which ones
> > have been reviewed?
> >
> > On Thu, Mar 21, 2019 at 1:10 PM Pierre-Luc Dion <pd...@apache.org>
> wrote:
> >
> >> Look like there few pending PR waiting to be merge to this branch,
> wouldn't
> >> it make sense to merge all that first, then merge SPOT-181_odm branch
> into
> >> master?
> >> I'm not committer so I can't help on that but I can help with review
> >> wherever it's possible.
> >>
> >> the PR pile look stalled, a lot of PR are becoming hold :-(
> >>
> >> On Tue, Mar 19, 2019 at 4:59 PM Alan Ross <al...@apache.org> wrote:
> >>
> >>> Hey team,
> >>>
> >>> It's hard for people to find the ODM as it appears to be tied up in
> >> request
> >>> 181. Can someone merge this? Not sure if we need to bring it for vote
> >> but I
> >>> support it being merged.
> >>>
> >>>
> >>>
> >> https://github.com/apache/incubator-spot/blob/SPOT-181_
> ODM/docs/open-data-model.md
> >>>
> >>> Thanks, Alan
> >>>
> >>
>
>
Re: ODM Merge?
Posted by Pierre-Luc Dion <pd...@apache.org>.
About this one, It seams it could confict with SPOT-181_ODM branch merge.
On Mon, Mar 25, 2019 at 12:59 PM Tadd Wood <ta...@digitalminion.com>
wrote:
> Alan,
>
> Below are the statuses of the currently open PRs. I can merge any of
> these once they’ve got enough votes or have been reviewed furthered by
> other committers. Also happy to have discussions on any of these PRs
> individually if anyone wants to start new discussions/threads in the Dev
> list.
>
> Ok to merge, but need more votes:
> PR Title
> Author
> PR Link
> Notes
> PR #153
> Updates to Spot Website Copyright Year
> @schonz
> https://github.com/apache/incubator-spot/pull/153 <
> https://github.com/apache/incubator-spot/pull/153>
>
> PR #151
> Fix sudo command, add help, fix ShellCheck warnings
> @pdion891
> https://github.com/apache/incubator-spot/pull/151 <
> https://github.com/apache/incubator-spot/pull/151>
>
> PR #147
> Updates to Spot Website
> @schonz
> https://github.com/apache/incubator-spot/pull/147 <
> https://github.com/apache/incubator-spot/pull/147>
>
> PR #144
> spot-ingest for ODM with config-driven Spark streaming (Envelope)
> @curtishoward
> https://github.com/apache/incubator-spot/pull/144 <
> https://github.com/apache/incubator-spot/pull/144>
>
> PR #143
> clean config options via configurator.py
> @natedogs911
> https://github.com/apache/incubator-spot/pull/143 <
> https://github.com/apache/incubator-spot/pull/143>
>
> PR #140
> odm event schema updates
> @TaddWood
> https://github.com/apache/incubator-spot/pull/140 <
> https://github.com/apache/incubator-spot/pull/140>
>
> PR #126
> Fix broken ingest_summary generation
> @castleguarders
> https://github.com/apache/incubator-spot/pull/126 <
> https://github.com/apache/incubator-spot/pull/126>
>
> PR #102
> Proxy Spot Schema
> @mpereaji
> https://github.com/apache/incubator-spot/pull/102 <
> https://github.com/apache/incubator-spot/pull/102>
>
>
>
>
>
>
> Awaiting updates/fixes from committer:
>
>
>
>
> PR #95
> Edited to match documentation
> @rphi
> https://github.com/apache/incubator-spot/pull/95 <
> https://github.com/apache/incubator-spot/pull/95/commits>
>
>
>
>
>
>
> Has merge conflicts:
>
>
>
>
> PR #25
> Improved hdfs_setup.sh
> @kpeiruza
> https://github.com/apache/incubator-spot/pull/25 <
> https://github.com/apache/incubator-spot/pull/25>
>
> PR #24
> aiming to close spot-23
> @natedogs911
> https://github.com/apache/incubator-spot/pull/24 <
> https://github.com/apache/incubator-spot/pull/24>
>
> PR #21
> Fix low hanging fruit in the documentation
> @gustavstickley
> https://github.com/apache/incubator-spot/pull/21 <
> https://github.com/apache/incubator-spot/pull/21>
>
>
>
>
>
>
> Needs to be broken into a new branch:
>
>
>
>
> PR #150
> Spot 181 odm
>
> ./ml_ops.sh 20181102 flow 5000 0.5
> The order of the two parameters needs to be changed.
> @tzhou2018
> https://github.com/apache/incubator-spot/pull/150 <
> https://github.com/apache/incubator-spot/pull/150>
> I think this PR was mistakenly trying to merge the ODM into the master
> branch.
>
>
>
>
>
> Need further review by other committers:
>
>
>
>
> PR #141
> Ingestion using Spark Streaming
> @ktzoulas
> https://github.com/apache/incubator-spot/pull/141 <
> https://github.com/apache/incubator-spot/pull/141>
>
> PR #149
> Inconsistencies in the open data model descriptions
> @cgiraldo
> https://github.com/apache/incubator-spot/pull/149 <
> https://github.com/apache/incubator-spot/pull/149>
>
>
> Thank you,
> Tadd Wood
>
> > On Mar 22, 2019, at 10:48 AM, Austin Leahy <Au...@digitalminion.com>
> wrote:
> >
> > I may be able to leverage some other stuff we have been doing lately to
> > close some ui gaps. Will get back in and poke around this afternoon.
> >
> > On Thu, Mar 21, 2019 at 10:17 PM Nate Smith <na...@gmail.com>
> wrote:
> >
> >> As I recall I think we can merge envelope without affecting the existing
> >> code. But as you pointed out there are other gaps such as the UI.
> >> At this point I say merge as much as we can (assuming no obvious code
> >> quality issues) as any movement at this point is positive. If something
> >> breaks then we know what needs to be fixed.
> >>
> >> - nathanael
> >>
> >>> On Mar 21, 2019, at 6:07 PM, Tadd Wood <ta...@digitalminion.com>
> >> wrote:
> >>>
> >>> Alan,
> >>>
> >>> I can help organize the open PRs. Right now the biggest barrier to
> >> merging in the ODM branch is bridging the gap between the ingest code
> and
> >> the ODM.
> >>> @curtishoward did some great work in PR #144 using Envelope as the
> >> ingest framework for populating the ODM. I will reach out to see what
> work
> >> is left to finish up that PR so we can merge it in.
> >>>
> >>> Thank you,
> >>> Tadd Wood
> >>>
> >>>
> >>>> On Mar 21, 2019, at 4:02 PM, Alan Ross <al...@apache.org> wrote:
> >>>>
> >>>> thanks for the reply, Pierre-Luc.
> >>>>
> >>>> Any input on merging PRs? Is there a list of current open and which
> ones
> >>>> have been reviewed?
> >>>>
> >>>>> On Thu, Mar 21, 2019 at 1:10 PM Pierre-Luc Dion <pdion891@apache.org
> >
> >> wrote:
> >>>>>
> >>>>> Look like there few pending PR waiting to be merge to this branch,
> >> wouldn't
> >>>>> it make sense to merge all that first, then merge SPOT-181_odm branch
> >> into
> >>>>> master?
> >>>>> I'm not committer so I can't help on that but I can help with review
> >>>>> wherever it's possible.
> >>>>>
> >>>>> the PR pile look stalled, a lot of PR are becoming hold :-(
> >>>>>
> >>>>>> On Tue, Mar 19, 2019 at 4:59 PM Alan Ross <al...@apache.org> wrote:
> >>>>>>
> >>>>>> Hey team,
> >>>>>>
> >>>>>> It's hard for people to find the ODM as it appears to be tied up in
> >>>>> request
> >>>>>> 181. Can someone merge this? Not sure if we need to bring it for
> vote
> >>>>> but I
> >>>>>> support it being merged.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>
> https://github.com/apache/incubator-spot/blob/SPOT-181_ODM/docs/open-data-model.md
> >>>>>>
> >>>>>> Thanks, Alan
> >>>>>>
> >>>>>
> >>>
> >>
>
>
Re: ODM Merge?
Posted by Tadd Wood <ta...@digitalminion.com>.
Alan,
Below are the statuses of the currently open PRs. I can merge any of these once they’ve got enough votes or have been reviewed furthered by other committers. Also happy to have discussions on any of these PRs individually if anyone wants to start new discussions/threads in the Dev list.
Ok to merge, but need more votes:
PR Title
Author
PR Link
Notes
PR #153
Updates to Spot Website Copyright Year
@schonz
https://github.com/apache/incubator-spot/pull/153 <https://github.com/apache/incubator-spot/pull/153>
PR #151
Fix sudo command, add help, fix ShellCheck warnings
@pdion891
https://github.com/apache/incubator-spot/pull/151 <https://github.com/apache/incubator-spot/pull/151>
PR #147
Updates to Spot Website
@schonz
https://github.com/apache/incubator-spot/pull/147 <https://github.com/apache/incubator-spot/pull/147>
PR #144
spot-ingest for ODM with config-driven Spark streaming (Envelope)
@curtishoward
https://github.com/apache/incubator-spot/pull/144 <https://github.com/apache/incubator-spot/pull/144>
PR #143
clean config options via configurator.py
@natedogs911
https://github.com/apache/incubator-spot/pull/143 <https://github.com/apache/incubator-spot/pull/143>
PR #140
odm event schema updates
@TaddWood
https://github.com/apache/incubator-spot/pull/140 <https://github.com/apache/incubator-spot/pull/140>
PR #126
Fix broken ingest_summary generation
@castleguarders
https://github.com/apache/incubator-spot/pull/126 <https://github.com/apache/incubator-spot/pull/126>
PR #102
Proxy Spot Schema
@mpereaji
https://github.com/apache/incubator-spot/pull/102 <https://github.com/apache/incubator-spot/pull/102>
Awaiting updates/fixes from committer:
PR #95
Edited to match documentation
@rphi
https://github.com/apache/incubator-spot/pull/95 <https://github.com/apache/incubator-spot/pull/95/commits>
Has merge conflicts:
PR #25
Improved hdfs_setup.sh
@kpeiruza
https://github.com/apache/incubator-spot/pull/25 <https://github.com/apache/incubator-spot/pull/25>
PR #24
aiming to close spot-23
@natedogs911
https://github.com/apache/incubator-spot/pull/24 <https://github.com/apache/incubator-spot/pull/24>
PR #21
Fix low hanging fruit in the documentation
@gustavstickley
https://github.com/apache/incubator-spot/pull/21 <https://github.com/apache/incubator-spot/pull/21>
Needs to be broken into a new branch:
PR #150
Spot 181 odm
./ml_ops.sh 20181102 flow 5000 0.5
The order of the two parameters needs to be changed.
@tzhou2018
https://github.com/apache/incubator-spot/pull/150 <https://github.com/apache/incubator-spot/pull/150>
I think this PR was mistakenly trying to merge the ODM into the master branch.
Need further review by other committers:
PR #141
Ingestion using Spark Streaming
@ktzoulas
https://github.com/apache/incubator-spot/pull/141 <https://github.com/apache/incubator-spot/pull/141>
PR #149
Inconsistencies in the open data model descriptions
@cgiraldo
https://github.com/apache/incubator-spot/pull/149 <https://github.com/apache/incubator-spot/pull/149>
Thank you,
Tadd Wood
> On Mar 22, 2019, at 10:48 AM, Austin Leahy <Au...@digitalminion.com> wrote:
>
> I may be able to leverage some other stuff we have been doing lately to
> close some ui gaps. Will get back in and poke around this afternoon.
>
> On Thu, Mar 21, 2019 at 10:17 PM Nate Smith <na...@gmail.com> wrote:
>
>> As I recall I think we can merge envelope without affecting the existing
>> code. But as you pointed out there are other gaps such as the UI.
>> At this point I say merge as much as we can (assuming no obvious code
>> quality issues) as any movement at this point is positive. If something
>> breaks then we know what needs to be fixed.
>>
>> - nathanael
>>
>>> On Mar 21, 2019, at 6:07 PM, Tadd Wood <ta...@digitalminion.com>
>> wrote:
>>>
>>> Alan,
>>>
>>> I can help organize the open PRs. Right now the biggest barrier to
>> merging in the ODM branch is bridging the gap between the ingest code and
>> the ODM.
>>> @curtishoward did some great work in PR #144 using Envelope as the
>> ingest framework for populating the ODM. I will reach out to see what work
>> is left to finish up that PR so we can merge it in.
>>>
>>> Thank you,
>>> Tadd Wood
>>>
>>>
>>>> On Mar 21, 2019, at 4:02 PM, Alan Ross <al...@apache.org> wrote:
>>>>
>>>> thanks for the reply, Pierre-Luc.
>>>>
>>>> Any input on merging PRs? Is there a list of current open and which ones
>>>> have been reviewed?
>>>>
>>>>> On Thu, Mar 21, 2019 at 1:10 PM Pierre-Luc Dion <pd...@apache.org>
>> wrote:
>>>>>
>>>>> Look like there few pending PR waiting to be merge to this branch,
>> wouldn't
>>>>> it make sense to merge all that first, then merge SPOT-181_odm branch
>> into
>>>>> master?
>>>>> I'm not committer so I can't help on that but I can help with review
>>>>> wherever it's possible.
>>>>>
>>>>> the PR pile look stalled, a lot of PR are becoming hold :-(
>>>>>
>>>>>> On Tue, Mar 19, 2019 at 4:59 PM Alan Ross <al...@apache.org> wrote:
>>>>>>
>>>>>> Hey team,
>>>>>>
>>>>>> It's hard for people to find the ODM as it appears to be tied up in
>>>>> request
>>>>>> 181. Can someone merge this? Not sure if we need to bring it for vote
>>>>> but I
>>>>>> support it being merged.
>>>>>>
>>>>>>
>>>>>>
>>>>>
>> https://github.com/apache/incubator-spot/blob/SPOT-181_ODM/docs/open-data-model.md
>>>>>>
>>>>>> Thanks, Alan
>>>>>>
>>>>>
>>>
>>
Re: ODM Merge?
Posted by Austin Leahy <Au...@digitalminion.com>.
I may be able to leverage some other stuff we have been doing lately to
close some ui gaps. Will get back in and poke around this afternoon.
On Thu, Mar 21, 2019 at 10:17 PM Nate Smith <na...@gmail.com> wrote:
> As I recall I think we can merge envelope without affecting the existing
> code. But as you pointed out there are other gaps such as the UI.
> At this point I say merge as much as we can (assuming no obvious code
> quality issues) as any movement at this point is positive. If something
> breaks then we know what needs to be fixed.
>
> - nathanael
>
> > On Mar 21, 2019, at 6:07 PM, Tadd Wood <ta...@digitalminion.com>
> wrote:
> >
> > Alan,
> >
> > I can help organize the open PRs. Right now the biggest barrier to
> merging in the ODM branch is bridging the gap between the ingest code and
> the ODM.
> > @curtishoward did some great work in PR #144 using Envelope as the
> ingest framework for populating the ODM. I will reach out to see what work
> is left to finish up that PR so we can merge it in.
> >
> > Thank you,
> > Tadd Wood
> >
> >
> >> On Mar 21, 2019, at 4:02 PM, Alan Ross <al...@apache.org> wrote:
> >>
> >> thanks for the reply, Pierre-Luc.
> >>
> >> Any input on merging PRs? Is there a list of current open and which ones
> >> have been reviewed?
> >>
> >>> On Thu, Mar 21, 2019 at 1:10 PM Pierre-Luc Dion <pd...@apache.org>
> wrote:
> >>>
> >>> Look like there few pending PR waiting to be merge to this branch,
> wouldn't
> >>> it make sense to merge all that first, then merge SPOT-181_odm branch
> into
> >>> master?
> >>> I'm not committer so I can't help on that but I can help with review
> >>> wherever it's possible.
> >>>
> >>> the PR pile look stalled, a lot of PR are becoming hold :-(
> >>>
> >>>> On Tue, Mar 19, 2019 at 4:59 PM Alan Ross <al...@apache.org> wrote:
> >>>>
> >>>> Hey team,
> >>>>
> >>>> It's hard for people to find the ODM as it appears to be tied up in
> >>> request
> >>>> 181. Can someone merge this? Not sure if we need to bring it for vote
> >>> but I
> >>>> support it being merged.
> >>>>
> >>>>
> >>>>
> >>>
> https://github.com/apache/incubator-spot/blob/SPOT-181_ODM/docs/open-data-model.md
> >>>>
> >>>> Thanks, Alan
> >>>>
> >>>
> >
>
Re: ODM Merge?
Posted by Nate Smith <na...@gmail.com>.
As I recall I think we can merge envelope without affecting the existing code. But as you pointed out there are other gaps such as the UI.
At this point I say merge as much as we can (assuming no obvious code quality issues) as any movement at this point is positive. If something breaks then we know what needs to be fixed.
- nathanael
> On Mar 21, 2019, at 6:07 PM, Tadd Wood <ta...@digitalminion.com> wrote:
>
> Alan,
>
> I can help organize the open PRs. Right now the biggest barrier to merging in the ODM branch is bridging the gap between the ingest code and the ODM.
> @curtishoward did some great work in PR #144 using Envelope as the ingest framework for populating the ODM. I will reach out to see what work is left to finish up that PR so we can merge it in.
>
> Thank you,
> Tadd Wood
>
>
>> On Mar 21, 2019, at 4:02 PM, Alan Ross <al...@apache.org> wrote:
>>
>> thanks for the reply, Pierre-Luc.
>>
>> Any input on merging PRs? Is there a list of current open and which ones
>> have been reviewed?
>>
>>> On Thu, Mar 21, 2019 at 1:10 PM Pierre-Luc Dion <pd...@apache.org> wrote:
>>>
>>> Look like there few pending PR waiting to be merge to this branch, wouldn't
>>> it make sense to merge all that first, then merge SPOT-181_odm branch into
>>> master?
>>> I'm not committer so I can't help on that but I can help with review
>>> wherever it's possible.
>>>
>>> the PR pile look stalled, a lot of PR are becoming hold :-(
>>>
>>>> On Tue, Mar 19, 2019 at 4:59 PM Alan Ross <al...@apache.org> wrote:
>>>>
>>>> Hey team,
>>>>
>>>> It's hard for people to find the ODM as it appears to be tied up in
>>> request
>>>> 181. Can someone merge this? Not sure if we need to bring it for vote
>>> but I
>>>> support it being merged.
>>>>
>>>>
>>>>
>>> https://github.com/apache/incubator-spot/blob/SPOT-181_ODM/docs/open-data-model.md
>>>>
>>>> Thanks, Alan
>>>>
>>>
>
Re: ODM Merge?
Posted by Tadd Wood <ta...@digitalminion.com>.
Alan,
I can help organize the open PRs. Right now the biggest barrier to merging in the ODM branch is bridging the gap between the ingest code and the ODM.
@curtishoward did some great work in PR #144 using Envelope as the ingest framework for populating the ODM. I will reach out to see what work is left to finish up that PR so we can merge it in.
Thank you,
Tadd Wood
> On Mar 21, 2019, at 4:02 PM, Alan Ross <al...@apache.org> wrote:
>
> thanks for the reply, Pierre-Luc.
>
> Any input on merging PRs? Is there a list of current open and which ones
> have been reviewed?
>
> On Thu, Mar 21, 2019 at 1:10 PM Pierre-Luc Dion <pd...@apache.org> wrote:
>
>> Look like there few pending PR waiting to be merge to this branch, wouldn't
>> it make sense to merge all that first, then merge SPOT-181_odm branch into
>> master?
>> I'm not committer so I can't help on that but I can help with review
>> wherever it's possible.
>>
>> the PR pile look stalled, a lot of PR are becoming hold :-(
>>
>> On Tue, Mar 19, 2019 at 4:59 PM Alan Ross <al...@apache.org> wrote:
>>
>>> Hey team,
>>>
>>> It's hard for people to find the ODM as it appears to be tied up in
>> request
>>> 181. Can someone merge this? Not sure if we need to bring it for vote
>> but I
>>> support it being merged.
>>>
>>>
>>>
>> https://github.com/apache/incubator-spot/blob/SPOT-181_ODM/docs/open-data-model.md
>>>
>>> Thanks, Alan
>>>
>>
Re: ODM Merge?
Posted by Alan Ross <al...@apache.org>.
thanks for the reply, Pierre-Luc.
Any input on merging PRs? Is there a list of current open and which ones
have been reviewed?
On Thu, Mar 21, 2019 at 1:10 PM Pierre-Luc Dion <pd...@apache.org> wrote:
> Look like there few pending PR waiting to be merge to this branch, wouldn't
> it make sense to merge all that first, then merge SPOT-181_odm branch into
> master?
> I'm not committer so I can't help on that but I can help with review
> wherever it's possible.
>
> the PR pile look stalled, a lot of PR are becoming hold :-(
>
> On Tue, Mar 19, 2019 at 4:59 PM Alan Ross <al...@apache.org> wrote:
>
> > Hey team,
> >
> > It's hard for people to find the ODM as it appears to be tied up in
> request
> > 181. Can someone merge this? Not sure if we need to bring it for vote
> but I
> > support it being merged.
> >
> >
> >
> https://github.com/apache/incubator-spot/blob/SPOT-181_ODM/docs/open-data-model.md
> >
> > Thanks, Alan
> >
>
Re: ODM Merge?
Posted by Pierre-Luc Dion <pd...@apache.org>.
Look like there few pending PR waiting to be merge to this branch, wouldn't
it make sense to merge all that first, then merge SPOT-181_odm branch into
master?
I'm not committer so I can't help on that but I can help with review
wherever it's possible.
the PR pile look stalled, a lot of PR are becoming hold :-(
On Tue, Mar 19, 2019 at 4:59 PM Alan Ross <al...@apache.org> wrote:
> Hey team,
>
> It's hard for people to find the ODM as it appears to be tied up in request
> 181. Can someone merge this? Not sure if we need to bring it for vote but I
> support it being merged.
>
>
> https://github.com/apache/incubator-spot/blob/SPOT-181_ODM/docs/open-data-model.md
>
> Thanks, Alan
>