You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@myriad.apache.org by Swapnil Daingade <sw...@gmail.com> on 2015/08/05 01:44:17 UTC
Myriad 0.1 release scope
Hi all,
Was wondering what would be the scope for the Myriad 0.1 release.
It would be nice to have a roadmap page somewhere and target
features to releases (JIRA 'fix version' field perhaps)
Regards
Swapnil
Re: Myriad 0.1 release scope
Posted by Santosh Marella <sm...@maprtech.com>.
Not sure it's that simple. I think this requires reflection based API
invocation based on hadoop version. Also the hadoop version itself needs to
be passed from build file into Java code.
Hadoop eco system components like Hive adopt a technique called shim
loaders to solve this type of problems.
----------------------
Sent from mobile
On Aug 7, 2015 3:54 PM, "Darin Johnson" <db...@gmail.com> wrote:
> So I compiled the 2.5 fgs against 2.6 when I was testing. If we abstract
> this right it may just be an if statement or two.
> On Aug 7, 2015 6:47 PM, "Santosh Marella" <sm...@maprtech.com> wrote:
>
> > > Myriad code base compiled against hadoop 2.7 should work on hadoop 2.5
> > > cluster as long as FGS (i.e. zero profile NM) is not used.
> >
> > Verified the above. As long as FGS (zero profile NM) is not used,
> > Myriad compiled against hadoop 2.7 will work on hadoop 2.5.
> >
> > Thanks,
> > Santosh
> >
> > On Fri, Aug 7, 2015 at 2:20 PM, Santosh Marella <sm...@maprtech.com>
> > wrote:
> >
> > > > It will make working on HA easier
> > > Oh Yes!
> > >
> > > > how do we facilitate that? Profiles?
> > > Profiles might be one way. Currently, FGS is supported for "zero"
> profile
> > > only.
> > > And we have seen there was an API incompatibility from 2.5 to 2.6+ in
> FGS
> > > code.
> > > So, ideally (since I haven't tried it myself), when FGS is merged into
> > > phase1,
> > > the Myriad code base compiled against hadoop 2.7 should work on hadoop
> > 2.5
> > > cluster as long as FGS (i.e. zero profile NM) is not used. (I'll try
> this
> > > out and
> > > post back what I find)
> > >
> > > However, in the long term we need a mechanism to abstract out the APIs
> > > that are incompatible across versions.
> > >
> > > Thanks,
> > > Santosh
> > >
> > > On Fri, Aug 7, 2015 at 12:12 PM, Darin Johnson <
> dbjohnson1978@gmail.com>
> > > wrote:
> > >
> > >> It will make working on HA easier. However, one concern that's been
> > >> addressed previously is that FGS works for Hadoop 2.6.0+. Do we plan
> to
> > >> support 2.5.X (anything lower?) also as Santosh has a way to do that,
> if
> > >> so
> > >> how do we facilitate that? Profiles?
> > >>
> > >> Darin
> > >>
> > >> On Fri, Aug 7, 2015 at 1:46 PM, Santosh Marella <
> smarella@maprtech.com>
> > >> wrote:
> > >>
> > >> > Hello guys,
> > >> >
> > >> > I propose merging FGS into phase1. As I said before, I think it's
> at a
> > >> > point where the functionality works reasonably well.
> > >> > Any future improvements/fixes/UI changes can be done via separate
> > JIRAs.
> > >> >
> > >> > Unless there are any major concerns, I'd like to merge FGS into
> phase1
> > >> *EOD
> > >> > Monday* (PDT).
> > >> >
> > >> > Thanks,
> > >> > Santosh
> > >> >
> > >> > On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella <
> > smarella@maprtech.com>
> > >> > wrote:
> > >> >
> > >> > > I feel FGS is very close to making it into 0.1. PR 116 addresses
> > >> moving
> > >> > to
> > >> > > hadoop 2.7 and making FGS and CGS coexist. This PR was recently
> > >> reviewed
> > >> > by
> > >> > > Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and
> > >> 2.7.x
> > >> > > clusters and it seemed to have worked as expected. Unless there
> are
> > >> more
> > >> > > reviews/feedback, it can be merged into issue_14. Once PR 116 is
> > >> merged
> > >> > > into issue_14, issue_14 can be merged into phase1.
> > >> > >
> > >> > > Thanks,
> > >> > > Santosh
> > >> > >
> > >> > > On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <adam@mesosphere.io
> >
> > >> > wrote:
> > >> > >
> > >> > >> We do have a JIRA 0.1.0 "fix version" field, but none of our
> issues
> > >> use
> > >> > it
> > >> > >> yet.
> > >> > >> I think the goal was just to take what we have and make it work
> > under
> > >> > >> Apache infrastructure, then vote on that for 0.1.0.
> > >> > >> Although other features like HA or FGS would be great, let's try
> to
> > >> get
> > >> > >> our
> > >> > >> first Apache release out ASAP.
> > >> > >> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases
> > >> with
> > >> > >> other issues/features. Roadmap would be great.
> > >> > >> (I'm just summarizing what we discussed a month or two ago. Feel
> > >> free to
> > >> > >> correct me or disagree with this approach.)
> > >> > >>
> > >> > >> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
> > >> > >> swapnil.daingade@gmail.com
> > >> > >> > wrote:
> > >> > >>
> > >> > >> > Hi all,
> > >> > >> >
> > >> > >> > Was wondering what would be the scope for the Myriad 0.1
> release.
> > >> > >> > It would be nice to have a roadmap page somewhere and target
> > >> > >> > features to releases (JIRA 'fix version' field perhaps)
> > >> > >> >
> > >> > >> > Regards
> > >> > >> > Swapnil
> > >> > >> >
> > >> > >>
> > >> > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Re: Myriad 0.1 release scope
Posted by Darin Johnson <db...@gmail.com>.
So I compiled the 2.5 fgs against 2.6 when I was testing. If we abstract
this right it may just be an if statement or two.
On Aug 7, 2015 6:47 PM, "Santosh Marella" <sm...@maprtech.com> wrote:
> > Myriad code base compiled against hadoop 2.7 should work on hadoop 2.5
> > cluster as long as FGS (i.e. zero profile NM) is not used.
>
> Verified the above. As long as FGS (zero profile NM) is not used,
> Myriad compiled against hadoop 2.7 will work on hadoop 2.5.
>
> Thanks,
> Santosh
>
> On Fri, Aug 7, 2015 at 2:20 PM, Santosh Marella <sm...@maprtech.com>
> wrote:
>
> > > It will make working on HA easier
> > Oh Yes!
> >
> > > how do we facilitate that? Profiles?
> > Profiles might be one way. Currently, FGS is supported for "zero" profile
> > only.
> > And we have seen there was an API incompatibility from 2.5 to 2.6+ in FGS
> > code.
> > So, ideally (since I haven't tried it myself), when FGS is merged into
> > phase1,
> > the Myriad code base compiled against hadoop 2.7 should work on hadoop
> 2.5
> > cluster as long as FGS (i.e. zero profile NM) is not used. (I'll try this
> > out and
> > post back what I find)
> >
> > However, in the long term we need a mechanism to abstract out the APIs
> > that are incompatible across versions.
> >
> > Thanks,
> > Santosh
> >
> > On Fri, Aug 7, 2015 at 12:12 PM, Darin Johnson <db...@gmail.com>
> > wrote:
> >
> >> It will make working on HA easier. However, one concern that's been
> >> addressed previously is that FGS works for Hadoop 2.6.0+. Do we plan to
> >> support 2.5.X (anything lower?) also as Santosh has a way to do that, if
> >> so
> >> how do we facilitate that? Profiles?
> >>
> >> Darin
> >>
> >> On Fri, Aug 7, 2015 at 1:46 PM, Santosh Marella <sm...@maprtech.com>
> >> wrote:
> >>
> >> > Hello guys,
> >> >
> >> > I propose merging FGS into phase1. As I said before, I think it's at a
> >> > point where the functionality works reasonably well.
> >> > Any future improvements/fixes/UI changes can be done via separate
> JIRAs.
> >> >
> >> > Unless there are any major concerns, I'd like to merge FGS into phase1
> >> *EOD
> >> > Monday* (PDT).
> >> >
> >> > Thanks,
> >> > Santosh
> >> >
> >> > On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella <
> smarella@maprtech.com>
> >> > wrote:
> >> >
> >> > > I feel FGS is very close to making it into 0.1. PR 116 addresses
> >> moving
> >> > to
> >> > > hadoop 2.7 and making FGS and CGS coexist. This PR was recently
> >> reviewed
> >> > by
> >> > > Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and
> >> 2.7.x
> >> > > clusters and it seemed to have worked as expected. Unless there are
> >> more
> >> > > reviews/feedback, it can be merged into issue_14. Once PR 116 is
> >> merged
> >> > > into issue_14, issue_14 can be merged into phase1.
> >> > >
> >> > > Thanks,
> >> > > Santosh
> >> > >
> >> > > On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io>
> >> > wrote:
> >> > >
> >> > >> We do have a JIRA 0.1.0 "fix version" field, but none of our issues
> >> use
> >> > it
> >> > >> yet.
> >> > >> I think the goal was just to take what we have and make it work
> under
> >> > >> Apache infrastructure, then vote on that for 0.1.0.
> >> > >> Although other features like HA or FGS would be great, let's try to
> >> get
> >> > >> our
> >> > >> first Apache release out ASAP.
> >> > >> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases
> >> with
> >> > >> other issues/features. Roadmap would be great.
> >> > >> (I'm just summarizing what we discussed a month or two ago. Feel
> >> free to
> >> > >> correct me or disagree with this approach.)
> >> > >>
> >> > >> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
> >> > >> swapnil.daingade@gmail.com
> >> > >> > wrote:
> >> > >>
> >> > >> > Hi all,
> >> > >> >
> >> > >> > Was wondering what would be the scope for the Myriad 0.1 release.
> >> > >> > It would be nice to have a roadmap page somewhere and target
> >> > >> > features to releases (JIRA 'fix version' field perhaps)
> >> > >> >
> >> > >> > Regards
> >> > >> > Swapnil
> >> > >> >
> >> > >>
> >> > >
> >> > >
> >> >
> >>
> >
> >
>
Re: Myriad 0.1 release scope
Posted by Santosh Marella <sm...@maprtech.com>.
> Myriad code base compiled against hadoop 2.7 should work on hadoop 2.5
> cluster as long as FGS (i.e. zero profile NM) is not used.
Verified the above. As long as FGS (zero profile NM) is not used,
Myriad compiled against hadoop 2.7 will work on hadoop 2.5.
Thanks,
Santosh
On Fri, Aug 7, 2015 at 2:20 PM, Santosh Marella <sm...@maprtech.com>
wrote:
> > It will make working on HA easier
> Oh Yes!
>
> > how do we facilitate that? Profiles?
> Profiles might be one way. Currently, FGS is supported for "zero" profile
> only.
> And we have seen there was an API incompatibility from 2.5 to 2.6+ in FGS
> code.
> So, ideally (since I haven't tried it myself), when FGS is merged into
> phase1,
> the Myriad code base compiled against hadoop 2.7 should work on hadoop 2.5
> cluster as long as FGS (i.e. zero profile NM) is not used. (I'll try this
> out and
> post back what I find)
>
> However, in the long term we need a mechanism to abstract out the APIs
> that are incompatible across versions.
>
> Thanks,
> Santosh
>
> On Fri, Aug 7, 2015 at 12:12 PM, Darin Johnson <db...@gmail.com>
> wrote:
>
>> It will make working on HA easier. However, one concern that's been
>> addressed previously is that FGS works for Hadoop 2.6.0+. Do we plan to
>> support 2.5.X (anything lower?) also as Santosh has a way to do that, if
>> so
>> how do we facilitate that? Profiles?
>>
>> Darin
>>
>> On Fri, Aug 7, 2015 at 1:46 PM, Santosh Marella <sm...@maprtech.com>
>> wrote:
>>
>> > Hello guys,
>> >
>> > I propose merging FGS into phase1. As I said before, I think it's at a
>> > point where the functionality works reasonably well.
>> > Any future improvements/fixes/UI changes can be done via separate JIRAs.
>> >
>> > Unless there are any major concerns, I'd like to merge FGS into phase1
>> *EOD
>> > Monday* (PDT).
>> >
>> > Thanks,
>> > Santosh
>> >
>> > On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella <sm...@maprtech.com>
>> > wrote:
>> >
>> > > I feel FGS is very close to making it into 0.1. PR 116 addresses
>> moving
>> > to
>> > > hadoop 2.7 and making FGS and CGS coexist. This PR was recently
>> reviewed
>> > by
>> > > Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and
>> 2.7.x
>> > > clusters and it seemed to have worked as expected. Unless there are
>> more
>> > > reviews/feedback, it can be merged into issue_14. Once PR 116 is
>> merged
>> > > into issue_14, issue_14 can be merged into phase1.
>> > >
>> > > Thanks,
>> > > Santosh
>> > >
>> > > On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io>
>> > wrote:
>> > >
>> > >> We do have a JIRA 0.1.0 "fix version" field, but none of our issues
>> use
>> > it
>> > >> yet.
>> > >> I think the goal was just to take what we have and make it work under
>> > >> Apache infrastructure, then vote on that for 0.1.0.
>> > >> Although other features like HA or FGS would be great, let's try to
>> get
>> > >> our
>> > >> first Apache release out ASAP.
>> > >> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases
>> with
>> > >> other issues/features. Roadmap would be great.
>> > >> (I'm just summarizing what we discussed a month or two ago. Feel
>> free to
>> > >> correct me or disagree with this approach.)
>> > >>
>> > >> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
>> > >> swapnil.daingade@gmail.com
>> > >> > wrote:
>> > >>
>> > >> > Hi all,
>> > >> >
>> > >> > Was wondering what would be the scope for the Myriad 0.1 release.
>> > >> > It would be nice to have a roadmap page somewhere and target
>> > >> > features to releases (JIRA 'fix version' field perhaps)
>> > >> >
>> > >> > Regards
>> > >> > Swapnil
>> > >> >
>> > >>
>> > >
>> > >
>> >
>>
>
>
Re: Myriad 0.1 release scope
Posted by Santosh Marella <sm...@maprtech.com>.
> It will make working on HA easier
Oh Yes!
> how do we facilitate that? Profiles?
Profiles might be one way. Currently, FGS is supported for "zero" profile
only.
And we have seen there was an API incompatibility from 2.5 to 2.6+ in FGS
code.
So, ideally (since I haven't tried it myself), when FGS is merged into
phase1,
the Myriad code base compiled against hadoop 2.7 should work on hadoop 2.5
cluster as long as FGS (i.e. zero profile NM) is not used. (I'll try this
out and
post back what I find)
However, in the long term we need a mechanism to abstract out the APIs
that are incompatible across versions.
Thanks,
Santosh
On Fri, Aug 7, 2015 at 12:12 PM, Darin Johnson <db...@gmail.com>
wrote:
> It will make working on HA easier. However, one concern that's been
> addressed previously is that FGS works for Hadoop 2.6.0+. Do we plan to
> support 2.5.X (anything lower?) also as Santosh has a way to do that, if so
> how do we facilitate that? Profiles?
>
> Darin
>
> On Fri, Aug 7, 2015 at 1:46 PM, Santosh Marella <sm...@maprtech.com>
> wrote:
>
> > Hello guys,
> >
> > I propose merging FGS into phase1. As I said before, I think it's at a
> > point where the functionality works reasonably well.
> > Any future improvements/fixes/UI changes can be done via separate JIRAs.
> >
> > Unless there are any major concerns, I'd like to merge FGS into phase1
> *EOD
> > Monday* (PDT).
> >
> > Thanks,
> > Santosh
> >
> > On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella <sm...@maprtech.com>
> > wrote:
> >
> > > I feel FGS is very close to making it into 0.1. PR 116 addresses moving
> > to
> > > hadoop 2.7 and making FGS and CGS coexist. This PR was recently
> reviewed
> > by
> > > Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x
> > > clusters and it seemed to have worked as expected. Unless there are
> more
> > > reviews/feedback, it can be merged into issue_14. Once PR 116 is merged
> > > into issue_14, issue_14 can be merged into phase1.
> > >
> > > Thanks,
> > > Santosh
> > >
> > > On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io>
> > wrote:
> > >
> > >> We do have a JIRA 0.1.0 "fix version" field, but none of our issues
> use
> > it
> > >> yet.
> > >> I think the goal was just to take what we have and make it work under
> > >> Apache infrastructure, then vote on that for 0.1.0.
> > >> Although other features like HA or FGS would be great, let's try to
> get
> > >> our
> > >> first Apache release out ASAP.
> > >> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with
> > >> other issues/features. Roadmap would be great.
> > >> (I'm just summarizing what we discussed a month or two ago. Feel free
> to
> > >> correct me or disagree with this approach.)
> > >>
> > >> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
> > >> swapnil.daingade@gmail.com
> > >> > wrote:
> > >>
> > >> > Hi all,
> > >> >
> > >> > Was wondering what would be the scope for the Myriad 0.1 release.
> > >> > It would be nice to have a roadmap page somewhere and target
> > >> > features to releases (JIRA 'fix version' field perhaps)
> > >> >
> > >> > Regards
> > >> > Swapnil
> > >> >
> > >>
> > >
> > >
> >
>
Re: Myriad 0.1 release scope
Posted by Darin Johnson <db...@gmail.com>.
It will make working on HA easier. However, one concern that's been
addressed previously is that FGS works for Hadoop 2.6.0+. Do we plan to
support 2.5.X (anything lower?) also as Santosh has a way to do that, if so
how do we facilitate that? Profiles?
Darin
On Fri, Aug 7, 2015 at 1:46 PM, Santosh Marella <sm...@maprtech.com>
wrote:
> Hello guys,
>
> I propose merging FGS into phase1. As I said before, I think it's at a
> point where the functionality works reasonably well.
> Any future improvements/fixes/UI changes can be done via separate JIRAs.
>
> Unless there are any major concerns, I'd like to merge FGS into phase1 *EOD
> Monday* (PDT).
>
> Thanks,
> Santosh
>
> On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella <sm...@maprtech.com>
> wrote:
>
> > I feel FGS is very close to making it into 0.1. PR 116 addresses moving
> to
> > hadoop 2.7 and making FGS and CGS coexist. This PR was recently reviewed
> by
> > Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x
> > clusters and it seemed to have worked as expected. Unless there are more
> > reviews/feedback, it can be merged into issue_14. Once PR 116 is merged
> > into issue_14, issue_14 can be merged into phase1.
> >
> > Thanks,
> > Santosh
> >
> > On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io>
> wrote:
> >
> >> We do have a JIRA 0.1.0 "fix version" field, but none of our issues use
> it
> >> yet.
> >> I think the goal was just to take what we have and make it work under
> >> Apache infrastructure, then vote on that for 0.1.0.
> >> Although other features like HA or FGS would be great, let's try to get
> >> our
> >> first Apache release out ASAP.
> >> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with
> >> other issues/features. Roadmap would be great.
> >> (I'm just summarizing what we discussed a month or two ago. Feel free to
> >> correct me or disagree with this approach.)
> >>
> >> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
> >> swapnil.daingade@gmail.com
> >> > wrote:
> >>
> >> > Hi all,
> >> >
> >> > Was wondering what would be the scope for the Myriad 0.1 release.
> >> > It would be nice to have a roadmap page somewhere and target
> >> > features to releases (JIRA 'fix version' field perhaps)
> >> >
> >> > Regards
> >> > Swapnil
> >> >
> >>
> >
> >
>
Re: Myriad 0.1 release scope
Posted by Darin Johnson <db...@gmail.com>.
John,
The remote distribution doesn't require the nm to be run from marathon
though it's possible. Essentially, it's the same configuration for the rm
you'd do for the non remote version + adding a uri for the tarball.
I've got jsons for running the rm in marathon, I'll try to get them and
some documentation up soon. Currently at a conference though which means
probably next week.
Darin
Darin
On Aug 18, 2015 2:49 PM, "John Omernik" <jo...@omernik.com> wrote:
> Ok, so I tried the remote distribution of the Myriad per the docs, I
> guess,it could probably use some information related to "how" to run
> resource manager if it's in the tar.gz. Perhaps an example marathon json.
> I am playing with it now to figure it out.
>
> On Tue, Aug 18, 2015 at 3:48 PM, yuliya Feldman
> <yufeldman@yahoo.com.invalid
> > wrote:
>
> > mesos/myriad is the right one so far
> > From: John Omernik <jo...@omernik.com>
> > To: dev@myriad.incubator.apache.org; yuliya Feldman <
> yufeldman@yahoo.com>
> > Sent: Tuesday, August 18, 2015 1:44 PM
> > Subject: Re: Myriad 0.1 release scope
> >
> > (So if I clone that repo, am I cloning the right one?)
> >
> >
> >
> > On Tue, Aug 18, 2015 at 3:43 PM, John Omernik <jo...@omernik.com> wrote:
> >
> > > Ok, I was going off
> > >
> https://github.com/mesos/myriad/blob/phase1/docs/myriad-configuration.md
> > >
> > > I will try it.
> > >
> > > John
> > >
> > > On Tue, Aug 18, 2015 at 3:40 PM, yuliya Feldman <
> > > yufeldman@yahoo.com.invalid> wrote:
> > >
> > >> You actually do not need to rebuild even today - just keep this file
> in
> > >> hadoop config directory that is on the classpath: like .../etc/hadoop
> > >> From: John Omernik <jo...@omernik.com>
> > >> To: dev@myriad.incubator.apache.org
> > >> Sent: Tuesday, August 18, 2015 1:35 PM
> > >> Subject: Re: Myriad 0.1 release scope
> > >>
> > >> On the release scope, will having the myriad configuration file exist
> > >> outside the jar (i.e. you can change configuration without rebuilding)
> > be
> > >> part of the .1 release scope?
> > >>
> > >>
> > >>
> > >> On Mon, Aug 10, 2015 at 10:01 PM, Santosh Marella <
> > smarella@maprtech.com>
> > >> wrote:
> > >>
> > >> > Hello All,
> > >> >
> > >> > I've merged the FGS changes into phase1. Built and tested both
> coarse
> > >> > grained scaling and fine grained scaling, UI on a 4 node cluster.
> > >> >
> > >> > If anyone finds things are not working as expected, please let me
> > know.
> > >> >
> > >> > Thanks,
> > >> > Santosh
> > >> >
> > >> > On Fri, Aug 7, 2015 at 10:46 AM, Santosh Marella <
> > smarella@maprtech.com
> > >> >
> > >> > wrote:
> > >> >
> > >> > > Hello guys,
> > >> > >
> > >> > > I propose merging FGS into phase1. As I said before, I think it's
> > at a
> > >> > > point where the functionality works reasonably well.
> > >> > > Any future improvements/fixes/UI changes can be done via separate
> > >> JIRAs.
> > >> > >
> > >> > > Unless there are any major concerns, I'd like to merge FGS into
> > phase1
> > >> > > *EOD Monday* (PDT).
> > >> > >
> > >> > > Thanks,
> > >> > > Santosh
> > >> > >
> > >> > > On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella <
> > >> smarella@maprtech.com>
> > >> > > wrote:
> > >> > >
> > >> > >> I feel FGS is very close to making it into 0.1. PR 116 addresses
> > >> moving
> > >> > >> to hadoop 2.7 and making FGS and CGS coexist. This PR was
> recently
> > >> > reviewed
> > >> > >> by Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x
> > and
> > >> > 2.7.x
> > >> > >> clusters and it seemed to have worked as expected. Unless there
> are
> > >> more
> > >> > >> reviews/feedback, it can be merged into issue_14. Once PR 116 is
> > >> merged
> > >> > >> into issue_14, issue_14 can be merged into phase1.
> > >> > >>
> > >> > >> Thanks,
> > >> > >> Santosh
> > >> > >>
> > >> > >> On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <
> adam@mesosphere.io>
> > >> > wrote:
> > >> > >>
> > >> > >>> We do have a JIRA 0.1.0 "fix version" field, but none of our
> > issues
> > >> use
> > >> > >>> it
> > >> > >>> yet.
> > >> > >>> I think the goal was just to take what we have and make it work
> > >> under
> > >> > >>> Apache infrastructure, then vote on that for 0.1.0.
> > >> > >>> Although other features like HA or FGS would be great, let's try
> > to
> > >> get
> > >> > >>> our
> > >> > >>> first Apache release out ASAP.
> > >> > >>> We can create 0.1.1 or 0.2.0 fix versions for subsequent
> releases
> > >> with
> > >> > >>> other issues/features. Roadmap would be great.
> > >> > >>> (I'm just summarizing what we discussed a month or two ago. Feel
> > >> free
> > >> > to
> > >> > >>> correct me or disagree with this approach.)
> > >> > >>>
> > >> > >>> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
> > >> > >>> swapnil.daingade@gmail.com
> > >> > >>> > wrote:
> > >> > >>>
> > >> > >>> > Hi all,
> > >> > >>> >
> > >> > >>> > Was wondering what would be the scope for the Myriad 0.1
> > release.
> > >> > >>> > It would be nice to have a roadmap page somewhere and target
> > >> > >>> > features to releases (JIRA 'fix version' field perhaps)
> > >> > >>> >
> > >> > >>> > Regards
> > >> > >>> > Swapnil
> > >> > >>> >
> > >> > >>>
> > >> > >>
> > >> > >>
> > >> > >
> > >> >
> > >>
> > >>
> > >>
> > >>
> > >
> > >
> >
> >
> >
> >
>
Re: Myriad 0.1 release scope
Posted by John Omernik <jo...@omernik.com>.
Ok, so I tried the remote distribution of the Myriad per the docs, I
guess,it could probably use some information related to "how" to run
resource manager if it's in the tar.gz. Perhaps an example marathon json.
I am playing with it now to figure it out.
On Tue, Aug 18, 2015 at 3:48 PM, yuliya Feldman <yufeldman@yahoo.com.invalid
> wrote:
> mesos/myriad is the right one so far
> From: John Omernik <jo...@omernik.com>
> To: dev@myriad.incubator.apache.org; yuliya Feldman <yu...@yahoo.com>
> Sent: Tuesday, August 18, 2015 1:44 PM
> Subject: Re: Myriad 0.1 release scope
>
> (So if I clone that repo, am I cloning the right one?)
>
>
>
> On Tue, Aug 18, 2015 at 3:43 PM, John Omernik <jo...@omernik.com> wrote:
>
> > Ok, I was going off
> > https://github.com/mesos/myriad/blob/phase1/docs/myriad-configuration.md
> >
> > I will try it.
> >
> > John
> >
> > On Tue, Aug 18, 2015 at 3:40 PM, yuliya Feldman <
> > yufeldman@yahoo.com.invalid> wrote:
> >
> >> You actually do not need to rebuild even today - just keep this file in
> >> hadoop config directory that is on the classpath: like .../etc/hadoop
> >> From: John Omernik <jo...@omernik.com>
> >> To: dev@myriad.incubator.apache.org
> >> Sent: Tuesday, August 18, 2015 1:35 PM
> >> Subject: Re: Myriad 0.1 release scope
> >>
> >> On the release scope, will having the myriad configuration file exist
> >> outside the jar (i.e. you can change configuration without rebuilding)
> be
> >> part of the .1 release scope?
> >>
> >>
> >>
> >> On Mon, Aug 10, 2015 at 10:01 PM, Santosh Marella <
> smarella@maprtech.com>
> >> wrote:
> >>
> >> > Hello All,
> >> >
> >> > I've merged the FGS changes into phase1. Built and tested both coarse
> >> > grained scaling and fine grained scaling, UI on a 4 node cluster.
> >> >
> >> > If anyone finds things are not working as expected, please let me
> know.
> >> >
> >> > Thanks,
> >> > Santosh
> >> >
> >> > On Fri, Aug 7, 2015 at 10:46 AM, Santosh Marella <
> smarella@maprtech.com
> >> >
> >> > wrote:
> >> >
> >> > > Hello guys,
> >> > >
> >> > > I propose merging FGS into phase1. As I said before, I think it's
> at a
> >> > > point where the functionality works reasonably well.
> >> > > Any future improvements/fixes/UI changes can be done via separate
> >> JIRAs.
> >> > >
> >> > > Unless there are any major concerns, I'd like to merge FGS into
> phase1
> >> > > *EOD Monday* (PDT).
> >> > >
> >> > > Thanks,
> >> > > Santosh
> >> > >
> >> > > On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella <
> >> smarella@maprtech.com>
> >> > > wrote:
> >> > >
> >> > >> I feel FGS is very close to making it into 0.1. PR 116 addresses
> >> moving
> >> > >> to hadoop 2.7 and making FGS and CGS coexist. This PR was recently
> >> > reviewed
> >> > >> by Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x
> and
> >> > 2.7.x
> >> > >> clusters and it seemed to have worked as expected. Unless there are
> >> more
> >> > >> reviews/feedback, it can be merged into issue_14. Once PR 116 is
> >> merged
> >> > >> into issue_14, issue_14 can be merged into phase1.
> >> > >>
> >> > >> Thanks,
> >> > >> Santosh
> >> > >>
> >> > >> On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io>
> >> > wrote:
> >> > >>
> >> > >>> We do have a JIRA 0.1.0 "fix version" field, but none of our
> issues
> >> use
> >> > >>> it
> >> > >>> yet.
> >> > >>> I think the goal was just to take what we have and make it work
> >> under
> >> > >>> Apache infrastructure, then vote on that for 0.1.0.
> >> > >>> Although other features like HA or FGS would be great, let's try
> to
> >> get
> >> > >>> our
> >> > >>> first Apache release out ASAP.
> >> > >>> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases
> >> with
> >> > >>> other issues/features. Roadmap would be great.
> >> > >>> (I'm just summarizing what we discussed a month or two ago. Feel
> >> free
> >> > to
> >> > >>> correct me or disagree with this approach.)
> >> > >>>
> >> > >>> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
> >> > >>> swapnil.daingade@gmail.com
> >> > >>> > wrote:
> >> > >>>
> >> > >>> > Hi all,
> >> > >>> >
> >> > >>> > Was wondering what would be the scope for the Myriad 0.1
> release.
> >> > >>> > It would be nice to have a roadmap page somewhere and target
> >> > >>> > features to releases (JIRA 'fix version' field perhaps)
> >> > >>> >
> >> > >>> > Regards
> >> > >>> > Swapnil
> >> > >>> >
> >> > >>>
> >> > >>
> >> > >>
> >> > >
> >> >
> >>
> >>
> >>
> >>
> >
> >
>
>
>
>
Re: Myriad 0.1 release scope
Posted by yuliya Feldman <yu...@yahoo.com.INVALID>.
mesos/myriad is the right one so far
From: John Omernik <jo...@omernik.com>
To: dev@myriad.incubator.apache.org; yuliya Feldman <yu...@yahoo.com>
Sent: Tuesday, August 18, 2015 1:44 PM
Subject: Re: Myriad 0.1 release scope
(So if I clone that repo, am I cloning the right one?)
On Tue, Aug 18, 2015 at 3:43 PM, John Omernik <jo...@omernik.com> wrote:
> Ok, I was going off
> https://github.com/mesos/myriad/blob/phase1/docs/myriad-configuration.md
>
> I will try it.
>
> John
>
> On Tue, Aug 18, 2015 at 3:40 PM, yuliya Feldman <
> yufeldman@yahoo.com.invalid> wrote:
>
>> You actually do not need to rebuild even today - just keep this file in
>> hadoop config directory that is on the classpath: like .../etc/hadoop
>> From: John Omernik <jo...@omernik.com>
>> To: dev@myriad.incubator.apache.org
>> Sent: Tuesday, August 18, 2015 1:35 PM
>> Subject: Re: Myriad 0.1 release scope
>>
>> On the release scope, will having the myriad configuration file exist
>> outside the jar (i.e. you can change configuration without rebuilding) be
>> part of the .1 release scope?
>>
>>
>>
>> On Mon, Aug 10, 2015 at 10:01 PM, Santosh Marella <sm...@maprtech.com>
>> wrote:
>>
>> > Hello All,
>> >
>> > I've merged the FGS changes into phase1. Built and tested both coarse
>> > grained scaling and fine grained scaling, UI on a 4 node cluster.
>> >
>> > If anyone finds things are not working as expected, please let me know.
>> >
>> > Thanks,
>> > Santosh
>> >
>> > On Fri, Aug 7, 2015 at 10:46 AM, Santosh Marella <smarella@maprtech.com
>> >
>> > wrote:
>> >
>> > > Hello guys,
>> > >
>> > > I propose merging FGS into phase1. As I said before, I think it's at a
>> > > point where the functionality works reasonably well.
>> > > Any future improvements/fixes/UI changes can be done via separate
>> JIRAs.
>> > >
>> > > Unless there are any major concerns, I'd like to merge FGS into phase1
>> > > *EOD Monday* (PDT).
>> > >
>> > > Thanks,
>> > > Santosh
>> > >
>> > > On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella <
>> smarella@maprtech.com>
>> > > wrote:
>> > >
>> > >> I feel FGS is very close to making it into 0.1. PR 116 addresses
>> moving
>> > >> to hadoop 2.7 and making FGS and CGS coexist. This PR was recently
>> > reviewed
>> > >> by Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and
>> > 2.7.x
>> > >> clusters and it seemed to have worked as expected. Unless there are
>> more
>> > >> reviews/feedback, it can be merged into issue_14. Once PR 116 is
>> merged
>> > >> into issue_14, issue_14 can be merged into phase1.
>> > >>
>> > >> Thanks,
>> > >> Santosh
>> > >>
>> > >> On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io>
>> > wrote:
>> > >>
>> > >>> We do have a JIRA 0.1.0 "fix version" field, but none of our issues
>> use
>> > >>> it
>> > >>> yet.
>> > >>> I think the goal was just to take what we have and make it work
>> under
>> > >>> Apache infrastructure, then vote on that for 0.1.0.
>> > >>> Although other features like HA or FGS would be great, let's try to
>> get
>> > >>> our
>> > >>> first Apache release out ASAP.
>> > >>> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases
>> with
>> > >>> other issues/features. Roadmap would be great.
>> > >>> (I'm just summarizing what we discussed a month or two ago. Feel
>> free
>> > to
>> > >>> correct me or disagree with this approach.)
>> > >>>
>> > >>> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
>> > >>> swapnil.daingade@gmail.com
>> > >>> > wrote:
>> > >>>
>> > >>> > Hi all,
>> > >>> >
>> > >>> > Was wondering what would be the scope for the Myriad 0.1 release.
>> > >>> > It would be nice to have a roadmap page somewhere and target
>> > >>> > features to releases (JIRA 'fix version' field perhaps)
>> > >>> >
>> > >>> > Regards
>> > >>> > Swapnil
>> > >>> >
>> > >>>
>> > >>
>> > >>
>> > >
>> >
>>
>>
>>
>>
>
>
Re: Myriad 0.1 release scope
Posted by John Omernik <jo...@omernik.com>.
(So if I clone that repo, am I cloning the right one?)
On Tue, Aug 18, 2015 at 3:43 PM, John Omernik <jo...@omernik.com> wrote:
> Ok, I was going off
> https://github.com/mesos/myriad/blob/phase1/docs/myriad-configuration.md
>
> I will try it.
>
> John
>
> On Tue, Aug 18, 2015 at 3:40 PM, yuliya Feldman <
> yufeldman@yahoo.com.invalid> wrote:
>
>> You actually do not need to rebuild even today - just keep this file in
>> hadoop config directory that is on the classpath: like .../etc/hadoop
>> From: John Omernik <jo...@omernik.com>
>> To: dev@myriad.incubator.apache.org
>> Sent: Tuesday, August 18, 2015 1:35 PM
>> Subject: Re: Myriad 0.1 release scope
>>
>> On the release scope, will having the myriad configuration file exist
>> outside the jar (i.e. you can change configuration without rebuilding) be
>> part of the .1 release scope?
>>
>>
>>
>> On Mon, Aug 10, 2015 at 10:01 PM, Santosh Marella <sm...@maprtech.com>
>> wrote:
>>
>> > Hello All,
>> >
>> > I've merged the FGS changes into phase1. Built and tested both coarse
>> > grained scaling and fine grained scaling, UI on a 4 node cluster.
>> >
>> > If anyone finds things are not working as expected, please let me know.
>> >
>> > Thanks,
>> > Santosh
>> >
>> > On Fri, Aug 7, 2015 at 10:46 AM, Santosh Marella <smarella@maprtech.com
>> >
>> > wrote:
>> >
>> > > Hello guys,
>> > >
>> > > I propose merging FGS into phase1. As I said before, I think it's at a
>> > > point where the functionality works reasonably well.
>> > > Any future improvements/fixes/UI changes can be done via separate
>> JIRAs.
>> > >
>> > > Unless there are any major concerns, I'd like to merge FGS into phase1
>> > > *EOD Monday* (PDT).
>> > >
>> > > Thanks,
>> > > Santosh
>> > >
>> > > On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella <
>> smarella@maprtech.com>
>> > > wrote:
>> > >
>> > >> I feel FGS is very close to making it into 0.1. PR 116 addresses
>> moving
>> > >> to hadoop 2.7 and making FGS and CGS coexist. This PR was recently
>> > reviewed
>> > >> by Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and
>> > 2.7.x
>> > >> clusters and it seemed to have worked as expected. Unless there are
>> more
>> > >> reviews/feedback, it can be merged into issue_14. Once PR 116 is
>> merged
>> > >> into issue_14, issue_14 can be merged into phase1.
>> > >>
>> > >> Thanks,
>> > >> Santosh
>> > >>
>> > >> On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io>
>> > wrote:
>> > >>
>> > >>> We do have a JIRA 0.1.0 "fix version" field, but none of our issues
>> use
>> > >>> it
>> > >>> yet.
>> > >>> I think the goal was just to take what we have and make it work
>> under
>> > >>> Apache infrastructure, then vote on that for 0.1.0.
>> > >>> Although other features like HA or FGS would be great, let's try to
>> get
>> > >>> our
>> > >>> first Apache release out ASAP.
>> > >>> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases
>> with
>> > >>> other issues/features. Roadmap would be great.
>> > >>> (I'm just summarizing what we discussed a month or two ago. Feel
>> free
>> > to
>> > >>> correct me or disagree with this approach.)
>> > >>>
>> > >>> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
>> > >>> swapnil.daingade@gmail.com
>> > >>> > wrote:
>> > >>>
>> > >>> > Hi all,
>> > >>> >
>> > >>> > Was wondering what would be the scope for the Myriad 0.1 release.
>> > >>> > It would be nice to have a roadmap page somewhere and target
>> > >>> > features to releases (JIRA 'fix version' field perhaps)
>> > >>> >
>> > >>> > Regards
>> > >>> > Swapnil
>> > >>> >
>> > >>>
>> > >>
>> > >>
>> > >
>> >
>>
>>
>>
>>
>
>
Re: Myriad 0.1 release scope
Posted by John Omernik <jo...@omernik.com>.
Ok, I was going off
https://github.com/mesos/myriad/blob/phase1/docs/myriad-configuration.md
I will try it.
John
On Tue, Aug 18, 2015 at 3:40 PM, yuliya Feldman <yufeldman@yahoo.com.invalid
> wrote:
> You actually do not need to rebuild even today - just keep this file in
> hadoop config directory that is on the classpath: like .../etc/hadoop
> From: John Omernik <jo...@omernik.com>
> To: dev@myriad.incubator.apache.org
> Sent: Tuesday, August 18, 2015 1:35 PM
> Subject: Re: Myriad 0.1 release scope
>
> On the release scope, will having the myriad configuration file exist
> outside the jar (i.e. you can change configuration without rebuilding) be
> part of the .1 release scope?
>
>
>
> On Mon, Aug 10, 2015 at 10:01 PM, Santosh Marella <sm...@maprtech.com>
> wrote:
>
> > Hello All,
> >
> > I've merged the FGS changes into phase1. Built and tested both coarse
> > grained scaling and fine grained scaling, UI on a 4 node cluster.
> >
> > If anyone finds things are not working as expected, please let me know.
> >
> > Thanks,
> > Santosh
> >
> > On Fri, Aug 7, 2015 at 10:46 AM, Santosh Marella <sm...@maprtech.com>
> > wrote:
> >
> > > Hello guys,
> > >
> > > I propose merging FGS into phase1. As I said before, I think it's at a
> > > point where the functionality works reasonably well.
> > > Any future improvements/fixes/UI changes can be done via separate
> JIRAs.
> > >
> > > Unless there are any major concerns, I'd like to merge FGS into phase1
> > > *EOD Monday* (PDT).
> > >
> > > Thanks,
> > > Santosh
> > >
> > > On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella <smarella@maprtech.com
> >
> > > wrote:
> > >
> > >> I feel FGS is very close to making it into 0.1. PR 116 addresses
> moving
> > >> to hadoop 2.7 and making FGS and CGS coexist. This PR was recently
> > reviewed
> > >> by Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and
> > 2.7.x
> > >> clusters and it seemed to have worked as expected. Unless there are
> more
> > >> reviews/feedback, it can be merged into issue_14. Once PR 116 is
> merged
> > >> into issue_14, issue_14 can be merged into phase1.
> > >>
> > >> Thanks,
> > >> Santosh
> > >>
> > >> On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io>
> > wrote:
> > >>
> > >>> We do have a JIRA 0.1.0 "fix version" field, but none of our issues
> use
> > >>> it
> > >>> yet.
> > >>> I think the goal was just to take what we have and make it work under
> > >>> Apache infrastructure, then vote on that for 0.1.0.
> > >>> Although other features like HA or FGS would be great, let's try to
> get
> > >>> our
> > >>> first Apache release out ASAP.
> > >>> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases
> with
> > >>> other issues/features. Roadmap would be great.
> > >>> (I'm just summarizing what we discussed a month or two ago. Feel free
> > to
> > >>> correct me or disagree with this approach.)
> > >>>
> > >>> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
> > >>> swapnil.daingade@gmail.com
> > >>> > wrote:
> > >>>
> > >>> > Hi all,
> > >>> >
> > >>> > Was wondering what would be the scope for the Myriad 0.1 release.
> > >>> > It would be nice to have a roadmap page somewhere and target
> > >>> > features to releases (JIRA 'fix version' field perhaps)
> > >>> >
> > >>> > Regards
> > >>> > Swapnil
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>
>
>
>
Re: Myriad 0.1 release scope
Posted by yuliya Feldman <yu...@yahoo.com.INVALID>.
You actually do not need to rebuild even today - just keep this file in hadoop config directory that is on the classpath: like .../etc/hadoop
From: John Omernik <jo...@omernik.com>
To: dev@myriad.incubator.apache.org
Sent: Tuesday, August 18, 2015 1:35 PM
Subject: Re: Myriad 0.1 release scope
On the release scope, will having the myriad configuration file exist
outside the jar (i.e. you can change configuration without rebuilding) be
part of the .1 release scope?
On Mon, Aug 10, 2015 at 10:01 PM, Santosh Marella <sm...@maprtech.com>
wrote:
> Hello All,
>
> I've merged the FGS changes into phase1. Built and tested both coarse
> grained scaling and fine grained scaling, UI on a 4 node cluster.
>
> If anyone finds things are not working as expected, please let me know.
>
> Thanks,
> Santosh
>
> On Fri, Aug 7, 2015 at 10:46 AM, Santosh Marella <sm...@maprtech.com>
> wrote:
>
> > Hello guys,
> >
> > I propose merging FGS into phase1. As I said before, I think it's at a
> > point where the functionality works reasonably well.
> > Any future improvements/fixes/UI changes can be done via separate JIRAs.
> >
> > Unless there are any major concerns, I'd like to merge FGS into phase1
> > *EOD Monday* (PDT).
> >
> > Thanks,
> > Santosh
> >
> > On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella <sm...@maprtech.com>
> > wrote:
> >
> >> I feel FGS is very close to making it into 0.1. PR 116 addresses moving
> >> to hadoop 2.7 and making FGS and CGS coexist. This PR was recently
> reviewed
> >> by Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and
> 2.7.x
> >> clusters and it seemed to have worked as expected. Unless there are more
> >> reviews/feedback, it can be merged into issue_14. Once PR 116 is merged
> >> into issue_14, issue_14 can be merged into phase1.
> >>
> >> Thanks,
> >> Santosh
> >>
> >> On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io>
> wrote:
> >>
> >>> We do have a JIRA 0.1.0 "fix version" field, but none of our issues use
> >>> it
> >>> yet.
> >>> I think the goal was just to take what we have and make it work under
> >>> Apache infrastructure, then vote on that for 0.1.0.
> >>> Although other features like HA or FGS would be great, let's try to get
> >>> our
> >>> first Apache release out ASAP.
> >>> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with
> >>> other issues/features. Roadmap would be great.
> >>> (I'm just summarizing what we discussed a month or two ago. Feel free
> to
> >>> correct me or disagree with this approach.)
> >>>
> >>> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
> >>> swapnil.daingade@gmail.com
> >>> > wrote:
> >>>
> >>> > Hi all,
> >>> >
> >>> > Was wondering what would be the scope for the Myriad 0.1 release.
> >>> > It would be nice to have a roadmap page somewhere and target
> >>> > features to releases (JIRA 'fix version' field perhaps)
> >>> >
> >>> > Regards
> >>> > Swapnil
> >>> >
> >>>
> >>
> >>
> >
>
Re: Myriad 0.1 release scope
Posted by John Omernik <jo...@omernik.com>.
On the release scope, will having the myriad configuration file exist
outside the jar (i.e. you can change configuration without rebuilding) be
part of the .1 release scope?
On Mon, Aug 10, 2015 at 10:01 PM, Santosh Marella <sm...@maprtech.com>
wrote:
> Hello All,
>
> I've merged the FGS changes into phase1. Built and tested both coarse
> grained scaling and fine grained scaling, UI on a 4 node cluster.
>
> If anyone finds things are not working as expected, please let me know.
>
> Thanks,
> Santosh
>
> On Fri, Aug 7, 2015 at 10:46 AM, Santosh Marella <sm...@maprtech.com>
> wrote:
>
> > Hello guys,
> >
> > I propose merging FGS into phase1. As I said before, I think it's at a
> > point where the functionality works reasonably well.
> > Any future improvements/fixes/UI changes can be done via separate JIRAs.
> >
> > Unless there are any major concerns, I'd like to merge FGS into phase1
> > *EOD Monday* (PDT).
> >
> > Thanks,
> > Santosh
> >
> > On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella <sm...@maprtech.com>
> > wrote:
> >
> >> I feel FGS is very close to making it into 0.1. PR 116 addresses moving
> >> to hadoop 2.7 and making FGS and CGS coexist. This PR was recently
> reviewed
> >> by Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and
> 2.7.x
> >> clusters and it seemed to have worked as expected. Unless there are more
> >> reviews/feedback, it can be merged into issue_14. Once PR 116 is merged
> >> into issue_14, issue_14 can be merged into phase1.
> >>
> >> Thanks,
> >> Santosh
> >>
> >> On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io>
> wrote:
> >>
> >>> We do have a JIRA 0.1.0 "fix version" field, but none of our issues use
> >>> it
> >>> yet.
> >>> I think the goal was just to take what we have and make it work under
> >>> Apache infrastructure, then vote on that for 0.1.0.
> >>> Although other features like HA or FGS would be great, let's try to get
> >>> our
> >>> first Apache release out ASAP.
> >>> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with
> >>> other issues/features. Roadmap would be great.
> >>> (I'm just summarizing what we discussed a month or two ago. Feel free
> to
> >>> correct me or disagree with this approach.)
> >>>
> >>> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
> >>> swapnil.daingade@gmail.com
> >>> > wrote:
> >>>
> >>> > Hi all,
> >>> >
> >>> > Was wondering what would be the scope for the Myriad 0.1 release.
> >>> > It would be nice to have a roadmap page somewhere and target
> >>> > features to releases (JIRA 'fix version' field perhaps)
> >>> >
> >>> > Regards
> >>> > Swapnil
> >>> >
> >>>
> >>
> >>
> >
>
Re: Myriad 0.1 release scope
Posted by Santosh Marella <sm...@maprtech.com>.
Hello All,
I've merged the FGS changes into phase1. Built and tested both coarse
grained scaling and fine grained scaling, UI on a 4 node cluster.
If anyone finds things are not working as expected, please let me know.
Thanks,
Santosh
On Fri, Aug 7, 2015 at 10:46 AM, Santosh Marella <sm...@maprtech.com>
wrote:
> Hello guys,
>
> I propose merging FGS into phase1. As I said before, I think it's at a
> point where the functionality works reasonably well.
> Any future improvements/fixes/UI changes can be done via separate JIRAs.
>
> Unless there are any major concerns, I'd like to merge FGS into phase1
> *EOD Monday* (PDT).
>
> Thanks,
> Santosh
>
> On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella <sm...@maprtech.com>
> wrote:
>
>> I feel FGS is very close to making it into 0.1. PR 116 addresses moving
>> to hadoop 2.7 and making FGS and CGS coexist. This PR was recently reviewed
>> by Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x
>> clusters and it seemed to have worked as expected. Unless there are more
>> reviews/feedback, it can be merged into issue_14. Once PR 116 is merged
>> into issue_14, issue_14 can be merged into phase1.
>>
>> Thanks,
>> Santosh
>>
>> On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io> wrote:
>>
>>> We do have a JIRA 0.1.0 "fix version" field, but none of our issues use
>>> it
>>> yet.
>>> I think the goal was just to take what we have and make it work under
>>> Apache infrastructure, then vote on that for 0.1.0.
>>> Although other features like HA or FGS would be great, let's try to get
>>> our
>>> first Apache release out ASAP.
>>> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with
>>> other issues/features. Roadmap would be great.
>>> (I'm just summarizing what we discussed a month or two ago. Feel free to
>>> correct me or disagree with this approach.)
>>>
>>> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
>>> swapnil.daingade@gmail.com
>>> > wrote:
>>>
>>> > Hi all,
>>> >
>>> > Was wondering what would be the scope for the Myriad 0.1 release.
>>> > It would be nice to have a roadmap page somewhere and target
>>> > features to releases (JIRA 'fix version' field perhaps)
>>> >
>>> > Regards
>>> > Swapnil
>>> >
>>>
>>
>>
>
Re: Myriad 0.1 release scope
Posted by Santosh Marella <sm...@maprtech.com>.
Hello guys,
I propose merging FGS into phase1. As I said before, I think it's at a
point where the functionality works reasonably well.
Any future improvements/fixes/UI changes can be done via separate JIRAs.
Unless there are any major concerns, I'd like to merge FGS into phase1 *EOD
Monday* (PDT).
Thanks,
Santosh
On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella <sm...@maprtech.com>
wrote:
> I feel FGS is very close to making it into 0.1. PR 116 addresses moving to
> hadoop 2.7 and making FGS and CGS coexist. This PR was recently reviewed by
> Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x
> clusters and it seemed to have worked as expected. Unless there are more
> reviews/feedback, it can be merged into issue_14. Once PR 116 is merged
> into issue_14, issue_14 can be merged into phase1.
>
> Thanks,
> Santosh
>
> On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io> wrote:
>
>> We do have a JIRA 0.1.0 "fix version" field, but none of our issues use it
>> yet.
>> I think the goal was just to take what we have and make it work under
>> Apache infrastructure, then vote on that for 0.1.0.
>> Although other features like HA or FGS would be great, let's try to get
>> our
>> first Apache release out ASAP.
>> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with
>> other issues/features. Roadmap would be great.
>> (I'm just summarizing what we discussed a month or two ago. Feel free to
>> correct me or disagree with this approach.)
>>
>> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
>> swapnil.daingade@gmail.com
>> > wrote:
>>
>> > Hi all,
>> >
>> > Was wondering what would be the scope for the Myriad 0.1 release.
>> > It would be nice to have a roadmap page somewhere and target
>> > features to releases (JIRA 'fix version' field perhaps)
>> >
>> > Regards
>> > Swapnil
>> >
>>
>
>
Re: Myriad 0.1 release scope
Posted by Santosh Marella <sm...@maprtech.com>.
Hi Jim,
Nothing major, but looking at the current FGS experience, I have
something to share.
Currently, if the user launches a NM with "zero" profile, that NM is
eligible for FGS.
This means, the YARN containers that will be allocated to that NM will be
based upon
the Mesos offers received for that node.
The fact that "zero" profile corresponds to FGS might be confusing.
a) we need a better/clearer way to express this fact. Perhaps a
different name for the "zero" profile?
b) UI can perhaps show a "help" message explaining that a "zero" profile
NM pertains to FGS.
Thanks,
Santosh
On Thu, Aug 6, 2015 at 5:25 PM, Swapnil Daingade <swapnil.daingade@gmail.com
> wrote:
> Hi Jim,
>
> I cannot think of any UI work for HA.
> I'll let Santosh comment for FGS.
>
> Regards
> Swapnil
>
>
> On Thu, Aug 6, 2015 at 12:38 PM, Jim Klucar <kl...@gmail.com> wrote:
>
> > any UI or Rest tweaks desired for this?
> >
> > On Thu, Aug 6, 2015 at 3:01 PM, Swapnil Daingade <
> > swapnil.daingade@gmail.com
> > > wrote:
> >
> > > Just learned from Santosh that my email last night did not make it to
> the
> > > mailing list.
> > > resending...
> > >
> > > Regards
> > > Swapnil
> > >
> > >
> > > ---------- Forwarded message ----------
> > > From: Swapnil Daingade <sw...@gmail.com>
> > > Date: Thu, Aug 6, 2015 at 12:00 AM
> > > Subject: Re: Myriad 0.1 release scope
> > > To: dev@myriad.incubator.apache.org
> > >
> > >
> > >
> > > I think Myriad HA is also in good shape. We discussed work preserving
> NM
> > > restart internally and it looks like it is disabled by default in
> Apache
> > > Hadoop.
> > >
> > >
> >
> https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeManagerRestart.html
> > >
> > > All 3 major distro's seem to have it disabled by default. I think we
> can
> > > revisit it in the next Myriad release.
> > > For the time being we can continue with the NM + Executor merge. It too
> > has
> > > been tried out and reviewed by Darin and Santosh.
> > >
> > > The current PR we have for Myriad state store implementation should
> work
> > > for phase1.
> > > There is some additional work required for supporting FGS that I am
> > working
> > > on.
> > > Should be able to complete by the time FGS is merged into phase1.
> > >
> > > Regards
> > > Swapnil
> > >
> > >
> > >
> > > Santosh Marella wrote:
> > >
> > > I feel FGS is very close to making it into 0.1. PR 116 addresses moving
> > to
> > > hadoop 2.7 and making FGS and CGS coexist. This PR was recently
> reviewed
> > by
> > > Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x
> > > clusters and it seemed to have worked as expected. Unless there are
> more
> > > reviews/feedback, it can be merged into issue_14. Once PR 116 is merged
> > > into issue_14, issue_14 can be merged into phase1.
> > >
> > > Thanks,
> > > Santosh
> > >
> > > On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io>
> > > <ad...@mesosphere.io> wrote:
> > >
> > >
> > >
> > > We do have a JIRA 0.1.0 "fix version" field, but none of our issues use
> > it
> > > yet.
> > > I think the goal was just to take what we have and make it work under
> > > Apache infrastructure, then vote on that for 0.1.0.
> > > Although other features like HA or FGS would be great, let's try to get
> > our
> > > first Apache release out ASAP.
> > > We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with
> > > other issues/features. Roadmap would be great.
> > > (I'm just summarizing what we discussed a month or two ago. Feel free
> to
> > > correct me or disagree with this approach.)
> > >
> > > On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
> > > swapnil.daingade@gmail.com
> > >
> > > wrote:
> > >
> > >
> > > Hi all,
> > >
> > > Was wondering what would be the scope for the Myriad 0.1 release.
> > > It would be nice to have a roadmap page somewhere and target
> > > features to releases (JIRA 'fix version' field perhaps)
> > >
> > > Regards
> > > Swapnil
> > >
> >
>
Re: Myriad 0.1 release scope
Posted by Swapnil Daingade <sw...@gmail.com>.
Hi Jim,
I cannot think of any UI work for HA.
I'll let Santosh comment for FGS.
Regards
Swapnil
On Thu, Aug 6, 2015 at 12:38 PM, Jim Klucar <kl...@gmail.com> wrote:
> any UI or Rest tweaks desired for this?
>
> On Thu, Aug 6, 2015 at 3:01 PM, Swapnil Daingade <
> swapnil.daingade@gmail.com
> > wrote:
>
> > Just learned from Santosh that my email last night did not make it to the
> > mailing list.
> > resending...
> >
> > Regards
> > Swapnil
> >
> >
> > ---------- Forwarded message ----------
> > From: Swapnil Daingade <sw...@gmail.com>
> > Date: Thu, Aug 6, 2015 at 12:00 AM
> > Subject: Re: Myriad 0.1 release scope
> > To: dev@myriad.incubator.apache.org
> >
> >
> >
> > I think Myriad HA is also in good shape. We discussed work preserving NM
> > restart internally and it looks like it is disabled by default in Apache
> > Hadoop.
> >
> >
> https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeManagerRestart.html
> >
> > All 3 major distro's seem to have it disabled by default. I think we can
> > revisit it in the next Myriad release.
> > For the time being we can continue with the NM + Executor merge. It too
> has
> > been tried out and reviewed by Darin and Santosh.
> >
> > The current PR we have for Myriad state store implementation should work
> > for phase1.
> > There is some additional work required for supporting FGS that I am
> working
> > on.
> > Should be able to complete by the time FGS is merged into phase1.
> >
> > Regards
> > Swapnil
> >
> >
> >
> > Santosh Marella wrote:
> >
> > I feel FGS is very close to making it into 0.1. PR 116 addresses moving
> to
> > hadoop 2.7 and making FGS and CGS coexist. This PR was recently reviewed
> by
> > Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x
> > clusters and it seemed to have worked as expected. Unless there are more
> > reviews/feedback, it can be merged into issue_14. Once PR 116 is merged
> > into issue_14, issue_14 can be merged into phase1.
> >
> > Thanks,
> > Santosh
> >
> > On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io>
> > <ad...@mesosphere.io> wrote:
> >
> >
> >
> > We do have a JIRA 0.1.0 "fix version" field, but none of our issues use
> it
> > yet.
> > I think the goal was just to take what we have and make it work under
> > Apache infrastructure, then vote on that for 0.1.0.
> > Although other features like HA or FGS would be great, let's try to get
> our
> > first Apache release out ASAP.
> > We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with
> > other issues/features. Roadmap would be great.
> > (I'm just summarizing what we discussed a month or two ago. Feel free to
> > correct me or disagree with this approach.)
> >
> > On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
> > swapnil.daingade@gmail.com
> >
> > wrote:
> >
> >
> > Hi all,
> >
> > Was wondering what would be the scope for the Myriad 0.1 release.
> > It would be nice to have a roadmap page somewhere and target
> > features to releases (JIRA 'fix version' field perhaps)
> >
> > Regards
> > Swapnil
> >
>
Re: Myriad 0.1 release scope
Posted by Jim Klucar <kl...@gmail.com>.
any UI or Rest tweaks desired for this?
On Thu, Aug 6, 2015 at 3:01 PM, Swapnil Daingade <swapnil.daingade@gmail.com
> wrote:
> Just learned from Santosh that my email last night did not make it to the
> mailing list.
> resending...
>
> Regards
> Swapnil
>
>
> ---------- Forwarded message ----------
> From: Swapnil Daingade <sw...@gmail.com>
> Date: Thu, Aug 6, 2015 at 12:00 AM
> Subject: Re: Myriad 0.1 release scope
> To: dev@myriad.incubator.apache.org
>
>
>
> I think Myriad HA is also in good shape. We discussed work preserving NM
> restart internally and it looks like it is disabled by default in Apache
> Hadoop.
>
> https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeManagerRestart.html
>
> All 3 major distro's seem to have it disabled by default. I think we can
> revisit it in the next Myriad release.
> For the time being we can continue with the NM + Executor merge. It too has
> been tried out and reviewed by Darin and Santosh.
>
> The current PR we have for Myriad state store implementation should work
> for phase1.
> There is some additional work required for supporting FGS that I am working
> on.
> Should be able to complete by the time FGS is merged into phase1.
>
> Regards
> Swapnil
>
>
>
> Santosh Marella wrote:
>
> I feel FGS is very close to making it into 0.1. PR 116 addresses moving to
> hadoop 2.7 and making FGS and CGS coexist. This PR was recently reviewed by
> Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x
> clusters and it seemed to have worked as expected. Unless there are more
> reviews/feedback, it can be merged into issue_14. Once PR 116 is merged
> into issue_14, issue_14 can be merged into phase1.
>
> Thanks,
> Santosh
>
> On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io>
> <ad...@mesosphere.io> wrote:
>
>
>
> We do have a JIRA 0.1.0 "fix version" field, but none of our issues use it
> yet.
> I think the goal was just to take what we have and make it work under
> Apache infrastructure, then vote on that for 0.1.0.
> Although other features like HA or FGS would be great, let's try to get our
> first Apache release out ASAP.
> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with
> other issues/features. Roadmap would be great.
> (I'm just summarizing what we discussed a month or two ago. Feel free to
> correct me or disagree with this approach.)
>
> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
> swapnil.daingade@gmail.com
>
> wrote:
>
>
> Hi all,
>
> Was wondering what would be the scope for the Myriad 0.1 release.
> It would be nice to have a roadmap page somewhere and target
> features to releases (JIRA 'fix version' field perhaps)
>
> Regards
> Swapnil
>
Fwd: Myriad 0.1 release scope
Posted by Swapnil Daingade <sw...@gmail.com>.
Just learned from Santosh that my email last night did not make it to the
mailing list.
resending...
Regards
Swapnil
---------- Forwarded message ----------
From: Swapnil Daingade <sw...@gmail.com>
Date: Thu, Aug 6, 2015 at 12:00 AM
Subject: Re: Myriad 0.1 release scope
To: dev@myriad.incubator.apache.org
I think Myriad HA is also in good shape. We discussed work preserving NM
restart internally and it looks like it is disabled by default in Apache
Hadoop.
https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeManagerRestart.html
All 3 major distro's seem to have it disabled by default. I think we can
revisit it in the next Myriad release.
For the time being we can continue with the NM + Executor merge. It too has
been tried out and reviewed by Darin and Santosh.
The current PR we have for Myriad state store implementation should work
for phase1.
There is some additional work required for supporting FGS that I am working
on.
Should be able to complete by the time FGS is merged into phase1.
Regards
Swapnil
Santosh Marella wrote:
I feel FGS is very close to making it into 0.1. PR 116 addresses moving to
hadoop 2.7 and making FGS and CGS coexist. This PR was recently reviewed by
Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x
clusters and it seemed to have worked as expected. Unless there are more
reviews/feedback, it can be merged into issue_14. Once PR 116 is merged
into issue_14, issue_14 can be merged into phase1.
Thanks,
Santosh
On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io>
<ad...@mesosphere.io> wrote:
We do have a JIRA 0.1.0 "fix version" field, but none of our issues use it
yet.
I think the goal was just to take what we have and make it work under
Apache infrastructure, then vote on that for 0.1.0.
Although other features like HA or FGS would be great, let's try to get our
first Apache release out ASAP.
We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with
other issues/features. Roadmap would be great.
(I'm just summarizing what we discussed a month or two ago. Feel free to
correct me or disagree with this approach.)
On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <swapnil.daingade@gmail.com
wrote:
Hi all,
Was wondering what would be the scope for the Myriad 0.1 release.
It would be nice to have a roadmap page somewhere and target
features to releases (JIRA 'fix version' field perhaps)
Regards
Swapnil
Re: Myriad 0.1 release scope
Posted by Santosh Marella <sm...@maprtech.com>.
I feel FGS is very close to making it into 0.1. PR 116 addresses moving to
hadoop 2.7 and making FGS and CGS coexist. This PR was recently reviewed by
Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x
clusters and it seemed to have worked as expected. Unless there are more
reviews/feedback, it can be merged into issue_14. Once PR 116 is merged
into issue_14, issue_14 can be merged into phase1.
Thanks,
Santosh
On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <ad...@mesosphere.io> wrote:
> We do have a JIRA 0.1.0 "fix version" field, but none of our issues use it
> yet.
> I think the goal was just to take what we have and make it work under
> Apache infrastructure, then vote on that for 0.1.0.
> Although other features like HA or FGS would be great, let's try to get our
> first Apache release out ASAP.
> We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with
> other issues/features. Roadmap would be great.
> (I'm just summarizing what we discussed a month or two ago. Feel free to
> correct me or disagree with this approach.)
>
> On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <
> swapnil.daingade@gmail.com
> > wrote:
>
> > Hi all,
> >
> > Was wondering what would be the scope for the Myriad 0.1 release.
> > It would be nice to have a roadmap page somewhere and target
> > features to releases (JIRA 'fix version' field perhaps)
> >
> > Regards
> > Swapnil
> >
>
Re: Myriad 0.1 release scope
Posted by Adam Bordelon <ad...@mesosphere.io>.
We do have a JIRA 0.1.0 "fix version" field, but none of our issues use it
yet.
I think the goal was just to take what we have and make it work under
Apache infrastructure, then vote on that for 0.1.0.
Although other features like HA or FGS would be great, let's try to get our
first Apache release out ASAP.
We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with
other issues/features. Roadmap would be great.
(I'm just summarizing what we discussed a month or two ago. Feel free to
correct me or disagree with this approach.)
On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade <swapnil.daingade@gmail.com
> wrote:
> Hi all,
>
> Was wondering what would be the scope for the Myriad 0.1 release.
> It would be nice to have a roadmap page somewhere and target
> features to releases (JIRA 'fix version' field perhaps)
>
> Regards
> Swapnil
>