Re: Introducing myself and adding QFS [WARNING: DKIM validation failed]

[Faraaz Sareshwala <fs...@quantcast.com>]

Hi RJ,

Yes, I’ll be adding myself as maintainer for QFS. You can reach out to me if there are ever any issues.

Faraaz



On 1/26/16, 7:04 PM, "RJ Nowling" <rn...@gmail.com> wrote:

>Oops... meant QFS, not Apache Apex.  Sorry!  But the question about having
>a maintainer still stands.  :)
>
>On Tue, Jan 26, 2016 at 9:02 PM, RJ Nowling <rn...@gmail.com> wrote:
>
>> Hi Faraaz,
>>
>> Is there someone who would be willing to take responsibility for
>> maintaining the Apache Apex packages in Bigtop?
>>
>> We generally like to find a maintainer for each package and add them to
>> our maintainers list.  This way we know who to contact if a build fails and
>> blocks a release.  If there is no maintainer (and the build has problems),
>> we use that as grounds for removing the package from Bigtop.
>>
>> Thanks!
>> RJ
>>
>> On Tue, Jan 26, 2016 at 3:05 PM, Faraaz Sareshwala <
>> fsareshwala@quantcast.com> wrote:
>>
>>> Thanks for the warm welcome guys :).
>>>
>>> I agree that smaller patches will be easier to parse and review. Right
>>> now, I have Debian and RPM packaging complete. I have created a JIRA ticket
>>> (https://issues.apache.org/jira/browse/BIGTOP-2283) which I will use to
>>> track the integration.
>>>
>>> I’ll start sending in pull requests as I complete each of the tasks
>>> necessary to get this integration done. I’d love all of your guys’ guidance
>>> along the way so that the integration can be as correct to bigtop’s
>>> standards as possible.
>>>
>>> Also, regarding the single point of failure sidenote discussed in another
>>> thread, Quantcast is actively working on updating qfs to support a
>>> distributed metaserver so that we can avoid the single point of failure.
>>> It’s not ready for release yet but we will announce it when it is :).
>>>
>>>
>>> Faraaz
>>>
>>> On 1/26/16, 11:33 AM, "jay vyas" <ja...@gmail.com> wrote:
>>>
>>> >Please add Your name to the MAINTAINERS.txt file in the PR for "QFS" !
>>> >
>>> >On Tue, Jan 26, 2016 at 2:24 PM, Konstantin Boudnik <co...@apache.org>
>>> wrote:
>>> >
>>> >> On Wed, Jan 27, 2016 at 01:05AM, Evans Ye wrote:
>>> >> > Oh, if I made you uncomfortable,  you know I didn't mean that :)
>>> >>
>>> >> Oh believe me - it takes a lot more than that to make me uncomfortable
>>> ;)
>>> >> Besides, you gave me an opportunity to speak up about the SPOFs in the
>>> >> Hadoop
>>> >> ecosystem. Which I can do for hours!
>>> >>
>>> >> > I just want to know more about the design because I'm always
>>> interested
>>> >> in
>>> >> > the architecture things, especially when I see something different.
>>> >> > Even Spark just added HA in their master in recent release last
>>> year. So
>>> >> > it's totally fine.
>>> >> >
>>> >> > And the following is just  for the discussion:
>>> >> >
>>> >> >> Just a side node: as you know literally _all_ components in
>>> Hadoop-based
>>> >> >> stack have SPOF.
>>> >> >
>>> >> > I think it's more related to the maturity of a component, but
>>> somehow it
>>> >> > will get there to eliminate the SPOF just like what HDFS, YARN,
>>> Oozie,
>>> >> Spark
>>> >> > did.  For operations, two SPOFs is good enough for them to have a
>>> good
>>> >> sleep
>>> >> > at night ;)
>>> >>
>>> >> It might be ok for Oozie, but not for a file system. Because once your
>>> >> master
>>> >> has failed over to the standby you don't know how long that one has. It
>>> >> might
>>> >> crash in 5 months or in 2 minutes. That's why if you're an admin for a
>>> >> production critical cluster you'll get paged immediately. Perhaps 3
>>> >> o'clock in
>>> >> the morning.
>>> >>
>>> >> I spent last three years working on the active-active software for
>>> mission
>>> >> critical systems, so I would love to share some architectural thoughts
>>> with
>>> >> you, if you're interested.
>>> >>
>>> >> Cos
>>> >>
>>> >> > 2016-01-26 12:08 GMT+08:00 Jay Vyas <ja...@gmail.com>:
>>> >> >
>>> >> > > Awesome!  It's great to hear about another HDFS alternative :)
>>> >> > >
>>> >> > > , We tried pretty hard to make bigtop more "HCFS" compliant, by
>>> adding
>>> >> the
>>> >> > > init hcfs tooling and so on, but never completed making the puppet
>>> >> recipes
>>> >> > > add support for other file systems.
>>> >> > >
>>> >> > >
>>> >> > > Would be very interesting to see if the quant cast team can take
>>> over
>>> >> on
>>> >> > > the HCFS integration so that bigtop is a home for more Than just
>>> one
>>> >> under
>>> >> > > filesystem .
>>> >> > >
>>> >> > > Let us know what your planning on working on, looking forward to
>>> it...!
>>> >> > >
>>> >> > >
>>> >> > >
>>> >> > >
>>> >> > > > On Jan 25, 2016, at 9:06 PM, Faraaz Sareshwala <
>>> >> > > fsareshwala@quantcast.com> wrote:
>>> >> > > >
>>> >> > > > Hello everyone in the Bigtop community!
>>> >> > > >
>>> >> > > > My name is Faraaz Sareshwala and I am an engineer working at
>>> >> Quantcast
>>> >> > > to integrate QFS (https://quantcast.github.io/qfs) into the Bigtop
>>> >> > > project. I just wanted to send a small note out to introduce
>>> myself to
>>> >> the
>>> >> > > community and announce that we are working on an integration. Feel
>>> >> free to
>>> >> > > reach out to me if you have any questions about QFS, our
>>> integration,
>>> >> or
>>> >> > > Quantcast itself.
>>> >> > > >
>>> >> > > > At this point, I am nearly complete integrating QFS. I plan to
>>> >> submit a
>>> >> > > patch through github and JIRA later in the week or early next week
>>> >> once I
>>> >> > > get a chance to test everything is working correctly. I’ve read the
>>> >> wiki on
>>> >> > > contribution to the project, but if you guys have any other tips or
>>> >> > > guidance for me, I’d really appreciate that.
>>> >> > > >
>>> >> > > > Patch coming soon!
>>> >> > > >
>>> >> > > > Take care,
>>> >> > > > Faraaz
>>> >> > > >
>>> >> > >
>>> >>
>>> >
>>> >
>>> >
>>> >--
>>> >jay vyas
>>>
>>
>>