You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Sreejith Ramakrishnan <sr...@gmail.com> on 2013/07/17 22:44:39 UTC
I'm interested in working with HDFS-4680. Can somebody be a mentor?
Hey,
I was originally researching options to work on ACCUMULO-1197. Basically,
it was a bid to pass trace functionality through the DFSClient. I discussed
with the guys over there on implementing a Google Dapper-style trace with
HTrace. The guys at HBase are also trying to achieve the same HTrace
integration [HBASE-6449]
But, that meant adding stuff to the RPC in HDFS. For a start, we've to add
a 64-bit span-id to every RPC with tracing enabled. There's some more in
the original Dapper paper and HTrace documentation.
I was told by the Accumulo people to talk with and seek help from the
experts at HDFS. I'm open to suggestions.
Additionally, I'm participating in a Joint Mentoring Programme by Apache
which is quite similar to GSoC. Luciano Resende (Community Development,
Apache) is incharge of the programme. I'll attach a link. The last date is
19th July. So, I'm pretty tensed without any mentors :(
[1] https://issues.apache.org/jira/browse/ACCUMULO-1197
[2] https://issues.apache.org/jira/browse/HDFS-4680
[3] https://github.com/cloudera/htrace
[4] http://community.apache.org/mentoringprogramme-icfoss-pilot.html
[5] https://issues.apache.org/jira/browse/HBASE-6449
Thank you,
Sreejith R
Re: I'm interested in working with HDFS-4680. Can somebody be a mentor?
Posted by Colin McCabe <cm...@alumni.cmu.edu>.
Andrew Wang has been working on getting this kind of Dapper-style
trace functionality in HDFS. He is on vacation this week, but next
week he might have some ideas about how you could contribute and/or
integrate with his patch. Doing this right with security, etc is a
pretty big project and I think he wanted to do it incrementally.
best,
Colin McCabe
On Wed, Jul 17, 2013 at 1:44 PM, Sreejith Ramakrishnan
<sr...@gmail.com> wrote:
> Hey,
>
> I was originally researching options to work on ACCUMULO-1197. Basically,
> it was a bid to pass trace functionality through the DFSClient. I discussed
> with the guys over there on implementing a Google Dapper-style trace with
> HTrace. The guys at HBase are also trying to achieve the same HTrace
> integration [HBASE-6449]
>
> But, that meant adding stuff to the RPC in HDFS. For a start, we've to add
> a 64-bit span-id to every RPC with tracing enabled. There's some more in
> the original Dapper paper and HTrace documentation.
>
> I was told by the Accumulo people to talk with and seek help from the
> experts at HDFS. I'm open to suggestions.
>
> Additionally, I'm participating in a Joint Mentoring Programme by Apache
> which is quite similar to GSoC. Luciano Resende (Community Development,
> Apache) is incharge of the programme. I'll attach a link. The last date is
> 19th July. So, I'm pretty tensed without any mentors :(
>
> [1] https://issues.apache.org/jira/browse/ACCUMULO-1197
> [2] https://issues.apache.org/jira/browse/HDFS-4680
> [3] https://github.com/cloudera/htrace
> [4] http://community.apache.org/mentoringprogramme-icfoss-pilot.html
> [5] https://issues.apache.org/jira/browse/HBASE-6449
>
> Thank you,
> Sreejith R
Re: I'm interested in working with HDFS-4680. Can somebody be a mentor?
Posted by Sreejith Ramakrishnan <sr...@gmail.com>.
Being a mentor, the duty is to introduce the candidate to open source
development and guide him in aspects related to how open source works. Like
how to make patches, how to ask for help in a mailing list etc. That's the
essence.
I'm ready to work on this anyway. But, if somebody could be my mentor, I
could also be part of the programme and can have something to add on the
resume too when this ends. Please do tell me if someone is interested. The
deadline is 19th July which is tonight :(
On Thu, Jul 18, 2013 at 5:59 AM, Suresh Srinivas <su...@hortonworks.com>wrote:
> Please look at some of the work happening in HADOOP-9688, which is adding a
> unique UUID (16 bytes) for each RPC request. This is common to all Hadoop
> RPC, will be available in HDFS, YARN and MAPREDUCE. Please see the jira for
> more details. Reach out to me if you have any questions.
>
>
> On Wed, Jul 17, 2013 at 1:44 PM, Sreejith Ramakrishnan <
> sreejith.code@gmail.com> wrote:
>
> > Hey,
> >
> > I was originally researching options to work on ACCUMULO-1197. Basically,
> > it was a bid to pass trace functionality through the DFSClient. I
> discussed
> > with the guys over there on implementing a Google Dapper-style trace with
> > HTrace. The guys at HBase are also trying to achieve the same HTrace
> > integration [HBASE-6449]
> >
> > But, that meant adding stuff to the RPC in HDFS. For a start, we've to
> add
> > a 64-bit span-id to every RPC with tracing enabled. There's some more in
> > the original Dapper paper and HTrace documentation.
> >
> > I was told by the Accumulo people to talk with and seek help from the
> > experts at HDFS. I'm open to suggestions.
> >
> > Additionally, I'm participating in a Joint Mentoring Programme by Apache
> > which is quite similar to GSoC. Luciano Resende (Community Development,
> > Apache) is incharge of the programme. I'll attach a link. The last date
> is
> > 19th July. So, I'm pretty tensed without any mentors :(
> >
> > [1] https://issues.apache.org/jira/browse/ACCUMULO-1197
> > [2] https://issues.apache.org/jira/browse/HDFS-4680
> > [3] https://github.com/cloudera/htrace
> > [4] http://community.apache.org/mentoringprogramme-icfoss-pilot.html
> > [5] https://issues.apache.org/jira/browse/HBASE-6449
> >
> > Thank you,
> > Sreejith R
> >
>
>
>
> --
> http://hortonworks.com/download/
>
Re: I'm interested in working with HDFS-4680. Can somebody be a mentor?
Posted by Suresh Srinivas <su...@hortonworks.com>.
Please look at some of the work happening in HADOOP-9688, which is adding a
unique UUID (16 bytes) for each RPC request. This is common to all Hadoop
RPC, will be available in HDFS, YARN and MAPREDUCE. Please see the jira for
more details. Reach out to me if you have any questions.
On Wed, Jul 17, 2013 at 1:44 PM, Sreejith Ramakrishnan <
sreejith.code@gmail.com> wrote:
> Hey,
>
> I was originally researching options to work on ACCUMULO-1197. Basically,
> it was a bid to pass trace functionality through the DFSClient. I discussed
> with the guys over there on implementing a Google Dapper-style trace with
> HTrace. The guys at HBase are also trying to achieve the same HTrace
> integration [HBASE-6449]
>
> But, that meant adding stuff to the RPC in HDFS. For a start, we've to add
> a 64-bit span-id to every RPC with tracing enabled. There's some more in
> the original Dapper paper and HTrace documentation.
>
> I was told by the Accumulo people to talk with and seek help from the
> experts at HDFS. I'm open to suggestions.
>
> Additionally, I'm participating in a Joint Mentoring Programme by Apache
> which is quite similar to GSoC. Luciano Resende (Community Development,
> Apache) is incharge of the programme. I'll attach a link. The last date is
> 19th July. So, I'm pretty tensed without any mentors :(
>
> [1] https://issues.apache.org/jira/browse/ACCUMULO-1197
> [2] https://issues.apache.org/jira/browse/HDFS-4680
> [3] https://github.com/cloudera/htrace
> [4] http://community.apache.org/mentoringprogramme-icfoss-pilot.html
> [5] https://issues.apache.org/jira/browse/HBASE-6449
>
> Thank you,
> Sreejith R
>
--
http://hortonworks.com/download/
Re: I'm interested in working with HDFS-4680. Can somebody be a mentor?
Posted by Todd Lipcon <to...@cloudera.com>.
I'm happy to help with this as well. I actually have a prototype patch that
I built during a hackathon a few months ago, and was able to get a full
stack trace including Client, NN, and DN. I'm on vacation this week but
will try to post my prototype upstream when I get back. Feel free to ping
me on this if I slack :)
-Todd
On Wed, Jul 17, 2013 at 4:12 PM, Stack <st...@duboce.net> wrote:
> Folks over at HBase would be interested in helping out.
>
> What does a mentor have to do? I poked around the icfoss link but didn't
> see list of duties (I've been know to be certified blind on occasion).
>
> I am not up on the malleability of hdfs RPC; is it just a matter of adding
> the trace info to a pb header record or would it require more (Sanjay was
> saying something recently off-list that trace id is imminent -- but I've
> not done the digging)?
>
> St.Ack
>
>
> On Wed, Jul 17, 2013 at 1:44 PM, Sreejith Ramakrishnan <
> sreejith.code@gmail.com> wrote:
>
> > Hey,
> >
> > I was originally researching options to work on ACCUMULO-1197. Basically,
> > it was a bid to pass trace functionality through the DFSClient. I
> discussed
> > with the guys over there on implementing a Google Dapper-style trace with
> > HTrace. The guys at HBase are also trying to achieve the same HTrace
> > integration [HBASE-6449]
> >
> > But, that meant adding stuff to the RPC in HDFS. For a start, we've to
> add
> > a 64-bit span-id to every RPC with tracing enabled. There's some more in
> > the original Dapper paper and HTrace documentation.
> >
> > I was told by the Accumulo people to talk with and seek help from the
> > experts at HDFS. I'm open to suggestions.
> >
> > Additionally, I'm participating in a Joint Mentoring Programme by Apache
> > which is quite similar to GSoC. Luciano Resende (Community Development,
> > Apache) is incharge of the programme. I'll attach a link. The last date
> is
> > 19th July. So, I'm pretty tensed without any mentors :(
> >
> > [1] https://issues.apache.org/jira/browse/ACCUMULO-1197
> > [2] https://issues.apache.org/jira/browse/HDFS-4680
> > [3] https://github.com/cloudera/htrace
> > [4] http://community.apache.org/mentoringprogramme-icfoss-pilot.html
> > [5] https://issues.apache.org/jira/browse/HBASE-6449
> >
> > Thank you,
> > Sreejith R
> >
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: I'm interested in working with HDFS-4680. Can somebody be a mentor?
Posted by Stack <st...@duboce.net>.
Folks over at HBase would be interested in helping out.
What does a mentor have to do? I poked around the icfoss link but didn't
see list of duties (I've been know to be certified blind on occasion).
I am not up on the malleability of hdfs RPC; is it just a matter of adding
the trace info to a pb header record or would it require more (Sanjay was
saying something recently off-list that trace id is imminent -- but I've
not done the digging)?
St.Ack
On Wed, Jul 17, 2013 at 1:44 PM, Sreejith Ramakrishnan <
sreejith.code@gmail.com> wrote:
> Hey,
>
> I was originally researching options to work on ACCUMULO-1197. Basically,
> it was a bid to pass trace functionality through the DFSClient. I discussed
> with the guys over there on implementing a Google Dapper-style trace with
> HTrace. The guys at HBase are also trying to achieve the same HTrace
> integration [HBASE-6449]
>
> But, that meant adding stuff to the RPC in HDFS. For a start, we've to add
> a 64-bit span-id to every RPC with tracing enabled. There's some more in
> the original Dapper paper and HTrace documentation.
>
> I was told by the Accumulo people to talk with and seek help from the
> experts at HDFS. I'm open to suggestions.
>
> Additionally, I'm participating in a Joint Mentoring Programme by Apache
> which is quite similar to GSoC. Luciano Resende (Community Development,
> Apache) is incharge of the programme. I'll attach a link. The last date is
> 19th July. So, I'm pretty tensed without any mentors :(
>
> [1] https://issues.apache.org/jira/browse/ACCUMULO-1197
> [2] https://issues.apache.org/jira/browse/HDFS-4680
> [3] https://github.com/cloudera/htrace
> [4] http://community.apache.org/mentoringprogramme-icfoss-pilot.html
> [5] https://issues.apache.org/jira/browse/HBASE-6449
>
> Thank you,
> Sreejith R
>