You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@htrace.apache.org by Madhawa Kasun Gunasekara <ma...@gmail.com> on 2016/03/17 06:21:56 UTC
Gsoc 2016 with apache HTrace
Hi,
I am a final year student in IESL College of Engineering, Sri lanka. I am
interested with working with apache htrace. I'm interested in "Add HTrace
distributed tracing for s3 and other alternative Hadoop FS implementations"
project [1]. Please kindly give me further information on how I could
proceed.
[1] https://issues.apache.org/jira/browse/COMDEV-191
Thanks
Madhawa
Re: Gsoc 2016 with apache HTrace
Posted by Madhawa Kasun Gunasekara <ma...@gmail.com>.
Hi All,
Please find the proposal here [1].
[1]
https://docs.google.com/document/d/1l9b_FLVT3qqXqM6LEJ6yvpLw4_UqMQcn1-JxTKSbD94/edit?usp=sharing
Thanks,
Madhawa
Madhawa
On Mon, Mar 21, 2016 at 11:16 PM, Madhawa Kasun Gunasekara <
madhawa30@gmail.com> wrote:
> Hi Colin,
>
> I would like debug the code and understand the architecture of this s3A
> connectors.
>
> Can you please explain me how to do that? (any code snippet / tutorial)
>
> Thanks,
> Madhawa
>
>
> Madhawa
>
> On Fri, Mar 18, 2016 at 11:52 PM, Colin P. McCabe <cm...@apache.org>
> wrote:
>
>> Thanks for your interest, Madhawa!
>>
>> As a next step, I suggest filing a JIRA on the HADOOP bug tracker.
>> This bug tracker is used for things in the "Hadoop common" subproject.
>> This is where the code for the s3 connector and other filesystem
>> connectors live. Just add a few paragraphs of description of what
>> you'd like to do.
>>
>> Note that Hadoop actually has three different s3 connectors at the
>> moment. "s3", "s3n", and "s3a". s3a is the most modern connector and
>> probably the one that you should focus on. The other connectors are
>> older and we don't recommend using them.
>>
>> The code is here:
>>
>> https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
>>
>> Once you feel confident in the scope of your proposal, you can fill
>> out an application as described here:
>> http://community.apache.org/gsoc.html
>> See the "Application template" section.
>>
>> There's a timeline here:
>> https://developers.google.com/open-source/gsoc/timeline. I believe
>> the application deadline is March 25.
>>
>> best,
>> Colin
>>
>> On Thu, Mar 17, 2016 at 12:38 PM, Madhawa Kasun Gunasekara
>> <ma...@gmail.com> wrote:
>> > Hi Colin,
>> >
>> > Thanks for the response, yes I have seen the value of this project.
>> > I would like to know the suggestions, before drafting the project
>> proposal.
>> >
>> > Please kindly give me further information on how I could proceed.
>> >
>> > Thanks,
>> > Madhawa
>> >
>> > Madhawa
>> >
>> > On Thu, Mar 17, 2016 at 11:54 PM, Colin P. McCabe <cm...@apache.org>
>> > wrote:
>> >>
>> >> Hi Madhawa,
>> >>
>> >> s3 and other alternate FS connector support for HTrace is a really
>> >> great project, I think. The connector code is getting a lot of
>> >> real-world use, and it would be interesting to get performance numbers
>> >> here. It would be especially interesting to see some graphs of
>> >> latencies for various operations. I'll ask some folks I know who are
>> >> working on the connector code if they have any suggestions.
>> >>
>> >> best,
>> >> Colin
>> >>
>> >> On Wed, Mar 16, 2016 at 10:21 PM, Madhawa Kasun Gunasekara
>> >> <ma...@gmail.com> wrote:
>> >> > Hi,
>> >> >
>> >> > I am a final year student in IESL College of Engineering, Sri lanka.
>> I
>> >> > am
>> >> > interested with working with apache htrace. I'm interested in "Add
>> >> > HTrace
>> >> > distributed tracing for s3 and other alternative Hadoop FS
>> >> > implementations"
>> >> > project [1]. Please kindly give me further information on how I could
>> >> > proceed.
>> >> >
>> >> > [1] https://issues.apache.org/jira/browse/COMDEV-191
>> >> >
>> >> > Thanks
>> >> >
>> >> > Madhawa
>> >
>> >
>>
>
>
Re: Gsoc 2016 with apache HTrace
Posted by Madhawa Kasun Gunasekara <ma...@gmail.com>.
Hi Colin,
I would like debug the code and understand the architecture of this s3A
connectors.
Can you please explain me how to do that? (any code snippet / tutorial)
Thanks,
Madhawa
Madhawa
On Fri, Mar 18, 2016 at 11:52 PM, Colin P. McCabe <cm...@apache.org>
wrote:
> Thanks for your interest, Madhawa!
>
> As a next step, I suggest filing a JIRA on the HADOOP bug tracker.
> This bug tracker is used for things in the "Hadoop common" subproject.
> This is where the code for the s3 connector and other filesystem
> connectors live. Just add a few paragraphs of description of what
> you'd like to do.
>
> Note that Hadoop actually has three different s3 connectors at the
> moment. "s3", "s3n", and "s3a". s3a is the most modern connector and
> probably the one that you should focus on. The other connectors are
> older and we don't recommend using them.
>
> The code is here:
>
> https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
>
> Once you feel confident in the scope of your proposal, you can fill
> out an application as described here:
> http://community.apache.org/gsoc.html
> See the "Application template" section.
>
> There's a timeline here:
> https://developers.google.com/open-source/gsoc/timeline. I believe
> the application deadline is March 25.
>
> best,
> Colin
>
> On Thu, Mar 17, 2016 at 12:38 PM, Madhawa Kasun Gunasekara
> <ma...@gmail.com> wrote:
> > Hi Colin,
> >
> > Thanks for the response, yes I have seen the value of this project.
> > I would like to know the suggestions, before drafting the project
> proposal.
> >
> > Please kindly give me further information on how I could proceed.
> >
> > Thanks,
> > Madhawa
> >
> > Madhawa
> >
> > On Thu, Mar 17, 2016 at 11:54 PM, Colin P. McCabe <cm...@apache.org>
> > wrote:
> >>
> >> Hi Madhawa,
> >>
> >> s3 and other alternate FS connector support for HTrace is a really
> >> great project, I think. The connector code is getting a lot of
> >> real-world use, and it would be interesting to get performance numbers
> >> here. It would be especially interesting to see some graphs of
> >> latencies for various operations. I'll ask some folks I know who are
> >> working on the connector code if they have any suggestions.
> >>
> >> best,
> >> Colin
> >>
> >> On Wed, Mar 16, 2016 at 10:21 PM, Madhawa Kasun Gunasekara
> >> <ma...@gmail.com> wrote:
> >> > Hi,
> >> >
> >> > I am a final year student in IESL College of Engineering, Sri lanka. I
> >> > am
> >> > interested with working with apache htrace. I'm interested in "Add
> >> > HTrace
> >> > distributed tracing for s3 and other alternative Hadoop FS
> >> > implementations"
> >> > project [1]. Please kindly give me further information on how I could
> >> > proceed.
> >> >
> >> > [1] https://issues.apache.org/jira/browse/COMDEV-191
> >> >
> >> > Thanks
> >> >
> >> > Madhawa
> >
> >
>
Re: Gsoc 2016 with apache HTrace
Posted by "Colin P. McCabe" <cm...@apache.org>.
Thanks for your interest, Madhawa!
As a next step, I suggest filing a JIRA on the HADOOP bug tracker.
This bug tracker is used for things in the "Hadoop common" subproject.
This is where the code for the s3 connector and other filesystem
connectors live. Just add a few paragraphs of description of what
you'd like to do.
Note that Hadoop actually has three different s3 connectors at the
moment. "s3", "s3n", and "s3a". s3a is the most modern connector and
probably the one that you should focus on. The other connectors are
older and we don't recommend using them.
The code is here:
https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
Once you feel confident in the scope of your proposal, you can fill
out an application as described here:
http://community.apache.org/gsoc.html
See the "Application template" section.
There's a timeline here:
https://developers.google.com/open-source/gsoc/timeline. I believe
the application deadline is March 25.
best,
Colin
On Thu, Mar 17, 2016 at 12:38 PM, Madhawa Kasun Gunasekara
<ma...@gmail.com> wrote:
> Hi Colin,
>
> Thanks for the response, yes I have seen the value of this project.
> I would like to know the suggestions, before drafting the project proposal.
>
> Please kindly give me further information on how I could proceed.
>
> Thanks,
> Madhawa
>
> Madhawa
>
> On Thu, Mar 17, 2016 at 11:54 PM, Colin P. McCabe <cm...@apache.org>
> wrote:
>>
>> Hi Madhawa,
>>
>> s3 and other alternate FS connector support for HTrace is a really
>> great project, I think. The connector code is getting a lot of
>> real-world use, and it would be interesting to get performance numbers
>> here. It would be especially interesting to see some graphs of
>> latencies for various operations. I'll ask some folks I know who are
>> working on the connector code if they have any suggestions.
>>
>> best,
>> Colin
>>
>> On Wed, Mar 16, 2016 at 10:21 PM, Madhawa Kasun Gunasekara
>> <ma...@gmail.com> wrote:
>> > Hi,
>> >
>> > I am a final year student in IESL College of Engineering, Sri lanka. I
>> > am
>> > interested with working with apache htrace. I'm interested in "Add
>> > HTrace
>> > distributed tracing for s3 and other alternative Hadoop FS
>> > implementations"
>> > project [1]. Please kindly give me further information on how I could
>> > proceed.
>> >
>> > [1] https://issues.apache.org/jira/browse/COMDEV-191
>> >
>> > Thanks
>> >
>> > Madhawa
>
>
Re: Gsoc 2016 with apache HTrace
Posted by Madhawa Kasun Gunasekara <ma...@gmail.com>.
Hi Colin,
Thanks for the response, yes I have seen the value of this project.
I would like to know the suggestions, before drafting the project proposal.
Please kindly give me further information on how I could proceed.
Thanks,
Madhawa
Madhawa
On Thu, Mar 17, 2016 at 11:54 PM, Colin P. McCabe <cm...@apache.org>
wrote:
> Hi Madhawa,
>
> s3 and other alternate FS connector support for HTrace is a really
> great project, I think. The connector code is getting a lot of
> real-world use, and it would be interesting to get performance numbers
> here. It would be especially interesting to see some graphs of
> latencies for various operations. I'll ask some folks I know who are
> working on the connector code if they have any suggestions.
>
> best,
> Colin
>
> On Wed, Mar 16, 2016 at 10:21 PM, Madhawa Kasun Gunasekara
> <ma...@gmail.com> wrote:
> > Hi,
> >
> > I am a final year student in IESL College of Engineering, Sri lanka. I am
> > interested with working with apache htrace. I'm interested in "Add HTrace
> > distributed tracing for s3 and other alternative Hadoop FS
> implementations"
> > project [1]. Please kindly give me further information on how I could
> > proceed.
> >
> > [1] https://issues.apache.org/jira/browse/COMDEV-191
> >
> > Thanks
> >
> > Madhawa
>
Re: Gsoc 2016 with apache HTrace
Posted by "Colin P. McCabe" <cm...@apache.org>.
Hi Madhawa,
s3 and other alternate FS connector support for HTrace is a really
great project, I think. The connector code is getting a lot of
real-world use, and it would be interesting to get performance numbers
here. It would be especially interesting to see some graphs of
latencies for various operations. I'll ask some folks I know who are
working on the connector code if they have any suggestions.
best,
Colin
On Wed, Mar 16, 2016 at 10:21 PM, Madhawa Kasun Gunasekara
<ma...@gmail.com> wrote:
> Hi,
>
> I am a final year student in IESL College of Engineering, Sri lanka. I am
> interested with working with apache htrace. I'm interested in "Add HTrace
> distributed tracing for s3 and other alternative Hadoop FS implementations"
> project [1]. Please kindly give me further information on how I could
> proceed.
>
> [1] https://issues.apache.org/jira/browse/COMDEV-191
>
> Thanks
>
> Madhawa