You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Krishna Kumar <kr...@nechclst.in> on 2009/12/08 07:58:48 UTC
some current features in hadoop
Dear All,
Can anybody please let me know about some of the current features of
hadoop on which development work is going on / or planning to go in
future, like :
1. Record append
2. Snapshot
3. Erasure coding
Etc.
Thanks and Best Regards,
Krishna Kumar
Senior Storage Engineer
Why do we have to die? If we had to die, and everything is gone after
that, then nothing else matters on this earth - everything is temporary,
at least relative to me.
DISCLAIMER:
-----------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and
intended
for the named recipient(s) only.
It shall not attach any liability on the originator or NECHCL or its
affiliates. Any views or opinions presented in
this email are solely those of the author and may not necessarily reflect the
opinions of NECHCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification,
distribution and / or publication of
this message without the prior written consent of the author of this e-mail is
strictly prohibited. If you have
received this email in error please delete it and notify the sender
immediately. .
-----------------------------------------------------------------------------------------------------------------------
Re: some current features in hadoop
Posted by Owen O'Malley <om...@apache.org>.
On Dec 8, 2009, at 1:54 AM, Krishna Kumar wrote:
> Thanks for the reply. Can you please tell me about some more aspects
> which are currently going on hadoop development from where I can
> contribute something, some to do type of things.
I'd suggest trying out the framework and writing some example code.
The best things to contribute are things that are bothering you.
*smile* I'd also suggest starting small and working up.
If you have spare cycles, it would be great if someone went through
and made sure that all of the webapps using HtmlQuoting to quote all
of the values that come from the jobs in the mapreduce webui.
Another nice one would be to implement another quoter for streaming
that does:
tab -> \t
newline -> \n
return -> \r
slash -> \\
and leaves everything else alone. That would let streaming
applications use binary data with a much simpler translation than
typedbytes. See MAPREDUCE-606.
-- Owen
Re: some current features in hadoop
Posted by Todd Lipcon <to...@cloudera.com>.
On Tue, Dec 8, 2009 at 1:54 AM, Krishna Kumar <kr...@nechclst.in>wrote:
> Hi Todd,
>
> Thanks for the reply. Can you please tell me about some more aspects
> which are currently going on hadoop development from where I can
> contribute something, some to do type of things.
>
>
Hi Krishna,
Simply check the JIRA:
https://issues.apache.org/jira/browse/HADOOP
https://issues.apache.org/jira/browse/MAPREDUCE
https://issues.apache.org/jira/browse/HDFS
There's an ability to search for unresolved issues that are updated
recently, which will give you an idea about what's in progress. You can also
subscribe to various mailing lists which will keep you up to date as people
file new JIRAs.
-Todd
> Thanks and Best Regards,
> Krishna Kumar
> Senior Storage Engineer
> Why do we have to die? If we had to die, and everything is gone after
> that, then nothing else matters on this earth - everything is temporary,
> at least relative to me.
>
>
> -----Original Message-----
> From: Todd Lipcon [mailto:todd@cloudera.com]
> Sent: Tuesday, December 08, 2009 12:48 PM
> To: common-user@hadoop.apache.org
> Subject: Re: some current features in hadoop
>
> On Mon, Dec 7, 2009 at 10:58 PM, Krishna Kumar
> <kr...@nechclst.in>wrote:
>
> > Dear All,
> >
> >
> >
> > Can anybody please let me know about some of the current features of
> > hadoop on which development work is going on / or planning to go in
> > future, like :
> >
> >
> >
> > 1. Record append
> >
> > Not implemented and not on the road map.
>
>
> > 2. Snapshot
> >
> > Not implemented and some people are vaguely interested, but not on the
> near
> term road map.
>
>
> > 3. Erasure coding
> >
> > I believe there are a couple "prototype level" implementations of
> this.
> See Dhruba's blog post here:
>
> http://hadoopblog.blogspot.com/2009/08/hdfs-and-erasure-codes-hdfs-raid.
> html<http://hadoopblog.blogspot.com/2009/08/hdfs-and-erasure-codes-hdfs-raid.%0Ahtml>
>
>
> -Todd
>
>
>
> DISCLAIMER:
>
> -----------------------------------------------------------------------------------------------------------------------
> The contents of this e-mail and any attachment(s) are confidential and
> intended
> for the named recipient(s) only.
> It shall not attach any liability on the originator or NECHCL or its
> affiliates. Any views or opinions presented in
> this email are solely those of the author and may not necessarily reflect
> the
> opinions of NECHCL or its affiliates.
> Any form of reproduction, dissemination, copying, disclosure, modification,
> distribution and / or publication of
> this message without the prior written consent of the author of this e-mail
> is
> strictly prohibited. If you have
> received this email in error please delete it and notify the sender
> immediately. .
>
> -----------------------------------------------------------------------------------------------------------------------
>
RE: some current features in hadoop
Posted by Krishna Kumar <kr...@nechclst.in>.
Hi Todd,
Thanks for the reply. Can you please tell me about some more aspects
which are currently going on hadoop development from where I can
contribute something, some to do type of things.
Thanks and Best Regards,
Krishna Kumar
Senior Storage Engineer
Why do we have to die? If we had to die, and everything is gone after
that, then nothing else matters on this earth - everything is temporary,
at least relative to me.
-----Original Message-----
From: Todd Lipcon [mailto:todd@cloudera.com]
Sent: Tuesday, December 08, 2009 12:48 PM
To: common-user@hadoop.apache.org
Subject: Re: some current features in hadoop
On Mon, Dec 7, 2009 at 10:58 PM, Krishna Kumar
<kr...@nechclst.in>wrote:
> Dear All,
>
>
>
> Can anybody please let me know about some of the current features of
> hadoop on which development work is going on / or planning to go in
> future, like :
>
>
>
> 1. Record append
>
> Not implemented and not on the road map.
> 2. Snapshot
>
> Not implemented and some people are vaguely interested, but not on the
near
term road map.
> 3. Erasure coding
>
> I believe there are a couple "prototype level" implementations of
this.
See Dhruba's blog post here:
http://hadoopblog.blogspot.com/2009/08/hdfs-and-erasure-codes-hdfs-raid.
html
-Todd
DISCLAIMER:
-----------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and
intended
for the named recipient(s) only.
It shall not attach any liability on the originator or NECHCL or its
affiliates. Any views or opinions presented in
this email are solely those of the author and may not necessarily reflect the
opinions of NECHCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification,
distribution and / or publication of
this message without the prior written consent of the author of this e-mail is
strictly prohibited. If you have
received this email in error please delete it and notify the sender
immediately. .
-----------------------------------------------------------------------------------------------------------------------
Re: some current features in hadoop
Posted by Todd Lipcon <to...@cloudera.com>.
On Mon, Dec 7, 2009 at 10:58 PM, Krishna Kumar <kr...@nechclst.in>wrote:
> Dear All,
>
>
>
> Can anybody please let me know about some of the current features of
> hadoop on which development work is going on / or planning to go in
> future, like :
>
>
>
> 1. Record append
>
> Not implemented and not on the road map.
> 2. Snapshot
>
> Not implemented and some people are vaguely interested, but not on the near
term road map.
> 3. Erasure coding
>
> I believe there are a couple "prototype level" implementations of this.
See Dhruba's blog post here:
http://hadoopblog.blogspot.com/2009/08/hdfs-and-erasure-codes-hdfs-raid.html
-Todd