You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Ted Yu <yu...@gmail.com> on 2011/12/30 11:14:13 UTC
HBase support in SHDP Was: Hadoop with Spring / Guice
Hi, Costin:
I work on HBase.
I went over
http://static.springsource.org/spring-hadoop/docs/current/reference/hbase.htmlbut
didn't have time to download the source code.
Is there a typo: 'does more then easily'
Should 'then' be 'than' ?
For the following config:
<hdp:hbase-configuration stop-proxy="false" delete-connection="true" />
May I ask what would the proxies do without HBase connection ?
What HBase version does SHDP work with ? I assume it is 0.90.x
0.92 is coming out in January which has more features than 0.90.x
If there is anything desirable from HBase, please let us know.
We will try to accommodate your requests.
Cheers
On Fri, Dec 30, 2011 at 1:53 AM, Costin Leau <co...@gmail.com> wrote:
> Hi,
>
> My name is Costin Leau and I'm the lead of Spring Hadoop (SHDP) project.
>
> SHDP provides DI support allowing basic POJOs to be used as
> mapper/reducers. This feature is currently developed on a dedicated
> branch [1] and we plan to merge it in master in the near future.
>
> In addition to the pojo/DI support, you might be interested in the SHDP
> integration with HBase/Hive/Pig - see [2]
>
> Everything is open-source, under ASL 2, available on github. We plan to
> release the milestone in Jan/Feb 2012.
>
> Feedback and ideas are more then welcome!
>
> [1] https://github.com/SpringSource/spring-hadoop/tree/mr-pojo
> [2] http://static.springsource.org/spring-hadoop/docs/current/reference/
>
> Hope this helps,
> --
> Costin
>
Re: HBase support in SHDP Was: Hadoop with Spring / Guice
Posted by Costin Leau <co...@gmail.com>.
On 12/30/2011 12:43 PM, Ted Yu wrote:
> Costin:
> Please include user@hbase.apache.org <ma...@hbase.apache.org> in
> your reply.
> My discussion would mostly be HBase-related.
# included hbase though I'm not subscribed to that list so I don't think
my messages will go through.
>
> I guess you're dealing with HBase 0.90.4 where connection sharing
> provided by HBASE-3777 is absent.
> HBase 0.90.5, just released, contains HBASE-4508 which backports
> HBASE-3777 to 0.90 codebase.
>
> Please kindly indicate whether the presence of HBASE-4508 would simplify
> hbase-configuration bean config.
> If it doesn't, I wonder what proxies would be doing without HBase
> connection.
>
I need to spend a bit more time to go through the issues you mentioned
to understand them better however at first glance looks like a valuable
improvement.
As for the proxy flag, this translate to the stopProxy argument on
HConnectionManager.deleteConnection() [1]
[1] http://j.mp/twrsH8
> Cheers
>
> On Fri, Dec 30, 2011 at 2:31 AM, Costin Leau <costin.leau@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi,
>
> Thanks for spotting the typo.
>
> We're currently running against 0.90.x. As for the lack of connection,
> we use HConnectionManager in the background to clean connections up as
> it already provides tracking and a cleanup mechanism (based on the
> HbaseConfiguration object).
>
> Cheers and keep the feedback coming,
>
> P.S. just want to point out there's a dedicated forum for SHDP, if the
> discussion becomes too Spring specific.
>
> [1] http://forum.springsource.org/forumdisplay.php?80-NoSQL
>
> On 12/30/2011 12:14 PM, Ted Yu wrote:
> > Hi, Costin:
> > I work on HBase.
> >
> > I went over
> >
> http://static.springsource.org/spring-hadoop/docs/current/reference/hbase.html
> > but didn't have time to download the source code.
> >
> > Is there a typo: 'does more then easily'
> > Should 'then' be 'than' ?
> >
> > For the following config:
> > <hdp:hbase-configuration stop-proxy="false" delete-connection="true"/>
> >
> > May I ask what would the proxies do without HBase connection ?
> >
> > What HBase version does SHDP work with ? I assume it is 0.90.x
> > 0.92 is coming out in January which has more features than 0.90.x
> >
> > If there is anything desirable from HBase, please let us know.
> > We will try to accommodate your requests.
> >
> > Cheers
> >
> > On Fri, Dec 30, 2011 at 1:53 AM, Costin Leau
> <costin.leau@gmail.com <ma...@gmail.com>
> > <mailto:costin.leau@gmail.com <ma...@gmail.com>>> wrote:
> >
> > Hi,
> >
> > My name is Costin Leau and I'm the lead of Spring Hadoop
> (SHDP) project.
> >
> > SHDP provides DI support allowing basic POJOs to be used as
> > mapper/reducers. This feature is currently developed on a
> dedicated
> > branch [1] and we plan to merge it in master in the near future.
> >
> > In addition to the pojo/DI support, you might be interested in
> the SHDP
> > integration with HBase/Hive/Pig - see [2]
> >
> > Everything is open-source, under ASL 2, available on github.
> We plan to
> > release the milestone in Jan/Feb 2012.
> >
> > Feedback and ideas are more then welcome!
> >
> > [1] https://github.com/SpringSource/spring-hadoop/tree/mr-pojo
> > [2]
> http://static.springsource.org/spring-hadoop/docs/current/reference/
> >
> > Hope this helps,
> > --
> > Costin
> >
> >
>
>
> --
> Costin
>
>
--
Costin
Re: HBase support in SHDP Was: Hadoop with Spring / Guice
Posted by Costin Leau <co...@gmail.com>.
On 12/30/2011 12:43 PM, Ted Yu wrote:
> Costin:
> Please include user@hbase.apache.org <ma...@hbase.apache.org> in
> your reply.
> My discussion would mostly be HBase-related.
# included hbase though I'm not subscribed to that list so I don't think
my messages will go through.
>
> I guess you're dealing with HBase 0.90.4 where connection sharing
> provided by HBASE-3777 is absent.
> HBase 0.90.5, just released, contains HBASE-4508 which backports
> HBASE-3777 to 0.90 codebase.
>
> Please kindly indicate whether the presence of HBASE-4508 would simplify
> hbase-configuration bean config.
> If it doesn't, I wonder what proxies would be doing without HBase
> connection.
>
I need to spend a bit more time to go through the issues you mentioned
to understand them better however at first glance looks like a valuable
improvement.
As for the proxy flag, this translate to the stopProxy argument on
HConnectionManager.deleteConnection() [1]
[1] http://j.mp/twrsH8
> Cheers
>
> On Fri, Dec 30, 2011 at 2:31 AM, Costin Leau <costin.leau@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi,
>
> Thanks for spotting the typo.
>
> We're currently running against 0.90.x. As for the lack of connection,
> we use HConnectionManager in the background to clean connections up as
> it already provides tracking and a cleanup mechanism (based on the
> HbaseConfiguration object).
>
> Cheers and keep the feedback coming,
>
> P.S. just want to point out there's a dedicated forum for SHDP, if the
> discussion becomes too Spring specific.
>
> [1] http://forum.springsource.org/forumdisplay.php?80-NoSQL
>
> On 12/30/2011 12:14 PM, Ted Yu wrote:
> > Hi, Costin:
> > I work on HBase.
> >
> > I went over
> >
> http://static.springsource.org/spring-hadoop/docs/current/reference/hbase.html
> > but didn't have time to download the source code.
> >
> > Is there a typo: 'does more then easily'
> > Should 'then' be 'than' ?
> >
> > For the following config:
> > <hdp:hbase-configuration stop-proxy="false" delete-connection="true"/>
> >
> > May I ask what would the proxies do without HBase connection ?
> >
> > What HBase version does SHDP work with ? I assume it is 0.90.x
> > 0.92 is coming out in January which has more features than 0.90.x
> >
> > If there is anything desirable from HBase, please let us know.
> > We will try to accommodate your requests.
> >
> > Cheers
> >
> > On Fri, Dec 30, 2011 at 1:53 AM, Costin Leau
> <costin.leau@gmail.com <ma...@gmail.com>
> > <mailto:costin.leau@gmail.com <ma...@gmail.com>>> wrote:
> >
> > Hi,
> >
> > My name is Costin Leau and I'm the lead of Spring Hadoop
> (SHDP) project.
> >
> > SHDP provides DI support allowing basic POJOs to be used as
> > mapper/reducers. This feature is currently developed on a
> dedicated
> > branch [1] and we plan to merge it in master in the near future.
> >
> > In addition to the pojo/DI support, you might be interested in
> the SHDP
> > integration with HBase/Hive/Pig - see [2]
> >
> > Everything is open-source, under ASL 2, available on github.
> We plan to
> > release the milestone in Jan/Feb 2012.
> >
> > Feedback and ideas are more then welcome!
> >
> > [1] https://github.com/SpringSource/spring-hadoop/tree/mr-pojo
> > [2]
> http://static.springsource.org/spring-hadoop/docs/current/reference/
> >
> > Hope this helps,
> > --
> > Costin
> >
> >
>
>
> --
> Costin
>
>
--
Costin
Re: HBase support in SHDP Was: Hadoop with Spring / Guice
Posted by Ted Yu <yu...@gmail.com>.
Costin:
Please include user@hbase.apache.org in your reply.
My discussion would mostly be HBase-related.
I guess you're dealing with HBase 0.90.4 where connection sharing provided
by HBASE-3777 is absent.
HBase 0.90.5, just released, contains HBASE-4508 which backports HBASE-3777
to 0.90 codebase.
Please kindly indicate whether the presence of HBASE-4508 would simplify
hbase-configuration bean config.
If it doesn't, I wonder what proxies would be doing without HBase
connection.
Cheers
On Fri, Dec 30, 2011 at 2:31 AM, Costin Leau <co...@gmail.com> wrote:
> Hi,
>
> Thanks for spotting the typo.
>
> We're currently running against 0.90.x. As for the lack of connection,
> we use HConnectionManager in the background to clean connections up as
> it already provides tracking and a cleanup mechanism (based on the
> HbaseConfiguration object).
>
> Cheers and keep the feedback coming,
>
> P.S. just want to point out there's a dedicated forum for SHDP, if the
> discussion becomes too Spring specific.
>
> [1] http://forum.springsource.org/forumdisplay.php?80-NoSQL
>
> On 12/30/2011 12:14 PM, Ted Yu wrote:
> > Hi, Costin:
> > I work on HBase.
> >
> > I went over
> >
> http://static.springsource.org/spring-hadoop/docs/current/reference/hbase.html
> > but didn't have time to download the source code.
> >
> > Is there a typo: 'does more then easily'
> > Should 'then' be 'than' ?
> >
> > For the following config:
> > <hdp:hbase-configuration stop-proxy="false" delete-connection="true"/>
> >
> > May I ask what would the proxies do without HBase connection ?
> >
> > What HBase version does SHDP work with ? I assume it is 0.90.x
> > 0.92 is coming out in January which has more features than 0.90.x
> >
> > If there is anything desirable from HBase, please let us know.
> > We will try to accommodate your requests.
> >
> > Cheers
> >
> > On Fri, Dec 30, 2011 at 1:53 AM, Costin Leau <costin.leau@gmail.com
> > <ma...@gmail.com>> wrote:
> >
> > Hi,
> >
> > My name is Costin Leau and I'm the lead of Spring Hadoop (SHDP)
> project.
> >
> > SHDP provides DI support allowing basic POJOs to be used as
> > mapper/reducers. This feature is currently developed on a dedicated
> > branch [1] and we plan to merge it in master in the near future.
> >
> > In addition to the pojo/DI support, you might be interested in the
> SHDP
> > integration with HBase/Hive/Pig - see [2]
> >
> > Everything is open-source, under ASL 2, available on github. We plan
> to
> > release the milestone in Jan/Feb 2012.
> >
> > Feedback and ideas are more then welcome!
> >
> > [1] https://github.com/SpringSource/spring-hadoop/tree/mr-pojo
> > [2]
> http://static.springsource.org/spring-hadoop/docs/current/reference/
> >
> > Hope this helps,
> > --
> > Costin
> >
> >
>
>
> --
> Costin
>
Re: HBase support in SHDP Was: Hadoop with Spring / Guice
Posted by Ted Yu <yu...@gmail.com>.
Costin:
Please include user@hbase.apache.org in your reply.
My discussion would mostly be HBase-related.
I guess you're dealing with HBase 0.90.4 where connection sharing provided
by HBASE-3777 is absent.
HBase 0.90.5, just released, contains HBASE-4508 which backports HBASE-3777
to 0.90 codebase.
Please kindly indicate whether the presence of HBASE-4508 would simplify
hbase-configuration bean config.
If it doesn't, I wonder what proxies would be doing without HBase
connection.
Cheers
On Fri, Dec 30, 2011 at 2:31 AM, Costin Leau <co...@gmail.com> wrote:
> Hi,
>
> Thanks for spotting the typo.
>
> We're currently running against 0.90.x. As for the lack of connection,
> we use HConnectionManager in the background to clean connections up as
> it already provides tracking and a cleanup mechanism (based on the
> HbaseConfiguration object).
>
> Cheers and keep the feedback coming,
>
> P.S. just want to point out there's a dedicated forum for SHDP, if the
> discussion becomes too Spring specific.
>
> [1] http://forum.springsource.org/forumdisplay.php?80-NoSQL
>
> On 12/30/2011 12:14 PM, Ted Yu wrote:
> > Hi, Costin:
> > I work on HBase.
> >
> > I went over
> >
> http://static.springsource.org/spring-hadoop/docs/current/reference/hbase.html
> > but didn't have time to download the source code.
> >
> > Is there a typo: 'does more then easily'
> > Should 'then' be 'than' ?
> >
> > For the following config:
> > <hdp:hbase-configuration stop-proxy="false" delete-connection="true"/>
> >
> > May I ask what would the proxies do without HBase connection ?
> >
> > What HBase version does SHDP work with ? I assume it is 0.90.x
> > 0.92 is coming out in January which has more features than 0.90.x
> >
> > If there is anything desirable from HBase, please let us know.
> > We will try to accommodate your requests.
> >
> > Cheers
> >
> > On Fri, Dec 30, 2011 at 1:53 AM, Costin Leau <costin.leau@gmail.com
> > <ma...@gmail.com>> wrote:
> >
> > Hi,
> >
> > My name is Costin Leau and I'm the lead of Spring Hadoop (SHDP)
> project.
> >
> > SHDP provides DI support allowing basic POJOs to be used as
> > mapper/reducers. This feature is currently developed on a dedicated
> > branch [1] and we plan to merge it in master in the near future.
> >
> > In addition to the pojo/DI support, you might be interested in the
> SHDP
> > integration with HBase/Hive/Pig - see [2]
> >
> > Everything is open-source, under ASL 2, available on github. We plan
> to
> > release the milestone in Jan/Feb 2012.
> >
> > Feedback and ideas are more then welcome!
> >
> > [1] https://github.com/SpringSource/spring-hadoop/tree/mr-pojo
> > [2]
> http://static.springsource.org/spring-hadoop/docs/current/reference/
> >
> > Hope this helps,
> > --
> > Costin
> >
> >
>
>
> --
> Costin
>
Re: HBase support in SHDP Was: Hadoop with Spring / Guice
Posted by Costin Leau <co...@gmail.com>.
Hi,
Thanks for spotting the typo.
We're currently running against 0.90.x. As for the lack of connection,
we use HConnectionManager in the background to clean connections up as
it already provides tracking and a cleanup mechanism (based on the
HbaseConfiguration object).
Cheers and keep the feedback coming,
P.S. just want to point out there's a dedicated forum for SHDP, if the
discussion becomes too Spring specific.
[1] http://forum.springsource.org/forumdisplay.php?80-NoSQL
On 12/30/2011 12:14 PM, Ted Yu wrote:
> Hi, Costin:
> I work on HBase.
>
> I went over
> http://static.springsource.org/spring-hadoop/docs/current/reference/hbase.html
> but didn't have time to download the source code.
>
> Is there a typo: 'does more then easily'
> Should 'then' be 'than' ?
>
> For the following config:
> <hdp:hbase-configuration stop-proxy="false" delete-connection="true"/>
>
> May I ask what would the proxies do without HBase connection ?
>
> What HBase version does SHDP work with ? I assume it is 0.90.x
> 0.92 is coming out in January which has more features than 0.90.x
>
> If there is anything desirable from HBase, please let us know.
> We will try to accommodate your requests.
>
> Cheers
>
> On Fri, Dec 30, 2011 at 1:53 AM, Costin Leau <costin.leau@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi,
>
> My name is Costin Leau and I'm the lead of Spring Hadoop (SHDP) project.
>
> SHDP provides DI support allowing basic POJOs to be used as
> mapper/reducers. This feature is currently developed on a dedicated
> branch [1] and we plan to merge it in master in the near future.
>
> In addition to the pojo/DI support, you might be interested in the SHDP
> integration with HBase/Hive/Pig - see [2]
>
> Everything is open-source, under ASL 2, available on github. We plan to
> release the milestone in Jan/Feb 2012.
>
> Feedback and ideas are more then welcome!
>
> [1] https://github.com/SpringSource/spring-hadoop/tree/mr-pojo
> [2] http://static.springsource.org/spring-hadoop/docs/current/reference/
>
> Hope this helps,
> --
> Costin
>
>
--
Costin