You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Panshul Whisper <ou...@gmail.com> on 2013/01/18 23:34:07 UTC
Spring for hadoop
Hello,
I was wondering if anyone is using spring for hadoop to execute map reduce
jobs or to perform hbase operations on a hadoop cluster using spring data
for hadoop.
Please suggest me a working example as I am unable to find any working
sample and spring data documentation is of no use for beginners.
Thanks
Regards,
Ouch Whisper
01010101010
Re: Spring for hadoop
Posted by Radim Kolar <hs...@filez.com>.
Dne 23.1.2013 22:55, Panshul Whisper napsal(a):
> Hello Radim,
>
> Your solution sounds interesting. Is it possible for me to try the
> solution before I buy it?
>
i do not ship demo version since it would be identical to production
version. I do onsite presentations with live demo and you will get all
code examples used. Its possible to code something simple on custom
demand during presentation, for example i got request to show how to use
hadoop seq file for storing data extracted from jms messages received by
spring integration.
Re: Spring for hadoop
Posted by Radim Kolar <hs...@filez.com>.
Dne 23.1.2013 22:55, Panshul Whisper napsal(a):
> Hello Radim,
>
> Your solution sounds interesting. Is it possible for me to try the
> solution before I buy it?
>
i do not ship demo version since it would be identical to production
version. I do onsite presentations with live demo and you will get all
code examples used. Its possible to code something simple on custom
demand during presentation, for example i got request to show how to use
hadoop seq file for storing data extracted from jms messages received by
spring integration.
Re: Spring for hadoop
Posted by Radim Kolar <hs...@filez.com>.
Dne 23.1.2013 22:55, Panshul Whisper napsal(a):
> Hello Radim,
>
> Your solution sounds interesting. Is it possible for me to try the
> solution before I buy it?
>
i do not ship demo version since it would be identical to production
version. I do onsite presentations with live demo and you will get all
code examples used. Its possible to code something simple on custom
demand during presentation, for example i got request to show how to use
hadoop seq file for storing data extracted from jms messages received by
spring integration.
Re: Spring for hadoop
Posted by Radim Kolar <hs...@filez.com>.
Dne 23.1.2013 22:55, Panshul Whisper napsal(a):
> Hello Radim,
>
> Your solution sounds interesting. Is it possible for me to try the
> solution before I buy it?
>
i do not ship demo version since it would be identical to production
version. I do onsite presentations with live demo and you will get all
code examples used. Its possible to code something simple on custom
demand during presentation, for example i got request to show how to use
hadoop seq file for storing data extracted from jms messages received by
spring integration.
Re: Spring for hadoop
Posted by Panshul Whisper <ou...@gmail.com>.
Hello Radim,
Your solution sounds interesting. Is it possible for me to try the solution
before I buy it?
Thnx ,
Regards
On Wed, Jan 23, 2013 at 1:07 AM, Radim Kolar <hs...@filez.com> wrote:
> i have solution integrating spring beans and spring batch directly into
> hadoop core. its far more advanced then spring data hadoop support with
> pojo patch.
>
> in my solution every component of mapreduce can be hadoop bean. You will
> get spring batch integrated directly into mapper, which means that you can
> run multiple steps in one mapper pass and because of async write done by
> spring batch you will get about 3x higher write performance. i have
> rewriten HDFS which has way faster writes. Spring batch component replaces
> standard hadoop job manager (that thing with web gui) and spring
> integration is used for advanced stuff like multiresource scheduling. You
> can write simple java bean for every new resource you want to add into
> system and another bean for logic for assigning jobs based on that
> particular resource.
>
> I submitted few patches to hadoop but they were not interesting enough to
> get into core. If you want to buy my hadoop with integrated spring, let me
> know.
>
--
Regards,
Ouch Whisper
010101010101
Re: Spring for hadoop
Posted by Panshul Whisper <ou...@gmail.com>.
Hello Radim,
Your solution sounds interesting. Is it possible for me to try the solution
before I buy it?
Thnx ,
Regards
On Wed, Jan 23, 2013 at 1:07 AM, Radim Kolar <hs...@filez.com> wrote:
> i have solution integrating spring beans and spring batch directly into
> hadoop core. its far more advanced then spring data hadoop support with
> pojo patch.
>
> in my solution every component of mapreduce can be hadoop bean. You will
> get spring batch integrated directly into mapper, which means that you can
> run multiple steps in one mapper pass and because of async write done by
> spring batch you will get about 3x higher write performance. i have
> rewriten HDFS which has way faster writes. Spring batch component replaces
> standard hadoop job manager (that thing with web gui) and spring
> integration is used for advanced stuff like multiresource scheduling. You
> can write simple java bean for every new resource you want to add into
> system and another bean for logic for assigning jobs based on that
> particular resource.
>
> I submitted few patches to hadoop but they were not interesting enough to
> get into core. If you want to buy my hadoop with integrated spring, let me
> know.
>
--
Regards,
Ouch Whisper
010101010101
Re: Spring for hadoop
Posted by Panshul Whisper <ou...@gmail.com>.
Hello Radim,
Your solution sounds interesting. Is it possible for me to try the solution
before I buy it?
Thnx ,
Regards
On Wed, Jan 23, 2013 at 1:07 AM, Radim Kolar <hs...@filez.com> wrote:
> i have solution integrating spring beans and spring batch directly into
> hadoop core. its far more advanced then spring data hadoop support with
> pojo patch.
>
> in my solution every component of mapreduce can be hadoop bean. You will
> get spring batch integrated directly into mapper, which means that you can
> run multiple steps in one mapper pass and because of async write done by
> spring batch you will get about 3x higher write performance. i have
> rewriten HDFS which has way faster writes. Spring batch component replaces
> standard hadoop job manager (that thing with web gui) and spring
> integration is used for advanced stuff like multiresource scheduling. You
> can write simple java bean for every new resource you want to add into
> system and another bean for logic for assigning jobs based on that
> particular resource.
>
> I submitted few patches to hadoop but they were not interesting enough to
> get into core. If you want to buy my hadoop with integrated spring, let me
> know.
>
--
Regards,
Ouch Whisper
010101010101
Re: Spring for hadoop
Posted by Panshul Whisper <ou...@gmail.com>.
Hello Radim,
Your solution sounds interesting. Is it possible for me to try the solution
before I buy it?
Thnx ,
Regards
On Wed, Jan 23, 2013 at 1:07 AM, Radim Kolar <hs...@filez.com> wrote:
> i have solution integrating spring beans and spring batch directly into
> hadoop core. its far more advanced then spring data hadoop support with
> pojo patch.
>
> in my solution every component of mapreduce can be hadoop bean. You will
> get spring batch integrated directly into mapper, which means that you can
> run multiple steps in one mapper pass and because of async write done by
> spring batch you will get about 3x higher write performance. i have
> rewriten HDFS which has way faster writes. Spring batch component replaces
> standard hadoop job manager (that thing with web gui) and spring
> integration is used for advanced stuff like multiresource scheduling. You
> can write simple java bean for every new resource you want to add into
> system and another bean for logic for assigning jobs based on that
> particular resource.
>
> I submitted few patches to hadoop but they were not interesting enough to
> get into core. If you want to buy my hadoop with integrated spring, let me
> know.
>
--
Regards,
Ouch Whisper
010101010101
Re: Spring for hadoop
Posted by Radim Kolar <hs...@filez.com>.
i have solution integrating spring beans and spring batch directly into
hadoop core. its far more advanced then spring data hadoop support with
pojo patch.
in my solution every component of mapreduce can be hadoop bean. You will
get spring batch integrated directly into mapper, which means that you
can run multiple steps in one mapper pass and because of async write
done by spring batch you will get about 3x higher write performance. i
have rewriten HDFS which has way faster writes. Spring batch component
replaces standard hadoop job manager (that thing with web gui) and
spring integration is used for advanced stuff like multiresource
scheduling. You can write simple java bean for every new resource you
want to add into system and another bean for logic for assigning jobs
based on that particular resource.
I submitted few patches to hadoop but they were not interesting enough
to get into core. If you want to buy my hadoop with integrated spring,
let me know.
Re: Spring for hadoop
Posted by Radim Kolar <hs...@filez.com>.
i have solution integrating spring beans and spring batch directly into
hadoop core. its far more advanced then spring data hadoop support with
pojo patch.
in my solution every component of mapreduce can be hadoop bean. You will
get spring batch integrated directly into mapper, which means that you
can run multiple steps in one mapper pass and because of async write
done by spring batch you will get about 3x higher write performance. i
have rewriten HDFS which has way faster writes. Spring batch component
replaces standard hadoop job manager (that thing with web gui) and
spring integration is used for advanced stuff like multiresource
scheduling. You can write simple java bean for every new resource you
want to add into system and another bean for logic for assigning jobs
based on that particular resource.
I submitted few patches to hadoop but they were not interesting enough
to get into core. If you want to buy my hadoop with integrated spring,
let me know.
Re: Spring for hadoop
Posted by Radim Kolar <hs...@filez.com>.
i have solution integrating spring beans and spring batch directly into
hadoop core. its far more advanced then spring data hadoop support with
pojo patch.
in my solution every component of mapreduce can be hadoop bean. You will
get spring batch integrated directly into mapper, which means that you
can run multiple steps in one mapper pass and because of async write
done by spring batch you will get about 3x higher write performance. i
have rewriten HDFS which has way faster writes. Spring batch component
replaces standard hadoop job manager (that thing with web gui) and
spring integration is used for advanced stuff like multiresource
scheduling. You can write simple java bean for every new resource you
want to add into system and another bean for logic for assigning jobs
based on that particular resource.
I submitted few patches to hadoop but they were not interesting enough
to get into core. If you want to buy my hadoop with integrated spring,
let me know.
Re: Spring for hadoop
Posted by Jilani Shaik <ji...@gmail.com>.
Hi,
Please find below URL where you will find the sample code for spring hadoop.
https://github.com/SpringSource/spring-hadoop-samples
Thank you,
Jilani
On Sat, Jan 19, 2013 at 11:43 AM, Jilani Shaik <ji...@gmail.com> wrote:
> Yes, We have used spring hadoop data for our hbase data reading and
> writing to HBase.
>
> We have used the below link for implementation in our project.
>
>
> http://static.springsource.org/spring-hadoop/docs/current/reference/html/hbase.html
>
> Thank you,
> Jilani
>
>
>
> On Sat, Jan 19, 2013 at 4:06 AM, Mohammad Tariq <do...@gmail.com>wrote:
>
>> You might find this link <http://www.springsource.org/spring-data/hadoop>
>> useful.
>>
>> Warm Regards,
>> Tariq
>> https://mtariq.jux.com/
>> cloudfront.blogspot.com
>>
>>
>> On Sat, Jan 19, 2013 at 4:04 AM, Panshul Whisper <ou...@gmail.com>wrote:
>>
>>> Hello,
>>>
>>> I was wondering if anyone is using spring for hadoop to execute map
>>> reduce jobs or to perform hbase operations on a hadoop cluster using spring
>>> data for hadoop.
>>> Please suggest me a working example as I am unable to find any working
>>> sample and spring data documentation is of no use for beginners.
>>>
>>> Thanks
>>>
>>> Regards,
>>> Ouch Whisper
>>> 01010101010
>>>
>>
>>
>
Re: Spring for hadoop
Posted by Jilani Shaik <ji...@gmail.com>.
Hi,
Please find below URL where you will find the sample code for spring hadoop.
https://github.com/SpringSource/spring-hadoop-samples
Thank you,
Jilani
On Sat, Jan 19, 2013 at 11:43 AM, Jilani Shaik <ji...@gmail.com> wrote:
> Yes, We have used spring hadoop data for our hbase data reading and
> writing to HBase.
>
> We have used the below link for implementation in our project.
>
>
> http://static.springsource.org/spring-hadoop/docs/current/reference/html/hbase.html
>
> Thank you,
> Jilani
>
>
>
> On Sat, Jan 19, 2013 at 4:06 AM, Mohammad Tariq <do...@gmail.com>wrote:
>
>> You might find this link <http://www.springsource.org/spring-data/hadoop>
>> useful.
>>
>> Warm Regards,
>> Tariq
>> https://mtariq.jux.com/
>> cloudfront.blogspot.com
>>
>>
>> On Sat, Jan 19, 2013 at 4:04 AM, Panshul Whisper <ou...@gmail.com>wrote:
>>
>>> Hello,
>>>
>>> I was wondering if anyone is using spring for hadoop to execute map
>>> reduce jobs or to perform hbase operations on a hadoop cluster using spring
>>> data for hadoop.
>>> Please suggest me a working example as I am unable to find any working
>>> sample and spring data documentation is of no use for beginners.
>>>
>>> Thanks
>>>
>>> Regards,
>>> Ouch Whisper
>>> 01010101010
>>>
>>
>>
>
Re: Spring for hadoop
Posted by Jilani Shaik <ji...@gmail.com>.
Hi,
Please find below URL where you will find the sample code for spring hadoop.
https://github.com/SpringSource/spring-hadoop-samples
Thank you,
Jilani
On Sat, Jan 19, 2013 at 11:43 AM, Jilani Shaik <ji...@gmail.com> wrote:
> Yes, We have used spring hadoop data for our hbase data reading and
> writing to HBase.
>
> We have used the below link for implementation in our project.
>
>
> http://static.springsource.org/spring-hadoop/docs/current/reference/html/hbase.html
>
> Thank you,
> Jilani
>
>
>
> On Sat, Jan 19, 2013 at 4:06 AM, Mohammad Tariq <do...@gmail.com>wrote:
>
>> You might find this link <http://www.springsource.org/spring-data/hadoop>
>> useful.
>>
>> Warm Regards,
>> Tariq
>> https://mtariq.jux.com/
>> cloudfront.blogspot.com
>>
>>
>> On Sat, Jan 19, 2013 at 4:04 AM, Panshul Whisper <ou...@gmail.com>wrote:
>>
>>> Hello,
>>>
>>> I was wondering if anyone is using spring for hadoop to execute map
>>> reduce jobs or to perform hbase operations on a hadoop cluster using spring
>>> data for hadoop.
>>> Please suggest me a working example as I am unable to find any working
>>> sample and spring data documentation is of no use for beginners.
>>>
>>> Thanks
>>>
>>> Regards,
>>> Ouch Whisper
>>> 01010101010
>>>
>>
>>
>
Re: Spring for hadoop
Posted by Jilani Shaik <ji...@gmail.com>.
Hi,
Please find below URL where you will find the sample code for spring hadoop.
https://github.com/SpringSource/spring-hadoop-samples
Thank you,
Jilani
On Sat, Jan 19, 2013 at 11:43 AM, Jilani Shaik <ji...@gmail.com> wrote:
> Yes, We have used spring hadoop data for our hbase data reading and
> writing to HBase.
>
> We have used the below link for implementation in our project.
>
>
> http://static.springsource.org/spring-hadoop/docs/current/reference/html/hbase.html
>
> Thank you,
> Jilani
>
>
>
> On Sat, Jan 19, 2013 at 4:06 AM, Mohammad Tariq <do...@gmail.com>wrote:
>
>> You might find this link <http://www.springsource.org/spring-data/hadoop>
>> useful.
>>
>> Warm Regards,
>> Tariq
>> https://mtariq.jux.com/
>> cloudfront.blogspot.com
>>
>>
>> On Sat, Jan 19, 2013 at 4:04 AM, Panshul Whisper <ou...@gmail.com>wrote:
>>
>>> Hello,
>>>
>>> I was wondering if anyone is using spring for hadoop to execute map
>>> reduce jobs or to perform hbase operations on a hadoop cluster using spring
>>> data for hadoop.
>>> Please suggest me a working example as I am unable to find any working
>>> sample and spring data documentation is of no use for beginners.
>>>
>>> Thanks
>>>
>>> Regards,
>>> Ouch Whisper
>>> 01010101010
>>>
>>
>>
>
Re: Spring for hadoop
Posted by Jilani Shaik <ji...@gmail.com>.
Yes, We have used spring hadoop data for our hbase data reading and writing
to HBase.
We have used the below link for implementation in our project.
http://static.springsource.org/spring-hadoop/docs/current/reference/html/hbase.html
Thank you,
Jilani
On Sat, Jan 19, 2013 at 4:06 AM, Mohammad Tariq <do...@gmail.com> wrote:
> You might find this link <http://www.springsource.org/spring-data/hadoop>
> useful.
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Sat, Jan 19, 2013 at 4:04 AM, Panshul Whisper <ou...@gmail.com>wrote:
>
>> Hello,
>>
>> I was wondering if anyone is using spring for hadoop to execute map
>> reduce jobs or to perform hbase operations on a hadoop cluster using spring
>> data for hadoop.
>> Please suggest me a working example as I am unable to find any working
>> sample and spring data documentation is of no use for beginners.
>>
>> Thanks
>>
>> Regards,
>> Ouch Whisper
>> 01010101010
>>
>
>
Re: Spring for hadoop
Posted by Jilani Shaik <ji...@gmail.com>.
Yes, We have used spring hadoop data for our hbase data reading and writing
to HBase.
We have used the below link for implementation in our project.
http://static.springsource.org/spring-hadoop/docs/current/reference/html/hbase.html
Thank you,
Jilani
On Sat, Jan 19, 2013 at 4:06 AM, Mohammad Tariq <do...@gmail.com> wrote:
> You might find this link <http://www.springsource.org/spring-data/hadoop>
> useful.
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Sat, Jan 19, 2013 at 4:04 AM, Panshul Whisper <ou...@gmail.com>wrote:
>
>> Hello,
>>
>> I was wondering if anyone is using spring for hadoop to execute map
>> reduce jobs or to perform hbase operations on a hadoop cluster using spring
>> data for hadoop.
>> Please suggest me a working example as I am unable to find any working
>> sample and spring data documentation is of no use for beginners.
>>
>> Thanks
>>
>> Regards,
>> Ouch Whisper
>> 01010101010
>>
>
>
Re: Spring for hadoop
Posted by Jilani Shaik <ji...@gmail.com>.
Yes, We have used spring hadoop data for our hbase data reading and writing
to HBase.
We have used the below link for implementation in our project.
http://static.springsource.org/spring-hadoop/docs/current/reference/html/hbase.html
Thank you,
Jilani
On Sat, Jan 19, 2013 at 4:06 AM, Mohammad Tariq <do...@gmail.com> wrote:
> You might find this link <http://www.springsource.org/spring-data/hadoop>
> useful.
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Sat, Jan 19, 2013 at 4:04 AM, Panshul Whisper <ou...@gmail.com>wrote:
>
>> Hello,
>>
>> I was wondering if anyone is using spring for hadoop to execute map
>> reduce jobs or to perform hbase operations on a hadoop cluster using spring
>> data for hadoop.
>> Please suggest me a working example as I am unable to find any working
>> sample and spring data documentation is of no use for beginners.
>>
>> Thanks
>>
>> Regards,
>> Ouch Whisper
>> 01010101010
>>
>
>
Re: Spring for hadoop
Posted by Jilani Shaik <ji...@gmail.com>.
Yes, We have used spring hadoop data for our hbase data reading and writing
to HBase.
We have used the below link for implementation in our project.
http://static.springsource.org/spring-hadoop/docs/current/reference/html/hbase.html
Thank you,
Jilani
On Sat, Jan 19, 2013 at 4:06 AM, Mohammad Tariq <do...@gmail.com> wrote:
> You might find this link <http://www.springsource.org/spring-data/hadoop>
> useful.
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Sat, Jan 19, 2013 at 4:04 AM, Panshul Whisper <ou...@gmail.com>wrote:
>
>> Hello,
>>
>> I was wondering if anyone is using spring for hadoop to execute map
>> reduce jobs or to perform hbase operations on a hadoop cluster using spring
>> data for hadoop.
>> Please suggest me a working example as I am unable to find any working
>> sample and spring data documentation is of no use for beginners.
>>
>> Thanks
>>
>> Regards,
>> Ouch Whisper
>> 01010101010
>>
>
>
Re: Spring for hadoop
Posted by Mohammad Tariq <do...@gmail.com>.
You might find this link <http://www.springsource.org/spring-data/hadoop>
useful.
Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Sat, Jan 19, 2013 at 4:04 AM, Panshul Whisper <ou...@gmail.com>wrote:
> Hello,
>
> I was wondering if anyone is using spring for hadoop to execute map reduce
> jobs or to perform hbase operations on a hadoop cluster using spring data
> for hadoop.
> Please suggest me a working example as I am unable to find any working
> sample and spring data documentation is of no use for beginners.
>
> Thanks
>
> Regards,
> Ouch Whisper
> 01010101010
>
Re: Spring for hadoop
Posted by Mohammad Tariq <do...@gmail.com>.
You might find this link <http://www.springsource.org/spring-data/hadoop>
useful.
Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Sat, Jan 19, 2013 at 4:04 AM, Panshul Whisper <ou...@gmail.com>wrote:
> Hello,
>
> I was wondering if anyone is using spring for hadoop to execute map reduce
> jobs or to perform hbase operations on a hadoop cluster using spring data
> for hadoop.
> Please suggest me a working example as I am unable to find any working
> sample and spring data documentation is of no use for beginners.
>
> Thanks
>
> Regards,
> Ouch Whisper
> 01010101010
>
Re: Spring for hadoop
Posted by Radim Kolar <hs...@filez.com>.
i have solution integrating spring beans and spring batch directly into
hadoop core. its far more advanced then spring data hadoop support with
pojo patch.
in my solution every component of mapreduce can be hadoop bean. You will
get spring batch integrated directly into mapper, which means that you
can run multiple steps in one mapper pass and because of async write
done by spring batch you will get about 3x higher write performance. i
have rewriten HDFS which has way faster writes. Spring batch component
replaces standard hadoop job manager (that thing with web gui) and
spring integration is used for advanced stuff like multiresource
scheduling. You can write simple java bean for every new resource you
want to add into system and another bean for logic for assigning jobs
based on that particular resource.
I submitted few patches to hadoop but they were not interesting enough
to get into core. If you want to buy my hadoop with integrated spring,
let me know.
Re: Spring for hadoop
Posted by Mohammad Tariq <do...@gmail.com>.
You might find this link <http://www.springsource.org/spring-data/hadoop>
useful.
Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Sat, Jan 19, 2013 at 4:04 AM, Panshul Whisper <ou...@gmail.com>wrote:
> Hello,
>
> I was wondering if anyone is using spring for hadoop to execute map reduce
> jobs or to perform hbase operations on a hadoop cluster using spring data
> for hadoop.
> Please suggest me a working example as I am unable to find any working
> sample and spring data documentation is of no use for beginners.
>
> Thanks
>
> Regards,
> Ouch Whisper
> 01010101010
>
Re: Spring for hadoop
Posted by Mohammad Tariq <do...@gmail.com>.
You might find this link <http://www.springsource.org/spring-data/hadoop>
useful.
Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Sat, Jan 19, 2013 at 4:04 AM, Panshul Whisper <ou...@gmail.com>wrote:
> Hello,
>
> I was wondering if anyone is using spring for hadoop to execute map reduce
> jobs or to perform hbase operations on a hadoop cluster using spring data
> for hadoop.
> Please suggest me a working example as I am unable to find any working
> sample and spring data documentation is of no use for beginners.
>
> Thanks
>
> Regards,
> Ouch Whisper
> 01010101010
>