You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by YC Huang <yc...@gmail.com> on 2011/11/18 03:24:52 UTC
MongoDB storage handler for HIVE
I just have a quick and dirty implementation of a MongoDB storage handler
for HIVE, the project is hosted on GitHub:
https://github.com/yc-huang/Hive-mongo.
Since Hive table do not support 'update' efficiently, we use MongoDB to
store those data, e.g. 'meta' data like user profile info, which need to be
updated sometimes.
Post here in case maybe someone else need the same thing, and also welcome
bug reports and suggestions, thanks.
YC
Re: MongoDB storage handler for HIVE
Posted by YC Huang <yc...@gmail.com>.
Thanks Alex. Currently it's not fully tested yet, so please drop me an
email if you met any problem...
On Friday, November 18, 2011, Alexander C.H. Lorenz wrote:
> Hi YC,
>
> great work, I did a small test and it works. I'll write a small article
> about next week.
>
> best,
> Alex
>
> On Fri, Nov 18, 2011 at 6:41 AM, YC Huang <ychuang.cn@gmail.com<javascript:_e({}, 'cvml', 'ychuang.cn@gmail.com');>
> > wrote:
>
>> We only use their java driver, but do not change any code, is that also
>> aware of GPL license?
>>
>> I'm glad if it could be in hive-contribs, thanks.
>>
>> YC
>>
>>
>> On Friday, November 18, 2011, Ashutosh Chauhan wrote:
>>
>>> Great work YC. Alas, mongodb is GPL licensed, otherwise we could have it
>>> in hive-contribs (ofcourse only if you have decided it to contribute).
>>>
>>> Ashutosh
>>> On Thu, Nov 17, 2011 at 18:47, Stephen Boesch <ja...@gmail.com> wrote:
>>>
>>>> Nice idea! I have worked a bit with Mongo and am leaning towards hive
>>>> . This could be a nice combo. will check it out (pun intended)
>>>>
>>>>
>>>> 2011/11/17 YC Huang <yc...@gmail.com>
>>>>
>>>>> I just have a quick and dirty implementation of a MongoDB storage
>>>>> handler for HIVE, the project is hosted on GitHub:
>>>>> https://github.com/yc-huang/Hive-mongo.
>>>>> Since Hive table do not support 'update' efficiently, we use MongoDB
>>>>> to store those data, e.g. 'meta' data like user profile info, which need to
>>>>> be updated sometimes.
>>>>>
>>>>> Post here in case maybe someone else need the same thing, and also
>>>>> welcome bug reports and suggestions, thanks.
>>>>>
>>>>> YC
>>>>>
>>>>>
>>>>>
>>>>
>>>
>
>
> --
> Alexander Lorenz
> http://mapredit.blogspot.com
>
> *P **Think of the environment: please don't print this email unless you
> really need to.*
>
>
>
Re: MongoDB storage handler for HIVE
Posted by "Alexander C.H. Lorenz" <wg...@googlemail.com>.
Hi YC,
great work, I did a small test and it works. I'll write a small article
about next week.
best,
Alex
On Fri, Nov 18, 2011 at 6:41 AM, YC Huang <yc...@gmail.com> wrote:
> We only use their java driver, but do not change any code, is that also
> aware of GPL license?
>
> I'm glad if it could be in hive-contribs, thanks.
>
> YC
>
>
> On Friday, November 18, 2011, Ashutosh Chauhan wrote:
>
>> Great work YC. Alas, mongodb is GPL licensed, otherwise we could have it
>> in hive-contribs (ofcourse only if you have decided it to contribute).
>>
>> Ashutosh
>> On Thu, Nov 17, 2011 at 18:47, Stephen Boesch <ja...@gmail.com> wrote:
>>
>>> Nice idea! I have worked a bit with Mongo and am leaning towards hive .
>>> This could be a nice combo. will check it out (pun intended)
>>>
>>>
>>> 2011/11/17 YC Huang <yc...@gmail.com>
>>>
>>>> I just have a quick and dirty implementation of a MongoDB storage
>>>> handler for HIVE, the project is hosted on GitHub:
>>>> https://github.com/yc-huang/Hive-mongo.
>>>> Since Hive table do not support 'update' efficiently, we use MongoDB to
>>>> store those data, e.g. 'meta' data like user profile info, which need to be
>>>> updated sometimes.
>>>>
>>>> Post here in case maybe someone else need the same thing, and also
>>>> welcome bug reports and suggestions, thanks.
>>>>
>>>> YC
>>>>
>>>>
>>>>
>>>
>>
--
Alexander Lorenz
http://mapredit.blogspot.com
*P **Think of the environment: please don't print this email unless you
really need to.*
Re: MongoDB storage handler for HIVE
Posted by YC Huang <yc...@gmail.com>.
We only use their java driver, but do not change any code, is that also
aware of GPL license?
I'm glad if it could be in hive-contribs, thanks.
YC
On Friday, November 18, 2011, Ashutosh Chauhan wrote:
> Great work YC. Alas, mongodb is GPL licensed, otherwise we could have it
> in hive-contribs (ofcourse only if you have decided it to contribute).
>
> Ashutosh
> On Thu, Nov 17, 2011 at 18:47, Stephen Boesch <javadba@gmail.com<javascript:_e({}, 'cvml', 'javadba@gmail.com');>
> > wrote:
>
>> Nice idea! I have worked a bit with Mongo and am leaning towards hive .
>> This could be a nice combo. will check it out (pun intended)
>>
>>
>> 2011/11/17 YC Huang <ychuang.cn@gmail.com <javascript:_e({}, 'cvml',
>> 'ychuang.cn@gmail.com');>>
>>
>>> I just have a quick and dirty implementation of a MongoDB storage
>>> handler for HIVE, the project is hosted on GitHub:
>>> https://github.com/yc-huang/Hive-mongo.
>>> Since Hive table do not support 'update' efficiently, we use MongoDB to
>>> store those data, e.g. 'meta' data like user profile info, which need to be
>>> updated sometimes.
>>>
>>> Post here in case maybe someone else need the same thing, and also
>>> welcome bug reports and suggestions, thanks.
>>>
>>> YC
>>>
>>>
>>>
>>
>
Re: MongoDB storage handler for HIVE
Posted by Ashutosh Chauhan <ha...@apache.org>.
Great work YC. Alas, mongodb is GPL licensed, otherwise we could have it in
hive-contribs (ofcourse only if you have decided it to contribute).
Ashutosh
On Thu, Nov 17, 2011 at 18:47, Stephen Boesch <ja...@gmail.com> wrote:
> Nice idea! I have worked a bit with Mongo and am leaning towards hive .
> This could be a nice combo. will check it out (pun intended)
>
>
> 2011/11/17 YC Huang <yc...@gmail.com>
>
>> I just have a quick and dirty implementation of a MongoDB storage handler
>> for HIVE, the project is hosted on GitHub:
>> https://github.com/yc-huang/Hive-mongo.
>> Since Hive table do not support 'update' efficiently, we use MongoDB to
>> store those data, e.g. 'meta' data like user profile info, which need to be
>> updated sometimes.
>>
>> Post here in case maybe someone else need the same thing, and also
>> welcome bug reports and suggestions, thanks.
>>
>> YC
>>
>>
>>
>
Re: MongoDB storage handler for HIVE
Posted by Stephen Boesch <ja...@gmail.com>.
Nice idea! I have worked a bit with Mongo and am leaning towards hive .
This could be a nice combo. will check it out (pun intended)
2011/11/17 YC Huang <yc...@gmail.com>
> I just have a quick and dirty implementation of a MongoDB storage handler
> for HIVE, the project is hosted on GitHub:
> https://github.com/yc-huang/Hive-mongo.
> Since Hive table do not support 'update' efficiently, we use MongoDB to
> store those data, e.g. 'meta' data like user profile info, which need to be
> updated sometimes.
>
> Post here in case maybe someone else need the same thing, and also welcome
> bug reports and suggestions, thanks.
>
> YC
>
>
>