You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Billy <sa...@pearsonwholesale.com> on 2007/11/10 21:53:23 UTC
map/reduce with hbase
I am using map/reduce with hadoop-0.15.0-streaming.jar to process the data
with php scripts. I have coded to process the data the blow is an example of
word counts from the input.
bin/hadoop jar contrib/hadoop-0.15.0-streaming.jar -mapper
/var/www/search/hadoop/wc-mapper.php -reducer
/var/www/search/hadoop/wc-reducer.php -input input/* -output output
What I am looking to do is get and store the input and output from/in hbase.
Is that possible with hadoop-0.15.0-streaming.jar?
If so can I get a command line example?
thanks
Billy
Re: map/reduce with hbase
Posted by Billy <sa...@pearsonwholesale.com>.
Voted for when I ran across thanks for the work.
Billy
"edward yoon" <we...@udanax.org> wrote in
message news:BAY140-W2940DB80C74DEBF5420C0DC5830@phx.gbl...
http://shell.hadoop.co.kr/PHPClient.php
You can see the example by clicking the link above about the latest patch
(v02).
And, It is just first step work for proof of RDBMS ability
But, I'm not sure if it will be commited to svn source tree.
(I'm not a source committer.)
So, Click the "Vote it" button please. :)
Thanks,
Edward J. Yoon
------------------------------
B. Regards,
Edward yoon @ NHN, corp.
Home : http://www.udanax.org
> To: hadoop-user@lucene.apache.org
> From: sales@pearsonwholesale.com
> Subject: Re: map/reduce with hbase
> Date: Thu, 15 Nov 2007 22:58:15 -0600
>
> I was looking for a way to replace mysql with hbase so I could serve data
> on
> websites. HADOOP-2171 solves that problem seams. I have not tested it much
> but latest patch (v02) seams to be working correctly. I thank I can get
> the
> hadoop-Streaming to work for what I need but it seams if I want to take
> full
> advantage of hadoop map/reduce I am going to have to learn java :)
>
> Thanks for the help and patches.
> Keep up the good work and we will have a production version released
> before
> you know it.
>
> Billy
>
> "stack" wrote in message
> news:473A43E3.6030706@duboce.net...
>>I tried using streaming to dump into a hbase table. As things are
>>currently written, it unfortunately won't work. Streaming would seem to
>>presume keys and values of type Text whereas the TableOutputFormat takes a
>>key of type Text but the value is expected to be MapWritable (where the
>>keys are column names). Even if you could do types other than Text in
>>Streaming, an MW type is awkward for php/python, etc., to conpose.
>>
>> Regards your question as to how php might access hbase, at the moment
>> your
>> options are few:
>>
>> + There is the Edward Yoon patch that you've already tripped over,
>> hadoop-2171. It puts up an IPC server that fields HQL strings. The
>> server does the HQL parse and forwards the interpreted request to the
>> hbase cluster. Included is a first cut at php code that is capable of
>> making the basic method call against the remote java IPC server.
>> + If traffic is light and your requests are read-only, there is the HQL
>> page in the master's webui.
>>
>> If hbase had a REST interface, hadoop-2068, would that work for you?
>>
>> St.Ack
>>
>>
>> Billy wrote:
>>> Can you show me an example on how that would be down with the command
>>> line?
>>>
>>>
>>> "Michael Stack" wrote in
>>> message news:4736244E.1050804@duboce.net...
>>>
>>>> Billy wrote:
>>>>
>>>>> ..
>>>>>
>>>>> What I am looking to do is get and store the input and output from/in
>>>>> hbase.
>>>>>
>>>>>
>>>> I haven't tried it but it looks like you can specify input and output
>>>> classes for streaming with -inputformat and -outputformat options.
>>>>
>>>> Try setting these to TableInputFormat [1] and TableOutputFormat [2]
>>>> respectively.
>>>>
>>>> Usual caveats apply: These hbase classes need to be either bundled into
>>>> your job jar -- awkward in this case since you are using the streaming
>>>> job jar -- or they need to be on the cluster CLASSPATH (Add the
>>>> hadoop*hbase.jar to lib directory across the cluster is probably
>>>> easiest
>>>> thing to do).
>>>>
>>>> St.Ack
>>>>
>>>> 1.
>>>> http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/javadoc/org/apache/hadoop/hbase/mapred/TableInputFormat.html
>>>> 2.
>>>> http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/javadoc/org/apache/hadoop/hbase/mapred/TableOutputFormat.html
>>>>
>>>>
>>>
>>>
>>>
>>>
>>
>>
>
>
>
_________________________________________________________________
Climb to the top of the charts! Play Star Shuffle: the word scramble
challenge with star power.
http://club.live.com/star_shuffle.aspx?icid=starshuffle_wlmailtextlink_oct
RE: map/reduce with hbase
Posted by edward yoon <we...@udanax.org>.
http://shell.hadoop.co.kr/PHPClient.php
You can see the example by clicking the link above about the latest patch (v02).
And, It is just first step work for proof of RDBMS ability
But, I'm not sure if it will be commited to svn source tree.
(I'm not a source committer.)
So, Click the "Vote it" button please. :)
Thanks,
Edward J. Yoon
------------------------------
B. Regards,
Edward yoon @ NHN, corp.
Home : http://www.udanax.org
> To: hadoop-user@lucene.apache.org
> From: sales@pearsonwholesale.com
> Subject: Re: map/reduce with hbase
> Date: Thu, 15 Nov 2007 22:58:15 -0600
>
> I was looking for a way to replace mysql with hbase so I could serve data on
> websites. HADOOP-2171 solves that problem seams. I have not tested it much
> but latest patch (v02) seams to be working correctly. I thank I can get the
> hadoop-Streaming to work for what I need but it seams if I want to take full
> advantage of hadoop map/reduce I am going to have to learn java :)
>
> Thanks for the help and patches.
> Keep up the good work and we will have a production version released before
> you know it.
>
> Billy
>
> "stack" wrote in message
> news:473A43E3.6030706@duboce.net...
>>I tried using streaming to dump into a hbase table. As things are
>>currently written, it unfortunately won't work. Streaming would seem to
>>presume keys and values of type Text whereas the TableOutputFormat takes a
>>key of type Text but the value is expected to be MapWritable (where the
>>keys are column names). Even if you could do types other than Text in
>>Streaming, an MW type is awkward for php/python, etc., to conpose.
>>
>> Regards your question as to how php might access hbase, at the moment your
>> options are few:
>>
>> + There is the Edward Yoon patch that you've already tripped over,
>> hadoop-2171. It puts up an IPC server that fields HQL strings. The
>> server does the HQL parse and forwards the interpreted request to the
>> hbase cluster. Included is a first cut at php code that is capable of
>> making the basic method call against the remote java IPC server.
>> + If traffic is light and your requests are read-only, there is the HQL
>> page in the master's webui.
>>
>> If hbase had a REST interface, hadoop-2068, would that work for you?
>>
>> St.Ack
>>
>>
>> Billy wrote:
>>> Can you show me an example on how that would be down with the command
>>> line?
>>>
>>>
>>> "Michael Stack" wrote in
>>> message news:4736244E.1050804@duboce.net...
>>>
>>>> Billy wrote:
>>>>
>>>>> ..
>>>>>
>>>>> What I am looking to do is get and store the input and output from/in
>>>>> hbase.
>>>>>
>>>>>
>>>> I haven't tried it but it looks like you can specify input and output
>>>> classes for streaming with -inputformat and -outputformat options.
>>>>
>>>> Try setting these to TableInputFormat [1] and TableOutputFormat [2]
>>>> respectively.
>>>>
>>>> Usual caveats apply: These hbase classes need to be either bundled into
>>>> your job jar -- awkward in this case since you are using the streaming
>>>> job jar -- or they need to be on the cluster CLASSPATH (Add the
>>>> hadoop*hbase.jar to lib directory across the cluster is probably easiest
>>>> thing to do).
>>>>
>>>> St.Ack
>>>>
>>>> 1.
>>>> http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/javadoc/org/apache/hadoop/hbase/mapred/TableInputFormat.html
>>>> 2.
>>>> http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/javadoc/org/apache/hadoop/hbase/mapred/TableOutputFormat.html
>>>>
>>>>
>>>
>>>
>>>
>>>
>>
>>
>
>
>
_________________________________________________________________
Climb to the top of the charts! Play Star Shuffle: the word scramble challenge with star power.
http://club.live.com/star_shuffle.aspx?icid=starshuffle_wlmailtextlink_oct
Re: map/reduce with hbase
Posted by Billy <sa...@pearsonwholesale.com>.
I was looking for a way to replace mysql with hbase so I could serve data on
websites. HADOOP-2171 solves that problem seams. I have not tested it much
but latest patch (v02) seams to be working correctly. I thank I can get the
hadoop-Streaming to work for what I need but it seams if I want to take full
advantage of hadoop map/reduce I am going to have to learn java :)
Thanks for the help and patches.
Keep up the good work and we will have a production version released before
you know it.
Billy
"stack" <st...@duboce.net> wrote in message
news:473A43E3.6030706@duboce.net...
>I tried using streaming to dump into a hbase table. As things are
>currently written, it unfortunately won't work. Streaming would seem to
>presume keys and values of type Text whereas the TableOutputFormat takes a
>key of type Text but the value is expected to be MapWritable (where the
>keys are column names). Even if you could do types other than Text in
>Streaming, an MW type is awkward for php/python, etc., to conpose.
>
> Regards your question as to how php might access hbase, at the moment your
> options are few:
>
> + There is the Edward Yoon patch that you've already tripped over,
> hadoop-2171. It puts up an IPC server that fields HQL strings. The
> server does the HQL parse and forwards the interpreted request to the
> hbase cluster. Included is a first cut at php code that is capable of
> making the basic method call against the remote java IPC server.
> + If traffic is light and your requests are read-only, there is the HQL
> page in the master's webui.
>
> If hbase had a REST interface, hadoop-2068, would that work for you?
>
> St.Ack
>
>
> Billy wrote:
>> Can you show me an example on how that would be down with the command
>> line?
>>
>>
>> "Michael Stack" <st...@duboce.net> wrote in
>> message news:4736244E.1050804@duboce.net...
>>
>>> Billy wrote:
>>>
>>>> ..
>>>>
>>>> What I am looking to do is get and store the input and output from/in
>>>> hbase.
>>>>
>>>>
>>> I haven't tried it but it looks like you can specify input and output
>>> classes for streaming with -inputformat and -outputformat options.
>>>
>>> Try setting these to TableInputFormat [1] and TableOutputFormat [2]
>>> respectively.
>>>
>>> Usual caveats apply: These hbase classes need to be either bundled into
>>> your job jar -- awkward in this case since you are using the streaming
>>> job jar -- or they need to be on the cluster CLASSPATH (Add the
>>> hadoop*hbase.jar to lib directory across the cluster is probably easiest
>>> thing to do).
>>>
>>> St.Ack
>>>
>>> 1.
>>> http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/javadoc/org/apache/hadoop/hbase/mapred/TableInputFormat.html
>>> 2.
>>> http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/javadoc/org/apache/hadoop/hbase/mapred/TableOutputFormat.html
>>>
>>>
>>
>>
>>
>>
>
>
Re: map/reduce with hbase
Posted by stack <st...@duboce.net>.
I tried using streaming to dump into a hbase table. As things are
currently written, it unfortunately won't work. Streaming would seem to
presume keys and values of type Text whereas the TableOutputFormat takes
a key of type Text but the value is expected to be MapWritable (where
the keys are column names). Even if you could do types other than Text
in Streaming, an MW type is awkward for php/python, etc., to conpose.
Regards your question as to how php might access hbase, at the moment
your options are few:
+ There is the Edward Yoon patch that you've already tripped over,
hadoop-2171. It puts up an IPC server that fields HQL strings. The
server does the HQL parse and forwards the interpreted request to the
hbase cluster. Included is a first cut at php code that is capable of
making the basic method call against the remote java IPC server.
+ If traffic is light and your requests are read-only, there is the HQL
page in the master's webui.
If hbase had a REST interface, hadoop-2068, would that work for you?
St.Ack
Billy wrote:
> Can you show me an example on how that would be down with the command line?
>
>
> "Michael Stack" <st...@duboce.net> wrote in
> message news:4736244E.1050804@duboce.net...
>
>> Billy wrote:
>>
>>> ..
>>>
>>> What I am looking to do is get and store the input and output from/in
>>> hbase.
>>>
>>>
>> I haven't tried it but it looks like you can specify input and output
>> classes for streaming with -inputformat and -outputformat options.
>>
>> Try setting these to TableInputFormat [1] and TableOutputFormat [2]
>> respectively.
>>
>> Usual caveats apply: These hbase classes need to be either bundled into
>> your job jar -- awkward in this case since you are using the streaming job
>> jar -- or they need to be on the cluster CLASSPATH (Add the
>> hadoop*hbase.jar to lib directory across the cluster is probably easiest
>> thing to do).
>>
>> St.Ack
>>
>> 1.
>> http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/javadoc/org/apache/hadoop/hbase/mapred/TableInputFormat.html
>> 2.
>> http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/javadoc/org/apache/hadoop/hbase/mapred/TableOutputFormat.html
>>
>>
>
>
>
>
Re: map/reduce with hbase
Posted by Billy <sa...@pearsonwholesale.com>.
Can you show me an example on how that would be down with the command line?
"Michael Stack" <st...@duboce.net> wrote in
message news:4736244E.1050804@duboce.net...
> Billy wrote:
>> ..
>>
>> What I am looking to do is get and store the input and output from/in
>> hbase.
>>
> I haven't tried it but it looks like you can specify input and output
> classes for streaming with -inputformat and -outputformat options.
>
> Try setting these to TableInputFormat [1] and TableOutputFormat [2]
> respectively.
>
> Usual caveats apply: These hbase classes need to be either bundled into
> your job jar -- awkward in this case since you are using the streaming job
> jar -- or they need to be on the cluster CLASSPATH (Add the
> hadoop*hbase.jar to lib directory across the cluster is probably easiest
> thing to do).
>
> St.Ack
>
> 1.
> http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/javadoc/org/apache/hadoop/hbase/mapred/TableInputFormat.html
> 2.
> http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/javadoc/org/apache/hadoop/hbase/mapred/TableOutputFormat.html
>
Re: map/reduce with hbase
Posted by Michael Stack <st...@duboce.net>.
Billy wrote:
> ..
>
> What I am looking to do is get and store the input and output from/in hbase.
>
I haven't tried it but it looks like you can specify input and output
classes for streaming with -inputformat and -outputformat options.
Try setting these to TableInputFormat [1] and TableOutputFormat [2]
respectively.
Usual caveats apply: These hbase classes need to be either bundled into
your job jar -- awkward in this case since you are using the streaming
job jar -- or they need to be on the cluster CLASSPATH (Add the
hadoop*hbase.jar to lib directory across the cluster is probably easiest
thing to do).
St.Ack
1.
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/javadoc/org/apache/hadoop/hbase/mapred/TableInputFormat.html
2.
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/javadoc/org/apache/hadoop/hbase/mapred/TableOutputFormat.html
RE: map/reduce with hbase
Posted by edward yoon <we...@udanax.org>.
Sounds good.
Thanks, Billy.
------------------------------
B. Regards,
Edward yoon @ NHN, corp.
Home : http://www.udanax.org
> To: hadoop-user@lucene.apache.org
> From: sales@pearsonwholesale.com
> Subject: Re: map/reduce with hbase
> Date: Thu, 15 Nov 2007 23:26:54 -0600
>
> Thanks for the patches they seam to be working atm.
>
> I will use it for a while and see what suggestions or bugs I can find and
> let you know.
>
> right now the only thing I can think of is
> return 1 for successful insert/delete and 0 for failed insert/delete
>
> Thanks again.
>
> Billy
>
>
> "edward yoon" wrote in
> message news:BAY140-W38B38EF3B7095A628530A3C5800@phx.gbl...
>
> See my socket communication example between a php client and a java server.
>
> https://issues.apache.org/jira/secure/attachment/12369419/patch_v01.txt
>
> But, i don't know what you want exactly.
> Thanks.
> ------------------------------
>
> B. Regards,
>
> Edward yoon @ NHN, corp.
> Home : http://www.udanax.org
>
>
>> To: hadoop-user@lucene.apache.org
>> From: sales@pearsonwholesale.com
>> Subject: Re: map/reduce with hbase
>> Date: Sat, 10 Nov 2007 23:56:29 -0600
>>
>> Basically what I am trying to do is access hbase from php sense I do not
>> know java and have not found it fun to learn
>>
>> I was looking around and found this
>> https://issues.apache.org/jira/browse/HADOOP-2171
>>
>> but am unsure if it is what I thank it is looks like a way to access hbase
>> from a socket server I have coded socket server/client in the past in php
>> so
>> the question is how would I use this to get access from say php.
>>
>> would I add this and rebuild the hbase file somehow?
>>
>>
>> Remember I am new to java so have vary little knowing on how to do stuff.
>>
>> Billy
>>
>>
>>
>> "Billy" wrote in
>> message news:fh55ok$r4i$1@ger.gmane.org...
>>>I am using map/reduce with hadoop-0.15.0-streaming.jar to process the data
>>>with php scripts. I have coded to process the data the blow is an example
>>>of word counts from the input.
>>>
>>> bin/hadoop jar contrib/hadoop-0.15.0-streaming.jar -mapper
>>> /var/www/search/hadoop/wc-mapper.php -reducer
>>> /var/www/search/hadoop/wc-reducer.php -input input/* -output output
>>>
>>> What I am looking to do is get and store the input and output from/in
>>> hbase.
>>>
>>> Is that possible with hadoop-0.15.0-streaming.jar?
>>> If so can I get a command line example?
>>>
>>> thanks
>>> Billy
>>>
>>>
>>>
>>>
>>
>>
>>
>
> _________________________________________________________________
> Help yourself to FREE treats served up daily at the Messenger Café. Stop by
> today.
> http://www.cafemessenger.com/info/info_sweetstuff2.html?ocid=TXT_TAGLM_OctWLtagline
>
>
>
_________________________________________________________________
Climb to the top of the charts! Play Star Shuffle: the word scramble challenge with star power.
http://club.live.com/star_shuffle.aspx?icid=starshuffle_wlmailtextlink_oct
RE: map/reduce with hbase
Posted by edward yoon <we...@udanax.org>.
Sounds good.
Thanks, Billy.
------------------------------
B. Regards,
Edward yoon @ NHN, corp.
Home : http://www.udanax.org
> To: hadoop-user@lucene.apache.org
> From: sales@pearsonwholesale.com
> Subject: Re: map/reduce with hbase
> Date: Thu, 15 Nov 2007 23:26:54 -0600
>
> Thanks for the patches they seam to be working atm.
>
> I will use it for a while and see what suggestions or bugs I can find and
> let you know.
>
> right now the only thing I can think of is
> return 1 for successful insert/delete and 0 for failed insert/delete
>
> Thanks again.
>
> Billy
>
>
> "edward yoon" wrote in
> message news:BAY140-W38B38EF3B7095A628530A3C5800@phx.gbl...
>
> See my socket communication example between a php client and a java server.
>
> https://issues.apache.org/jira/secure/attachment/12369419/patch_v01.txt
>
> But, i don't know what you want exactly.
> Thanks.
> ------------------------------
>
> B. Regards,
>
> Edward yoon @ NHN, corp.
> Home : http://www.udanax.org
>
>
>> To: hadoop-user@lucene.apache.org
>> From: sales@pearsonwholesale.com
>> Subject: Re: map/reduce with hbase
>> Date: Sat, 10 Nov 2007 23:56:29 -0600
>>
>> Basically what I am trying to do is access hbase from php sense I do not
>> know java and have not found it fun to learn
>>
>> I was looking around and found this
>> https://issues.apache.org/jira/browse/HADOOP-2171
>>
>> but am unsure if it is what I thank it is looks like a way to access hbase
>> from a socket server I have coded socket server/client in the past in php
>> so
>> the question is how would I use this to get access from say php.
>>
>> would I add this and rebuild the hbase file somehow?
>>
>>
>> Remember I am new to java so have vary little knowing on how to do stuff.
>>
>> Billy
>>
>>
>>
>> "Billy" wrote in
>> message news:fh55ok$r4i$1@ger.gmane.org...
>>>I am using map/reduce with hadoop-0.15.0-streaming.jar to process the data
>>>with php scripts. I have coded to process the data the blow is an example
>>>of word counts from the input.
>>>
>>> bin/hadoop jar contrib/hadoop-0.15.0-streaming.jar -mapper
>>> /var/www/search/hadoop/wc-mapper.php -reducer
>>> /var/www/search/hadoop/wc-reducer.php -input input/* -output output
>>>
>>> What I am looking to do is get and store the input and output from/in
>>> hbase.
>>>
>>> Is that possible with hadoop-0.15.0-streaming.jar?
>>> If so can I get a command line example?
>>>
>>> thanks
>>> Billy
>>>
>>>
>>>
>>>
>>
>>
>>
>
> _________________________________________________________________
> Help yourself to FREE treats served up daily at the Messenger Café. Stop by
> today.
> http://www.cafemessenger.com/info/info_sweetstuff2.html?ocid=TXT_TAGLM_OctWLtagline
>
>
>
_________________________________________________________________
Help yourself to FREE treats served up daily at the Messenger Café. Stop by today.
http://www.cafemessenger.com/info/info_sweetstuff2.html?ocid=TXT_TAGLM_OctWLtagline
RE: map/reduce with hbase
Posted by edward yoon <we...@udanax.org>.
Sounds good.
Thanks, Billy.
------------------------------
B. Regards,
Edward yoon @ NHN, corp.
Home : http://www.udanax.org
> To: hadoop-user@lucene.apache.org
> From: sales@pearsonwholesale.com
> Subject: Re: map/reduce with hbase
> Date: Thu, 15 Nov 2007 23:26:54 -0600
>
> Thanks for the patches they seam to be working atm.
>
> I will use it for a while and see what suggestions or bugs I can find and
> let you know.
>
> right now the only thing I can think of is
> return 1 for successful insert/delete and 0 for failed insert/delete
>
> Thanks again.
>
> Billy
>
>
> "edward yoon" wrote in
> message news:BAY140-W38B38EF3B7095A628530A3C5800@phx.gbl...
>
> See my socket communication example between a php client and a java server.
>
> https://issues.apache.org/jira/secure/attachment/12369419/patch_v01.txt
>
> But, i don't know what you want exactly.
> Thanks.
> ------------------------------
>
> B. Regards,
>
> Edward yoon @ NHN, corp.
> Home : http://www.udanax.org
>
>
>> To: hadoop-user@lucene.apache.org
>> From: sales@pearsonwholesale.com
>> Subject: Re: map/reduce with hbase
>> Date: Sat, 10 Nov 2007 23:56:29 -0600
>>
>> Basically what I am trying to do is access hbase from php sense I do not
>> know java and have not found it fun to learn
>>
>> I was looking around and found this
>> https://issues.apache.org/jira/browse/HADOOP-2171
>>
>> but am unsure if it is what I thank it is looks like a way to access hbase
>> from a socket server I have coded socket server/client in the past in php
>> so
>> the question is how would I use this to get access from say php.
>>
>> would I add this and rebuild the hbase file somehow?
>>
>>
>> Remember I am new to java so have vary little knowing on how to do stuff.
>>
>> Billy
>>
>>
>>
>> "Billy" wrote in
>> message news:fh55ok$r4i$1@ger.gmane.org...
>>>I am using map/reduce with hadoop-0.15.0-streaming.jar to process the data
>>>with php scripts. I have coded to process the data the blow is an example
>>>of word counts from the input.
>>>
>>> bin/hadoop jar contrib/hadoop-0.15.0-streaming.jar -mapper
>>> /var/www/search/hadoop/wc-mapper.php -reducer
>>> /var/www/search/hadoop/wc-reducer.php -input input/* -output output
>>>
>>> What I am looking to do is get and store the input and output from/in
>>> hbase.
>>>
>>> Is that possible with hadoop-0.15.0-streaming.jar?
>>> If so can I get a command line example?
>>>
>>> thanks
>>> Billy
>>>
>>>
>>>
>>>
>>
>>
>>
>
> _________________________________________________________________
> Help yourself to FREE treats served up daily at the Messenger Café. Stop by
> today.
> http://www.cafemessenger.com/info/info_sweetstuff2.html?ocid=TXT_TAGLM_OctWLtagline
>
>
>
_________________________________________________________________
Boo! Scare away worms, viruses and so much more! Try Windows Live OneCare!
http://onecare.live.com/standard/en-us/purchase/trial.aspx?s_cid=wl_hotmailnews
Re: map/reduce with hbase
Posted by Billy <sa...@pearsonwholesale.com>.
Thanks for the patches they seam to be working atm.
I will use it for a while and see what suggestions or bugs I can find and
let you know.
right now the only thing I can think of is
return 1 for successful insert/delete and 0 for failed insert/delete
Thanks again.
Billy
"edward yoon" <we...@udanax.org> wrote in
message news:BAY140-W38B38EF3B7095A628530A3C5800@phx.gbl...
See my socket communication example between a php client and a java server.
https://issues.apache.org/jira/secure/attachment/12369419/patch_v01.txt
But, i don't know what you want exactly.
Thanks.
------------------------------
B. Regards,
Edward yoon @ NHN, corp.
Home : http://www.udanax.org
> To: hadoop-user@lucene.apache.org
> From: sales@pearsonwholesale.com
> Subject: Re: map/reduce with hbase
> Date: Sat, 10 Nov 2007 23:56:29 -0600
>
> Basically what I am trying to do is access hbase from php sense I do not
> know java and have not found it fun to learn
>
> I was looking around and found this
> https://issues.apache.org/jira/browse/HADOOP-2171
>
> but am unsure if it is what I thank it is looks like a way to access hbase
> from a socket server I have coded socket server/client in the past in php
> so
> the question is how would I use this to get access from say php.
>
> would I add this and rebuild the hbase file somehow?
>
>
> Remember I am new to java so have vary little knowing on how to do stuff.
>
> Billy
>
>
>
> "Billy" wrote in
> message news:fh55ok$r4i$1@ger.gmane.org...
>>I am using map/reduce with hadoop-0.15.0-streaming.jar to process the data
>>with php scripts. I have coded to process the data the blow is an example
>>of word counts from the input.
>>
>> bin/hadoop jar contrib/hadoop-0.15.0-streaming.jar -mapper
>> /var/www/search/hadoop/wc-mapper.php -reducer
>> /var/www/search/hadoop/wc-reducer.php -input input/* -output output
>>
>> What I am looking to do is get and store the input and output from/in
>> hbase.
>>
>> Is that possible with hadoop-0.15.0-streaming.jar?
>> If so can I get a command line example?
>>
>> thanks
>> Billy
>>
>>
>>
>>
>
>
>
_________________________________________________________________
Help yourself to FREE treats served up daily at the Messenger Caf�. Stop by
today.
http://www.cafemessenger.com/info/info_sweetstuff2.html?ocid=TXT_TAGLM_OctWLtagline
RE: map/reduce with hbase
Posted by edward yoon <we...@udanax.org>.
See my socket communication example between a php client and a java server.
https://issues.apache.org/jira/secure/attachment/12369419/patch_v01.txt
But, i don't know what you want exactly.
Thanks.
------------------------------
B. Regards,
Edward yoon @ NHN, corp.
Home : http://www.udanax.org
> To: hadoop-user@lucene.apache.org
> From: sales@pearsonwholesale.com
> Subject: Re: map/reduce with hbase
> Date: Sat, 10 Nov 2007 23:56:29 -0600
>
> Basically what I am trying to do is access hbase from php sense I do not
> know java and have not found it fun to learn
>
> I was looking around and found this
> https://issues.apache.org/jira/browse/HADOOP-2171
>
> but am unsure if it is what I thank it is looks like a way to access hbase
> from a socket server I have coded socket server/client in the past in php so
> the question is how would I use this to get access from say php.
>
> would I add this and rebuild the hbase file somehow?
>
>
> Remember I am new to java so have vary little knowing on how to do stuff.
>
> Billy
>
>
>
> "Billy" wrote in
> message news:fh55ok$r4i$1@ger.gmane.org...
>>I am using map/reduce with hadoop-0.15.0-streaming.jar to process the data
>>with php scripts. I have coded to process the data the blow is an example
>>of word counts from the input.
>>
>> bin/hadoop jar contrib/hadoop-0.15.0-streaming.jar -mapper
>> /var/www/search/hadoop/wc-mapper.php -reducer
>> /var/www/search/hadoop/wc-reducer.php -input input/* -output output
>>
>> What I am looking to do is get and store the input and output from/in
>> hbase.
>>
>> Is that possible with hadoop-0.15.0-streaming.jar?
>> If so can I get a command line example?
>>
>> thanks
>> Billy
>>
>>
>>
>>
>
>
>
_________________________________________________________________
Help yourself to FREE treats served up daily at the Messenger Café. Stop by today.
http://www.cafemessenger.com/info/info_sweetstuff2.html?ocid=TXT_TAGLM_OctWLtagline
Re: map/reduce with hbase
Posted by Vuk Ercegovac <ve...@us.ibm.com>.
In addition to setting the inputformat and outputformat classes, there
seems to be an option to specify additional name,value pairs
in the jobconf ("-jobconf"). The input and output format classes need
additional arguments that are not set using the other streaming arguments.
One example is the list of columns to scan. Please have a look at the
"initJob" method in o.a.h.hbase.mapred.TableMap and
o.a.h.hbase.mapred.TableReduce as well as the junit test,
o.a.h.hbase.mapred.TestTableMapReduce for how these are used.
I have not used the streaming package so have not tried this myself.
"Billy"
<sales@pearsonwho
lesale.com> To
Sent by: news hadoop-user@lucene.apache.org
<news@ger.gmane.o cc
rg>
Subject
Re: map/reduce with hbase
11/10/2007 09:56
PM
Please respond to
hadoop-user@lucen
e.apache.org
Basically what I am trying to do is access hbase from php sense I do not
know java and have not found it fun to learn
I was looking around and found this
https://issues.apache.org/jira/browse/HADOOP-2171
but am unsure if it is what I thank it is looks like a way to access hbase
from a socket server I have coded socket server/client in the past in php
so
the question is how would I use this to get access from say php.
would I add this and rebuild the hbase file somehow?
Remember I am new to java so have vary little knowing on how to do stuff.
Billy
"Billy" <sa...@pearsonwholesale.com> wrote in
message news:fh55ok$r4i$1@ger.gmane.org...
>I am using map/reduce with hadoop-0.15.0-streaming.jar to process the data
>with php scripts. I have coded to process the data the blow is an example
>of word counts from the input.
>
> bin/hadoop jar contrib/hadoop-0.15.0-streaming.jar -mapper
> /var/www/search/hadoop/wc-mapper.php -reducer
> /var/www/search/hadoop/wc-reducer.php -input input/* -output output
>
> What I am looking to do is get and store the input and output from/in
> hbase.
>
> Is that possible with hadoop-0.15.0-streaming.jar?
> If so can I get a command line example?
>
> thanks
> Billy
>
>
>
>
Re: map/reduce with hbase
Posted by Billy <sa...@pearsonwholesale.com>.
Basically what I am trying to do is access hbase from php sense I do not
know java and have not found it fun to learn
I was looking around and found this
https://issues.apache.org/jira/browse/HADOOP-2171
but am unsure if it is what I thank it is looks like a way to access hbase
from a socket server I have coded socket server/client in the past in php so
the question is how would I use this to get access from say php.
would I add this and rebuild the hbase file somehow?
Remember I am new to java so have vary little knowing on how to do stuff.
Billy
"Billy" <sa...@pearsonwholesale.com> wrote in
message news:fh55ok$r4i$1@ger.gmane.org...
>I am using map/reduce with hadoop-0.15.0-streaming.jar to process the data
>with php scripts. I have coded to process the data the blow is an example
>of word counts from the input.
>
> bin/hadoop jar contrib/hadoop-0.15.0-streaming.jar -mapper
> /var/www/search/hadoop/wc-mapper.php -reducer
> /var/www/search/hadoop/wc-reducer.php -input input/* -output output
>
> What I am looking to do is get and store the input and output from/in
> hbase.
>
> Is that possible with hadoop-0.15.0-streaming.jar?
> If so can I get a command line example?
>
> thanks
> Billy
>
>
>
>