You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Anh Vũ Nguyễn <an...@gmail.com> on 2009/02/06 09:14:17 UTC

re : How to use MapFile in C++ program

Hi, everybody. I am writing a project in C++ and want to use the power of
MapFile class(which belongs to org.apache.hadoop.io) of hadoop. Can you
please tell me how can I write code in C++ using MapFile or there is no way
to use API org.apache.hadoop.io in c++ (libhdfs only helps with
org.apache.hadoop.fs).
Thanks in advance!

Re: re : How to use MapFile in C++ program

Posted by jason hadoop <ja...@gmail.com>.
You may wish to look at the documentation on hadoop pipes, which provide a
interface for writing c++ map/reduce applications and a mechanism to pass
key/value data to C++ from hadoop.

The framework will read and write sequence file or mapfiles, and provide
key/value pairs to the map function and reduce function and accept key value
pairs from them.

I have not personally used pipes, so I do not speak from experience, but
from reading src/mapred/org/apache/hadoop/mapred/pipes/PipesMapRunner.java
it is clear that standard hadoop inputs are handled and the materialized key
and value pairs passed to the map method of the pipes job.



On Tue, Feb 24, 2009 at 7:41 PM, Anh Vũ Nguyễn <an...@gmail.com> wrote:

> Thanks for your help. I read your answer and then forgot to thank you.
> Sorry
> about that.
> Regards,
> Vu Nguyen.
>
> On 2/6/09, Enis Soztutar <en...@gmail.com> wrote:
> >
> > There is currently no way to read MapFiles in any language other than
> Java.
> > You can write a JNI wrapper similar to libhdfs.
> > Alternatively, you can also write the complete stack from scratch,
> however
> > this might prove very difficult or impossible. You might want to check
> the
> > ObjectFile/TFile specifications for which binary compatible
> reader/writers
> > can be developed in any language :
> >
> > https://issues.apache.org/jira/browse/HADOOP-3315
> >
> > Enis
> >
> > Anh Vũ Nguyễn wrote:
> >
> >> Hi, everybody. I am writing a project in C++ and want to use the power
> of
> >> MapFile class(which belongs to org.apache.hadoop.io) of hadoop. Can you
> >> please tell me how can I write code in C++ using MapFile or there is no
> >> way
> >> to use API org.apache.hadoop.io in c++ (libhdfs only helps with
> >> org.apache.hadoop.fs).
> >> Thanks in advance!
> >>
> >>
> >>
> >
>

Re: re : How to use MapFile in C++ program

Posted by Anh Vũ Nguyễn <an...@gmail.com>.
Thanks for your help. I read your answer and then forgot to thank you. Sorry
about that.
Regards,
Vu Nguyen.

On 2/6/09, Enis Soztutar <en...@gmail.com> wrote:
>
> There is currently no way to read MapFiles in any language other than Java.
> You can write a JNI wrapper similar to libhdfs.
> Alternatively, you can also write the complete stack from scratch, however
> this might prove very difficult or impossible. You might want to check the
> ObjectFile/TFile specifications for which binary compatible reader/writers
> can be developed in any language :
>
> https://issues.apache.org/jira/browse/HADOOP-3315
>
> Enis
>
> Anh Vũ Nguyễn wrote:
>
>> Hi, everybody. I am writing a project in C++ and want to use the power of
>> MapFile class(which belongs to org.apache.hadoop.io) of hadoop. Can you
>> please tell me how can I write code in C++ using MapFile or there is no
>> way
>> to use API org.apache.hadoop.io in c++ (libhdfs only helps with
>> org.apache.hadoop.fs).
>> Thanks in advance!
>>
>>
>>
>

Re: re : How to use MapFile in C++ program

Posted by Enis Soztutar <en...@gmail.com>.
There is currently no way to read MapFiles in any language other than 
Java. You can write a JNI wrapper similar to libhdfs.
Alternatively, you can also write the complete stack from scratch, 
however this might prove very difficult or impossible. You might want to 
check the ObjectFile/TFile specifications for which binary compatible 
reader/writers can be developed in any language :

https://issues.apache.org/jira/browse/HADOOP-3315

Enis

Anh Vũ Nguyễn wrote:
> Hi, everybody. I am writing a project in C++ and want to use the power of
> MapFile class(which belongs to org.apache.hadoop.io) of hadoop. Can you
> please tell me how can I write code in C++ using MapFile or there is no way
> to use API org.apache.hadoop.io in c++ (libhdfs only helps with
> org.apache.hadoop.fs).
> Thanks in advance!
>
>