question aboutc++ libhdfs and a bug comment on the libdfs test case

[Jason Venner <ja...@attributor.com>]

We were wondering how stable it was for use? We have some 10 gig data 
sets that need a little translation before our java based hadoop jobs 
can run on them, and the translation library is in c++. We are thinking 
of using libdfs to to a translate while copying the data in to hdfs.

Any gotchas with libhdfs that we should know about.

While looking at the the HEAD version of the


  /lucene/hadoop/trunk/src/c++/libhdfs/hdfs_test.c,

I noticed a small typeo


*int* *main*(*int* argc, *char* **argv) {

    hdfsFS fs = hdfsConnect(*"default"*, 0);
    *if*(!fs) {
        fprintf(stderr, *"Oops! Failed to connect to hdfs!\n"*);
        exit(-1);
    } 
 
    hdfsFS lfs = hdfsConnect(NULL, 0);
    *if*(!fs) {
        fprintf(stderr, *"Oops! Failed to connect to 'local' hdfs!\n"*);
        exit(-1);
    } 
 
I believe that test should be if(!lfs) {

RE: question aboutc++ libhdfs and a bug comment on the libdfs test case

[dhruba Borthakur <dh...@yahoo-inc.com>]

Hi Jason,

Good catch. It would be great if you can create a JIRA issue and submit
your code change as a patch for this problem.

There are some big sites (about 1000 node clusters) that use libhdfs to
access HDFS.

Thanks,
Dhruba

-----Original Message-----
From: Jason Venner [mailto:jason@attributor.com] 
Sent: Monday, January 21, 2008 2:38 PM
To: hadoop-user@lucene.apache.org
Subject: question aboutc++ libhdfs and a bug comment on the libdfs test
case

We were wondering how stable it was for use? We have some 10 gig data 
sets that need a little translation before our java based hadoop jobs 
can run on them, and the translation library is in c++. We are thinking 
of using libdfs to to a translate while copying the data in to hdfs.

Any gotchas with libhdfs that we should know about.

While looking at the the HEAD version of the


  /lucene/hadoop/trunk/src/c++/libhdfs/hdfs_test.c,

I noticed a small typeo


*int* *main*(*int* argc, *char* **argv) {

    hdfsFS fs = hdfsConnect(*"default"*, 0);
    *if*(!fs) {
        fprintf(stderr, *"Oops! Failed to connect to hdfs!\n"*);
        exit(-1);
    } 
 
    hdfsFS lfs = hdfsConnect(NULL, 0);
    *if*(!fs) {
        fprintf(stderr, *"Oops! Failed to connect to 'local' hdfs!\n"*);
        exit(-1);
    } 
 
I believe that test should be if(!lfs) {