You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Arun C Murthy (JIRA)" <ji...@apache.org> on 2006/05/26 22:34:29 UTC
[jira] Created: (HADOOP-256) Implement a C api for hadoop dfs
Implement a C api for hadoop dfs
--------------------------------
Key: HADOOP-256
URL: http://issues.apache.org/jira/browse/HADOOP-256
Project: Hadoop
Type: New Feature
Components: dfs
Reporter: Arun C Murthy
Implement a C api for hadoop dfs to ease talking to hadoop's dfs from native C/C++ applications.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Resolved: (HADOOP-256) Implement a C api for hadoop dfs
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-256?page=all ]
Doug Cutting resolved HADOOP-256:
---------------------------------
Resolution: Fixed
> Implement a C api for hadoop dfs
> --------------------------------
>
> Key: HADOOP-256
> URL: http://issues.apache.org/jira/browse/HADOOP-256
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Reporter: Arun C Murthy
> Fix For: 0.3.0
> Attachments: libhdfs-new.patch.tgz, libhdfs.patch
>
> Implement a C api for hadoop dfs to ease talking to hadoop's dfs from native C/C++ applications.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-256) Implement a C api for hadoop dfs
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-256?page=all ]
Arun C Murthy updated HADOOP-256:
---------------------------------
Attachment: libhdfs-new.patch.tgz
Doug,
Please find attached a new patch incorporating your feedback. Specifically: libhdfs is now integrated with build.xml, fixed TestDFSCIO.java to automatically copy the executables to /tmp/TestDFSCIO on the remoted filesystem and some minor enhancements to the Makefile.
thanks,
Arun
> Implement a C api for hadoop dfs
> --------------------------------
>
> Key: HADOOP-256
> URL: http://issues.apache.org/jira/browse/HADOOP-256
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Reporter: Arun C Murthy
> Attachments: libhdfs-new.patch.tgz, libhdfs.patch
>
> Implement a C api for hadoop dfs to ease talking to hadoop's dfs from native C/C++ applications.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-256) Implement a C api for hadoop dfs
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-256?page=comments#action_12413540 ]
Doug Cutting commented on HADOOP-256:
-------------------------------------
This looks good, but the build still needs some work.
I tried to run 'make' in the source directory, but had to first add $(LDFLAGS) to a number of the targets to stop complaints about finding libjvm.so. Also, it would be best to add a target to build.xml that builds this. Builds should not write files in the source tree: it would be better to have all of the compiler outputs placed in the top-level build directory (e.g., in ../../../build/libhdfs, or as specified in an environment variable passed from build.xml).
Most of the above are also valid complaints about librecordio. I must not have been paying enough attention when I committed that!
The tests depend on executables in /tmp, but, as far as I can see, there's nothing that puts them there.
The goal is that one should be able to check things out of subversion and run something like:
ant test-libhdfs
And have the code build and run. It is safe to assume that folks have gcc, make, etc. installed and available. Ideally this would work on Cygwin as well as linux, but that's a lower priority.
Thanks!
> Implement a C api for hadoop dfs
> --------------------------------
>
> Key: HADOOP-256
> URL: http://issues.apache.org/jira/browse/HADOOP-256
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Reporter: Arun C Murthy
> Attachments: libhdfs.patch
>
> Implement a C api for hadoop dfs to ease talking to hadoop's dfs from native C/C++ applications.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Closed: (HADOOP-256) Implement a C api for hadoop dfs
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-256?page=all ]
Doug Cutting closed HADOOP-256:
-------------------------------
> Implement a C api for hadoop dfs
> --------------------------------
>
> Key: HADOOP-256
> URL: http://issues.apache.org/jira/browse/HADOOP-256
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Reporter: Arun C Murthy
> Fix For: 0.3.0
> Attachments: libhdfs-new.patch.tgz, libhdfs.patch
>
> Implement a C api for hadoop dfs to ease talking to hadoop's dfs from native C/C++ applications.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-256) Implement a C api for hadoop dfs
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-256?page=comments#action_12413660 ]
Arun C Murthy commented on HADOOP-256:
--------------------------------------
Doug,
Appreciate your feedback... some minor queries/responses:
a) I'm working on getting the whole thing integrated with build.xml/ant and also about putting all outputs in the top-level build directory. librecordio led me down the wrong garden path.
b) Can you please elaborate on the issues you had with LDFLAGS? On my system (Gentoo GNU/Linux - 2.6.16.11) make/makeclean works for me i.e. there are no issues building it. I'm relying on $JAVA_HOME in my Makefile (it's set to /opt/sun-jdk-1.5.0.06 on my box - afaik it's the recommended way?) to point me libjvm.so. This isn't prefect (e.g. $(JAVA_HOME)/jre/lib/i386/server won't work on non i386 architectures??) Is there a better alternative?
Since I wasn't clear which system path to dump the .so I'm building (any suggestions?), I just used LD_PRELOAD to run my executable (hdfs_test) - admit I should have clearly pointed this out.
For the same reason I had to resort to --rpath tricks to get TestDFSCIO to work (i.e. both hdfs_read & hdfs_write executables have an --rpath for libhdfs.so.1). This was very necessary for getting the hdfs_{read|write} to work with map/reduce. Appreciate any suggestions...
c) Does it make sense to just copy the executables from $(HADOOP_HOME)/libhdfs to /tmp/TestDFSCIO from the client (initialization phase) on which the tests are initiated? Admit this completely slipped my mind, glaringly. My bad.
thanks,
Arun
> Implement a C api for hadoop dfs
> --------------------------------
>
> Key: HADOOP-256
> URL: http://issues.apache.org/jira/browse/HADOOP-256
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Reporter: Arun C Murthy
> Attachments: libhdfs.patch
>
> Implement a C api for hadoop dfs to ease talking to hadoop's dfs from native C/C++ applications.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-256) Implement a C api for hadoop dfs
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-256?page=comments#action_12414132 ]
Doug Cutting commented on HADOOP-256:
-------------------------------------
I just committed this. I changed the path to 'ld' from '/usr/bin/ld' to simply 'ld', since it's installed in /bin on my Ubuntu system. Thanks, Arun!
> Implement a C api for hadoop dfs
> --------------------------------
>
> Key: HADOOP-256
> URL: http://issues.apache.org/jira/browse/HADOOP-256
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Reporter: Arun C Murthy
> Fix For: 0.3
> Attachments: libhdfs-new.patch.tgz, libhdfs.patch
>
> Implement a C api for hadoop dfs to ease talking to hadoop's dfs from native C/C++ applications.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-256) Implement a C api for hadoop dfs
Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-256?page=all ]
Sameer Paranjpye updated HADOOP-256:
------------------------------------
Fix Version: 0.3
> Implement a C api for hadoop dfs
> --------------------------------
>
> Key: HADOOP-256
> URL: http://issues.apache.org/jira/browse/HADOOP-256
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Reporter: Arun C Murthy
> Fix For: 0.3
> Attachments: libhdfs-new.patch.tgz, libhdfs.patch
>
> Implement a C api for hadoop dfs to ease talking to hadoop's dfs from native C/C++ applications.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-256) Implement a C api for hadoop dfs
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-256?page=comments#action_12413538 ]
Arun C Murthy commented on HADOOP-256:
--------------------------------------
I have attached libhdfs.patch...
It creates a new sub-dir: hadoop/src/c++/libhdfs and also a port of hadoop/src/test/org/apache/hadoop/fs/TestDFSIO.java called src/test/org/apache/hadoop/fs/TestDFSCIO.java which I have added to src/test/org/apache/hadoop/test/AllTestDriver.java.
thanks,
Arun
> Implement a C api for hadoop dfs
> --------------------------------
>
> Key: HADOOP-256
> URL: http://issues.apache.org/jira/browse/HADOOP-256
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Reporter: Arun C Murthy
> Attachments: libhdfs.patch
>
> Implement a C api for hadoop dfs to ease talking to hadoop's dfs from native C/C++ applications.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
unsubscribe
Posted by lin yuan <li...@msn.com>.
unsubscribe
_________________________________________________________________
与联机的朋友进行交流，请使用 MSN Messenger: http://messenger.msn.com/cn
[jira] Updated: (HADOOP-256) Implement a C api for hadoop dfs
Posted by "Anonymous (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-256?page=all ]
updated HADOOP-256:
--------------------
Attachment: libhdfs.patch
> Implement a C api for hadoop dfs
> --------------------------------
>
> Key: HADOOP-256
> URL: http://issues.apache.org/jira/browse/HADOOP-256
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Reporter: Arun C Murthy
> Attachments: libhdfs.patch
>
> Implement a C api for hadoop dfs to ease talking to hadoop's dfs from native C/C++ applications.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira