You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Abraham Elmahrek (JIRA)" <ji...@apache.org> on 2014/06/30 22:32:25 UTC
[jira] [Commented] (HADOOP-10725) Implement listStatus and
getFileInfo in the native client
[ https://issues.apache.org/jira/browse/HADOOP-10725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048111#comment-14048111 ]
Abraham Elmahrek commented on HADOOP-10725:
-------------------------------------------
# (hadoop-native-core/src/main/native/fs/fs.c) Pull this out into a separate function? Seems like an operation that will have to be done frequently.
{code}
+ if (hdfs_bld->nn) {
+ if ((!index(hdfs_bld->nn, '/')) && (index(hdfs_bld->nn, ':'))) {
+ // If the connection URI was set via hdfsBuilderSetNameNode, it may
+ // not be a real URI, but just a <hostname>:<port> pair. This won't
+ // parse correctly unless we add a hdfs:// scheme in front of it.
+ err = dynprintf(&uri_str, "hdfs://%s", hdfs_bld->nn);
+ if (err)
+ goto done;
+ } else {
+ uri_str = strdup(hdfs_bld->nn);
+ if (!uri_str) {
+ err = hadoop_lerr_alloc(ENOMEM, "hdfs_builder_parse_conn_uri: OOM");
goto done;
}
- uri_str = malloced_uri_str;
{code}
# (hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/hdfs.c) Should precedence be given to the explicitly defined "port" member or the pre-existing port in the URI? It seems like an explicit definition in the builder should take precedence?
{code}
- if (lastColon && (strspn(lastColon + 1, "0123456789") ==
- strlen(lastColon + 1))) {
- fprintf(stderr, "port %d was given, but URI '%s' already "
- "contains a port!\n", bld->port, bld->nn);
- return EINVAL;
+ if (!(lastColon && (strspn(lastColon + 1, "0123456789") ==
+ strlen(lastColon + 1)))) {
+ // If the URI doesn't include a port, add bld->port.
+ // Note that bld->port is ignored if the URI already has a port.
+ snprintf(suffix, sizeof(suffix), ":%d", bld->port);
{code}
# (hadoop-native-core/src/main/native/ndfs/ndfs.c) Is this how the previous HDFS clients worked? Using the previous seen filename won't work if the file has been removed. Just curious...
{code}
+ if (entries_len > 0) {
+ prev = entries[entries_len - 1].mName;
+ } else {
+ prev = "";
+ }
{code}
# (hadoop-native-core/src/main/native/jni/jnifs.c) This code segment appears to be exactly the same as hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/hdfs.c. Maybe a utility function would be useful?
{code}
- if (lastColon && (strspn(lastColon + 1, "0123456789") ==
- strlen(lastColon + 1))) {
- fprintf(stderr, "port %d was given, but URI '%s' already "
- "contains a port!\n", bld->port, bld->nn);
- return EINVAL;
+ if (!(lastColon && (strspn(lastColon + 1, "0123456789") ==
+ strlen(lastColon + 1)))) {
+ // If the URI doesn't include a port, add bld->port.
+ // Note that bld->port is ignored if the URI already has a port.
+ snprintf(suffix, sizeof(suffix), ":%d", bld->port);
{code}
All in all good stuff! Make sure to create follow up Jira's. ie: Block location is left out in 'ndfs_list_partial'.
> Implement listStatus and getFileInfo in the native client
> ---------------------------------------------------------
>
> Key: HADOOP-10725
> URL: https://issues.apache.org/jira/browse/HADOOP-10725
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: native
> Affects Versions: HADOOP-10388
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HADOOP-10725-pnative.001.patch, HADOOP-10725-pnative.002.patch
>
>
> Implement listStatus and getFileInfo in the native client.
--
This message was sent by Atlassian JIRA
(v6.2#6252)