You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Andy Isaacson (JIRA)" <ji...@apache.org> on 2012/07/12 02:19:35 UTC
[jira] [Created] (HDFS-3643) hdfsJniHelper.c unchecked string
pointers
Andy Isaacson created HDFS-3643:
-----------------------------------
Summary: hdfsJniHelper.c unchecked string pointers
Key: HDFS-3643
URL: https://issues.apache.org/jira/browse/HDFS-3643
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Andy Isaacson
Assignee: Andy Isaacson
{code}
str = methSignature;
while (*str != ')') str++;
str++;
returnType = *str;
{code}
This loop needs to check for {{'\0'}}. Also the following {{if/else if/else if}} cascade doesn't handle unexpected values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3643) hdfsJniHelper.c unchecked string
pointers
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486425#comment-13486425 ]
Andy Isaacson commented on HDFS-3643:
-------------------------------------
bq. Wouldn't it be a string which comes from the signature? eg something like "Ljava/lang/String;" or "J"?
No, it's one of the constants JINT, JLONG, etc. Valid values are ASCII alphanumeric, but in the error case it could be an arbitrary {{char}} so printing it with {{%c}} is unlikely to be helpful. I suppose casting to {{(unsigned char)returnType}} helps if char is signed and <0. I'll do that and upload a new patch.
> hdfsJniHelper.c unchecked string pointers
> -----------------------------------------
>
> Key: HDFS-3643
> URL: https://issues.apache.org/jira/browse/HDFS-3643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs-3643-1.txt, hdfs3643-2.txt, hdfs3643.txt
>
>
> {code}
> str = methSignature;
> while (*str != ')') str++;
> str++;
> returnType = *str;
> {code}
> This loop needs to check for {{'\0'}}. Also the following {{if/else if/else if}} cascade doesn't handle unexpected values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3643) hdfsJniHelper.c unchecked string
pointers
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486399#comment-13486399 ]
Todd Lipcon commented on HDFS-3643:
-----------------------------------
{code}
+ else {
+ jthr = newRuntimeError(env, "invokeMethod: invalid return type 0x%02x",
+ returnType);
+ }
{code}
Why is returnType being printed in hex here? Wouldn't it be a string which comes from the signature? eg something like "Ljava/lang/String;" or "J"?
> hdfsJniHelper.c unchecked string pointers
> -----------------------------------------
>
> Key: HDFS-3643
> URL: https://issues.apache.org/jira/browse/HDFS-3643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs-3643-1.txt, hdfs3643.txt
>
>
> {code}
> str = methSignature;
> while (*str != ')') str++;
> str++;
> returnType = *str;
> {code}
> This loop needs to check for {{'\0'}}. Also the following {{if/else if/else if}} cascade doesn't handle unexpected values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3643) hdfsJniHelper.c unchecked string
pointers
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486535#comment-13486535 ]
Hadoop QA commented on HDFS-3643:
---------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12551259/hdfs3643-2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3421//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3421//console
This message is automatically generated.
> hdfsJniHelper.c unchecked string pointers
> -----------------------------------------
>
> Key: HDFS-3643
> URL: https://issues.apache.org/jira/browse/HDFS-3643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs-3643-1.txt, hdfs3643-2.txt, hdfs3643.txt
>
>
> {code}
> str = methSignature;
> while (*str != ')') str++;
> str++;
> returnType = *str;
> {code}
> This loop needs to check for {{'\0'}}. Also the following {{if/else if/else if}} cascade doesn't handle unexpected values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3643) hdfsJniHelper.c unchecked string
pointers
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419704#comment-13419704 ]
Hadoop QA commented on HDFS-3643:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12537422/hdfs3643.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2882//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2882//console
This message is automatically generated.
> hdfsJniHelper.c unchecked string pointers
> -----------------------------------------
>
> Key: HDFS-3643
> URL: https://issues.apache.org/jira/browse/HDFS-3643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs3643.txt
>
>
> {code}
> str = methSignature;
> while (*str != ')') str++;
> str++;
> returnType = *str;
> {code}
> This loop needs to check for {{'\0'}}. Also the following {{if/else if/else if}} cascade doesn't handle unexpected values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3643) hdfsJniHelper.c unchecked string
pointers
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Isaacson updated HDFS-3643:
--------------------------------
Attachment: hdfs3643-2.txt
> hdfsJniHelper.c unchecked string pointers
> -----------------------------------------
>
> Key: HDFS-3643
> URL: https://issues.apache.org/jira/browse/HDFS-3643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs-3643-1.txt, hdfs3643-2.txt, hdfs3643.txt
>
>
> {code}
> str = methSignature;
> while (*str != ')') str++;
> str++;
> returnType = *str;
> {code}
> This loop needs to check for {{'\0'}}. Also the following {{if/else if/else if}} cascade doesn't handle unexpected values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3643) hdfsJniHelper.c unchecked string
pointers
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430752#comment-13430752 ]
Colin Patrick McCabe commented on HDFS-3643:
--------------------------------------------
I meant to write index(3).
> hdfsJniHelper.c unchecked string pointers
> -----------------------------------------
>
> Key: HDFS-3643
> URL: https://issues.apache.org/jira/browse/HDFS-3643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs3643.txt
>
>
> {code}
> str = methSignature;
> while (*str != ')') str++;
> str++;
> returnType = *str;
> {code}
> This loop needs to check for {{'\0'}}. Also the following {{if/else if/else if}} cascade doesn't handle unexpected values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3643) hdfsJniHelper.c unchecked string
pointers
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Isaacson updated HDFS-3643:
--------------------------------
Status: Patch Available (was: Open)
Patch compiles.
> hdfsJniHelper.c unchecked string pointers
> -----------------------------------------
>
> Key: HDFS-3643
> URL: https://issues.apache.org/jira/browse/HDFS-3643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs3643.txt
>
>
> {code}
> str = methSignature;
> while (*str != ')') str++;
> str++;
> returnType = *str;
> {code}
> This loop needs to check for {{'\0'}}. Also the following {{if/else if/else if}} cascade doesn't handle unexpected values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3643) hdfsJniHelper.c unchecked string
pointers
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Isaacson updated HDFS-3643:
--------------------------------
Attachment: hdfs-3643-1.txt
New patch:
# sync to trunk
# use strchr rather than hand-rolled loop
# handle a few more error cases
# clean up exception messages
# fix invokeMethod javadoc
I haven't run tests on this diff, which is a prerequisite before checkin.
> hdfsJniHelper.c unchecked string pointers
> -----------------------------------------
>
> Key: HDFS-3643
> URL: https://issues.apache.org/jira/browse/HDFS-3643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs-3643-1.txt, hdfs3643.txt
>
>
> {code}
> str = methSignature;
> while (*str != ')') str++;
> str++;
> returnType = *str;
> {code}
> This loop needs to check for {{'\0'}}. Also the following {{if/else if/else if}} cascade doesn't handle unexpected values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3643) hdfsJniHelper.c unchecked string
pointers
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430750#comment-13430750 ]
Colin Patrick McCabe commented on HDFS-3643:
--------------------------------------------
Looks good in general. While you're refactoring, I would recommend using the libc function index(2) instead of the handrolled loop. It's a little more idiomatic, and probably better optimized to boot.
Also, you'll probably need to rebase now that invokeMethod returns an exception or NULL rather than a numeric error code. You probably want to use return newRuntimeException() on error, or similar.
> hdfsJniHelper.c unchecked string pointers
> -----------------------------------------
>
> Key: HDFS-3643
> URL: https://issues.apache.org/jira/browse/HDFS-3643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs3643.txt
>
>
> {code}
> str = methSignature;
> while (*str != ')') str++;
> str++;
> returnType = *str;
> {code}
> This loop needs to check for {{'\0'}}. Also the following {{if/else if/else if}} cascade doesn't handle unexpected values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3643) hdfsJniHelper.c unchecked string
pointers
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Isaacson updated HDFS-3643:
--------------------------------
Attachment: hdfs3643.txt
Attaching simple patch to avoid walking off the end of the string, and to do something sane when the return value is incorrectly specified.
> hdfsJniHelper.c unchecked string pointers
> -----------------------------------------
>
> Key: HDFS-3643
> URL: https://issues.apache.org/jira/browse/HDFS-3643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs3643.txt
>
>
> {code}
> str = methSignature;
> while (*str != ')') str++;
> str++;
> returnType = *str;
> {code}
> This loop needs to check for {{'\0'}}. Also the following {{if/else if/else if}} cascade doesn't handle unexpected values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3643) hdfsJniHelper.c unchecked string
pointers
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Isaacson updated HDFS-3643:
--------------------------------
Component/s: libhdfs
Target Version/s: 2.0.1-alpha
Affects Version/s: 2.0.0-alpha
> hdfsJniHelper.c unchecked string pointers
> -----------------------------------------
>
> Key: HDFS-3643
> URL: https://issues.apache.org/jira/browse/HDFS-3643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
>
> {code}
> str = methSignature;
> while (*str != ')') str++;
> str++;
> returnType = *str;
> {code}
> This loop needs to check for {{'\0'}}. Also the following {{if/else if/else if}} cascade doesn't handle unexpected values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3643) hdfsJniHelper.c unchecked string
pointers
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486432#comment-13486432 ]
Todd Lipcon commented on HDFS-3643:
-----------------------------------
That's only the case if the return type is a primitive type. If, instead, it were an array or a class type, then you would see a string there. My "J" example corresponded to JLONG, but if there were a String it would be "Ljava/lang/String;". The code here is looking at the first character of the string to determine which JNI invoke method to use. But if none of the matches, that's probably because the method signature is malformed, in which case either returning the entire method signature in the error is appropriate, or returning at least everything after the ')'.
> hdfsJniHelper.c unchecked string pointers
> -----------------------------------------
>
> Key: HDFS-3643
> URL: https://issues.apache.org/jira/browse/HDFS-3643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs-3643-1.txt, hdfs3643-2.txt, hdfs3643.txt
>
>
> {code}
> str = methSignature;
> while (*str != ')') str++;
> str++;
> returnType = *str;
> {code}
> This loop needs to check for {{'\0'}}. Also the following {{if/else if/else if}} cascade doesn't handle unexpected values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3643) hdfsJniHelper.c unchecked string
pointers
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13431439#comment-13431439 ]
Hadoop QA commented on HDFS-3643:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12539898/hdfs-3643-1.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestFileConcurrentReader
org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
org.apache.hadoop.hdfs.TestPersistBlocks
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2973//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2973//console
This message is automatically generated.
> hdfsJniHelper.c unchecked string pointers
> -----------------------------------------
>
> Key: HDFS-3643
> URL: https://issues.apache.org/jira/browse/HDFS-3643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs-3643-1.txt, hdfs3643.txt
>
>
> {code}
> str = methSignature;
> while (*str != ')') str++;
> str++;
> returnType = *str;
> {code}
> This loop needs to check for {{'\0'}}. Also the following {{if/else if/else if}} cascade doesn't handle unexpected values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira