A possible bug in unit test: TestDFSShell.testPut()

[侯松 <ho...@taobao.com>]

Hi all,

Recently I am rewriting testcases of Hadoop, and I’ve just found a possible bug in TestDFSShell.testPut().

testPut() tries to copy a file from local to a destination “dst” in HDFS, and uses a SecurityManager to capture this action. The new SecurityManager filters out this copy action from all actions it receives , and starts a second thread to copy a second file to the same destination “dst” in HDFS. The main thread sleeps 5 seconds immediately after. I think the original purpose is that, the copy from the main thread should succeed, and the second copy from the new thread should fail because the “dst” is being written. My guess can be confirmed by the asserts around these codes.

However, the filter in the new SecurityManager is not right. It tries to find "FileUtil.copyContent", but these is no method called copyContent in FileUtil, and therefore the second thread is never started. Although the asserts always succeed, this code is not able to find bugs in this area.

I also debugged it, and I can never hit the break points in the second copy thread. Even though I changed the filter from "FileUtil.copyContent" to "FileUtil.copy" and successfully started the second thread, the second thread succeeds before the main thread resumes, which is the opposite of the logic.

Any of you have any ideas about this issue?

Thanks a lot!

Hou Song

________________________________

This email (including any attachments) is confidential and may be legally privileged. If you received this email in error, please delete it immediately and do not copy it or use it for any purpose or disclose its contents to any other person. Thank you.

本电邮(包括任何附件)可能含有机密资料并受法律保护。如您不是正确的收件人，请您立即删除本邮件。请不要将本电邮进行复制并用作任何其他用途、或透露本邮件之内容。谢谢。

Re: A possible bug in unit test: TestDFSShell.testPut()

["Aaron T. Myers" <at...@cloudera.com>]

Hi Hou Song,

This sounds like a bug to me. Please file a JIRA in the HDFS project by
going to this link:
https://issues.apache.org/jira/secure/CreateIssue!default.jspa

--
Aaron T. Myers
Software Engineer, Cloudera



2011/10/12 侯松 <ho...@taobao.com>

> Hi all,
>
> Recently I am rewriting testcases of Hadoop, and I’ve just found a possible
> bug in TestDFSShell.testPut().
>
> testPut() tries to copy a file from local to a destination “dst” in HDFS,
> and uses a SecurityManager to capture this action. The new SecurityManager
> filters out this copy action from all actions it receives , and starts a
> second thread to copy a second file to the same destination “dst” in HDFS.
> The main thread sleeps 5 seconds immediately after. I think the original
> purpose is that, the copy from the main thread should succeed, and the
> second copy from the new thread should fail because the “dst” is being
> written. My guess can be confirmed by the asserts around these codes.
>
> However, the filter in the new SecurityManager is not right. It tries to
> find "FileUtil.copyContent", but these is no method called copyContent in
> FileUtil, and therefore the second thread is never started. Although the
> asserts always succeed, this code is not able to find bugs in this area.
>
> I also debugged it, and I can never hit the break points in the second copy
> thread. Even though I changed the filter from "FileUtil.copyContent" to
> "FileUtil.copy" and successfully started the second thread, the second
> thread succeeds before the main thread resumes, which is the opposite of the
> logic.
>
> Any of you have any ideas about this issue?
>
> Thanks a lot!
>
> Hou Song
>
> ________________________________
>
> This email (including any attachments) is confidential and may be legally
> privileged. If you received this email in error, please delete it
> immediately and do not copy it or use it for any purpose or disclose its
> contents to any other person. Thank you.
>
>
> 本电邮(包括任何附件)可能含有机密资料并受法律保护。如您不是正确的收件人，请您立即删除本邮件。请不要将本电邮进行复制并用作任何其他用途、或透露本邮件之内容。谢谢。
>