[jira] Commented: (NUTCH-116) TestNDFS a JUnit test specifically for NDFS

["Doug Cutting (JIRA)" <ji...@apache.org>]

    [ http://issues.apache.org/jira/browse/NUTCH-116?page=comments#action_12332493 ] 

Doug Cutting commented on NUTCH-116:
------------------------------------

Paul,

This looks like good stuff.

I could commit it more easily if changes were restricted to those required by TestNDFS.  Changes to comments, documentation, logging, etc. are better contributed as separate patches.  It's also okay to submit a unit test that fails and then to submit fixes as separate patches.  That makes my job easier: I can first see that the unit test looks reasonable, then see that it fails, then see how the patch fixes it.  As it stands it will take me some time to fully evalute this patch.

A few quick comments: 

If BLOCKREPORT_INTERVAL and DATANODE_STARTUP_PERIOD may be overridden (as is reasonable, and perhaps required by TestNDFS) then perhaps they should be removed from FSConstants entirely.  Does that make sense?

In Server.java, why is notifyAll() safer than notify()?  The intent is to wake one and only one waiting Handler thread.  notifyAll() would cause all of the Handler threads to become runnable even when only a single call has arrived.  Is this required by TestNDFS?

Thanks,

Doug

> TestNDFS a JUnit test specifically for NDFS
> -------------------------------------------
>
>          Key: NUTCH-116
>          URL: http://issues.apache.org/jira/browse/NUTCH-116
>      Project: Nutch
>         Type: Test
>   Components: fetcher, indexer, searcher
>     Versions: 0.8-dev
>     Reporter: Paul Baclace
>  Attachments: TestNDFS.java, required_by_TestNDFS.patch
>
> TestNDFS is a JUnit test for NDFS using "pseudo multiprocessing" (or more strictly, pseudo distributed) meaning all daemons run in one process and sockets are used to communicate between daemons.  
> The test permutes various block sizes, number of files, file sizes, and number of datanodes.  After creating 1 or more files and filling them with random data, one datanode is shutdown, and then the files are verfified. Next, all the random test files are deleted and we test for leakage (non-deletion) by directly checking the real directories corresponding to the datanodes still running.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira