You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Isabel Drost (JIRA)" <ji...@apache.org> on 2013/04/17 14:51:17 UTC
[jira] [Updated] (MAHOUT-1047) CVB hangs after completion
[ https://issues.apache.org/jira/browse/MAHOUT-1047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Isabel Drost updated MAHOUT-1047:
---------------------------------
Attachment: MAHOUT-1047-Show-Leak.patch
Attached a hack^Wpatch that might help re-produce the issue:
com.carrotsearch.randomizedtesting-runner tracks threads spawned during unit tests. I added RandomizedTest as parent class to our Mahout Test class, marked it such to not complain about leaking threads (there are a few test classes that do not join threads), enabled thread leakage tracking only for the TestCVBModelTrainer.
The hack about the patch is adding the randomizedtesting-runner dependency to compile instead of to test scope in mahout/math.
To use the patch: make sure you have compiled Mahout trunk at least once with "mvn install", apply the patch with -p1. Than from the project root build at least mahout/math with "mvn --projects math install", after that execute the tweaked test with "mvn --projects core -Dtest=TestCVBModelTrainer test". You should see complaints like the following in the output:
Tests in error:
» ThreadLeak 32 threads leaked from TEST scope at testInMemoryCVB0(org.apache...
» ThreadLeak There are still zombie threads that couldn't be terminated:
1...
» ThreadLeak 32 threads leaked from SUITE scope at org.apache.mahout.clusteri...
» ThreadLeak There are still zombie threads that couldn't be terminated:
1...
> CVB hangs after completion
> --------------------------
>
> Key: MAHOUT-1047
> URL: https://issues.apache.org/jira/browse/MAHOUT-1047
> Project: Mahout
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 0.7
> Environment: Ubuntu
> Reporter: seth boyles
> Priority: Minor
> Labels: cvb, lda
> Fix For: 0.7, 0.8
>
> Attachments: MAHOUT-1047-Show-Leak.patch
>
>
> After running the new LDA CVB implementation, it hangs and does not terminate the process like every other time I run Mahout
> Terminal output:
> 12/07/19 11:38:49 INFO mapred.LocalJobRunner:
> 12/07/19 11:38:49 INFO mapred.Task: Task 'attempt_local_0022_m_000000_0' done.
> 12/07/19 11:38:49 INFO mapred.JobClient: map 100% reduce 0%
> 12/07/19 11:38:49 INFO mapred.JobClient: Job complete: job_local_0022
> 12/07/19 11:38:49 INFO mapred.JobClient: Counters: 8
> 12/07/19 11:38:49 INFO mapred.JobClient: File Output Format Counters
> 12/07/19 11:38:49 INFO mapred.JobClient: Bytes Written=2247793
> 12/07/19 11:38:49 INFO mapred.JobClient: File Input Format Counters
> 12/07/19 11:38:49 INFO mapred.JobClient: Bytes Read=1920337
> 12/07/19 11:38:49 INFO mapred.JobClient: FileSystemCounters
> 12/07/19 11:38:49 INFO mapred.JobClient: FILE_BYTES_READ=1342812616
> 12/07/19 11:38:49 INFO mapred.JobClient: FILE_BYTES_WRITTEN=1326092302
> 12/07/19 11:38:49 INFO mapred.JobClient: Map-Reduce Framework
> 12/07/19 11:38:49 INFO mapred.JobClient: Map input records=2772
> 12/07/19 11:38:49 INFO mapred.JobClient: Spilled Records=0
> 12/07/19 11:38:49 INFO mapred.JobClient: SPLIT_RAW_BYTES=140
> 12/07/19 11:38:49 INFO mapred.JobClient: Map output records=2772
> 12/07/19 11:38:49 INFO driver.MahoutDriver: Program took 4089950 ms (Minutes: 68.16583333333334)
> $MAHOUT_HOME/mahout cvb -i /home/seth/Scripted/mahout_data/vectors/vectors/vectors-for-cvb/ -o /home/seth/Scripted/mahout_data/clusters/ -ow -k 90 -dt /home/seth/Scripted/mahout_data/distributions -dict /home/seth/Scripted/mahout_data/vectors/vectors/dictionary.file-0 -mt /home/seth/Scripted/mahout_data/temp/ -x 20 -cd 0.05
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira