You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2012/04/02 21:33:29 UTC
Build failed in Jenkins: Mahout-Examples-Cluster-Reuters #90
See <https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/90/>
------------------------------------------
[...truncated 6223 lines...]
12/04/02 19:32:53 INFO mapred.JobClient: Map output bytes=14791487
12/04/02 19:32:53 INFO mapred.JobClient: Combine input records=0
12/04/02 19:32:53 INFO mapred.JobClient: Map output records=21578
12/04/02 19:32:53 INFO mapred.JobClient: SPLIT_RAW_BYTES=636
12/04/02 19:32:53 INFO mapred.JobClient: Reduce input records=21578
12/04/02 19:32:54 INFO input.FileInputFormat: Total input paths to process : 1
12/04/02 19:32:54 INFO mapred.JobClient: Running job: job_local_0004
12/04/02 19:32:54 INFO mapred.MapTask: io.sort.mb = 100
12/04/02 19:32:54 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/02 19:32:54 INFO mapred.MapTask: record buffer = 262144/327680
12/04/02 19:32:55 INFO mapred.JobClient: map 0% reduce 0%
12/04/02 19:32:55 INFO mapred.MapTask: Starting flush of map output
12/04/02 19:32:55 INFO mapred.MapTask: Finished spill 0
12/04/02 19:32:55 INFO mapred.Task: Task:attempt_local_0004_m_000000_0 is done. And is in the process of commiting
12/04/02 19:32:57 INFO mapred.LocalJobRunner:
12/04/02 19:32:57 INFO mapred.Task: Task 'attempt_local_0004_m_000000_0' done.
12/04/02 19:32:57 INFO mapred.LocalJobRunner:
12/04/02 19:32:57 INFO mapred.Merger: Merging 1 sorted segments
12/04/02 19:32:57 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/02 19:32:57 INFO mapred.LocalJobRunner:
12/04/02 19:32:58 INFO mapred.JobClient: map 100% reduce 0%
12/04/02 19:32:58 INFO mapred.Task: Task:attempt_local_0004_r_000000_0 is done. And is in the process of commiting
12/04/02 19:32:58 INFO mapred.LocalJobRunner:
12/04/02 19:32:58 INFO mapred.Task: Task attempt_local_0004_r_000000_0 is allowed to commit now
12/04/02 19:32:58 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0004_r_000000_0' to /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/02 19:33:00 INFO mapred.LocalJobRunner: reduce > reduce
12/04/02 19:33:00 INFO mapred.Task: Task 'attempt_local_0004_r_000000_0' done.
12/04/02 19:33:01 INFO mapred.JobClient: map 100% reduce 100%
12/04/02 19:33:01 INFO mapred.JobClient: Job complete: job_local_0004
12/04/02 19:33:01 INFO mapred.JobClient: Counters: 16
12/04/02 19:33:01 INFO mapred.JobClient: File Output Format Counters
12/04/02 19:33:01 INFO mapred.JobClient: Bytes Written=17780621
12/04/02 19:33:01 INFO mapred.JobClient: FileSystemCounters
12/04/02 19:33:01 INFO mapred.JobClient: FILE_BYTES_READ=392447742
12/04/02 19:33:01 INFO mapred.JobClient: FILE_BYTES_WRITTEN=361927015
12/04/02 19:33:01 INFO mapred.JobClient: File Input Format Counters
12/04/02 19:33:01 INFO mapred.JobClient: Bytes Read=17780621
12/04/02 19:33:01 INFO mapred.JobClient: Map-Reduce Framework
12/04/02 19:33:01 INFO mapred.JobClient: Reduce input groups=21578
12/04/02 19:33:01 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/02 19:33:01 INFO mapred.JobClient: Combine output records=0
12/04/02 19:33:01 INFO mapred.JobClient: Map input records=21578
12/04/02 19:33:01 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/02 19:33:01 INFO mapred.JobClient: Reduce output records=21578
12/04/02 19:33:01 INFO mapred.JobClient: Spilled Records=43156
12/04/02 19:33:01 INFO mapred.JobClient: Map output bytes=17337483
12/04/02 19:33:01 INFO mapred.JobClient: Combine input records=0
12/04/02 19:33:01 INFO mapred.JobClient: Map output records=21578
12/04/02 19:33:01 INFO mapred.JobClient: SPLIT_RAW_BYTES=157
12/04/02 19:33:01 INFO mapred.JobClient: Reduce input records=21578
12/04/02 19:33:01 INFO common.HadoopUtil: Deleting /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/02 19:33:01 INFO input.FileInputFormat: Total input paths to process : 1
12/04/02 19:33:01 INFO mapred.JobClient: Running job: job_local_0005
12/04/02 19:33:01 INFO mapred.MapTask: io.sort.mb = 100
12/04/02 19:33:01 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/02 19:33:01 INFO mapred.MapTask: record buffer = 262144/327680
12/04/02 19:33:01 INFO mapred.MapTask: Spilling map output: record full = true
12/04/02 19:33:01 INFO mapred.MapTask: bufstart = 0; bufend = 3145728; bufvoid = 99614720
12/04/02 19:33:01 INFO mapred.MapTask: kvstart = 0; kvend = 262144; length = 327680
12/04/02 19:33:02 INFO mapred.JobClient: map 0% reduce 0%
12/04/02 19:33:02 INFO mapred.MapTask: Finished spill 0
12/04/02 19:33:02 INFO mapred.MapTask: Spilling map output: record full = true
12/04/02 19:33:02 INFO mapred.MapTask: bufstart = 3145728; bufend = 6291444; bufvoid = 99614720
12/04/02 19:33:02 INFO mapred.MapTask: kvstart = 262144; kvend = 196607; length = 327680
12/04/02 19:33:03 INFO mapred.MapTask: Finished spill 1
12/04/02 19:33:03 INFO mapred.MapTask: Spilling map output: record full = true
12/04/02 19:33:03 INFO mapred.MapTask: bufstart = 6291444; bufend = 9437160; bufvoid = 99614720
12/04/02 19:33:03 INFO mapred.MapTask: kvstart = 196607; kvend = 131070; length = 327680
12/04/02 19:33:03 INFO mapred.MapTask: Finished spill 2
12/04/02 19:33:03 INFO mapred.MapTask: Spilling map output: record full = true
12/04/02 19:33:03 INFO mapred.MapTask: bufstart = 9437160; bufend = 12582876; bufvoid = 99614720
12/04/02 19:33:03 INFO mapred.MapTask: kvstart = 131070; kvend = 65533; length = 327680
12/04/02 19:33:04 INFO mapred.MapTask: Finished spill 3
12/04/02 19:33:04 INFO mapred.MapTask: Spilling map output: record full = true
12/04/02 19:33:04 INFO mapred.MapTask: bufstart = 12582876; bufend = 15728604; bufvoid = 99614720
12/04/02 19:33:04 INFO mapred.MapTask: kvstart = 65533; kvend = 327677; length = 327680
12/04/02 19:33:04 INFO mapred.MapTask: Finished spill 4
12/04/02 19:33:04 INFO mapred.MapTask: Starting flush of map output
12/04/02 19:33:04 INFO mapred.MapTask: Finished spill 5
12/04/02 19:33:04 INFO mapred.Merger: Merging 6 sorted segments
12/04/02 19:33:04 INFO mapred.Merger: Down to the last merge-pass, with 6 segments left of total size: 1871742 bytes
12/04/02 19:33:05 INFO mapred.Task: Task:attempt_local_0005_m_000000_0 is done. And is in the process of commiting
12/04/02 19:33:07 INFO mapred.LocalJobRunner:
12/04/02 19:33:07 INFO mapred.LocalJobRunner:
12/04/02 19:33:07 INFO mapred.Task: Task 'attempt_local_0005_m_000000_0' done.
12/04/02 19:33:08 INFO mapred.LocalJobRunner:
12/04/02 19:33:08 INFO mapred.Merger: Merging 1 sorted segments
12/04/02 19:33:08 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 585314 bytes
12/04/02 19:33:08 INFO mapred.LocalJobRunner:
12/04/02 19:33:08 INFO mapred.Task: Task:attempt_local_0005_r_000000_0 is done. And is in the process of commiting
12/04/02 19:33:08 INFO mapred.LocalJobRunner:
12/04/02 19:33:08 INFO mapred.Task: Task attempt_local_0005_r_000000_0 is allowed to commit now
12/04/02 19:33:08 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0005_r_000000_0' to /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/df-count
12/04/02 19:33:08 INFO mapred.JobClient: map 100% reduce 0%
12/04/02 19:33:11 INFO mapred.LocalJobRunner: reduce > reduce
12/04/02 19:33:11 INFO mapred.Task: Task 'attempt_local_0005_r_000000_0' done.
12/04/02 19:33:11 INFO mapred.JobClient: map 100% reduce 100%
12/04/02 19:33:11 INFO mapred.JobClient: Job complete: job_local_0005
12/04/02 19:33:11 INFO mapred.JobClient: Counters: 16
12/04/02 19:33:11 INFO mapred.JobClient: File Output Format Counters
12/04/02 19:33:11 INFO mapred.JobClient: Bytes Written=851221
12/04/02 19:33:11 INFO mapred.JobClient: FileSystemCounters
12/04/02 19:33:11 INFO mapred.JobClient: FILE_BYTES_READ=498028082
12/04/02 19:33:11 INFO mapred.JobClient: FILE_BYTES_WRITTEN=434185285
12/04/02 19:33:11 INFO mapred.JobClient: File Input Format Counters
12/04/02 19:33:11 INFO mapred.JobClient: Bytes Read=17780621
12/04/02 19:33:11 INFO mapred.JobClient: Map-Reduce Framework
12/04/02 19:33:11 INFO mapred.JobClient: Reduce input groups=41808
12/04/02 19:33:11 INFO mapred.JobClient: Map output materialized bytes=585318
12/04/02 19:33:11 INFO mapred.JobClient: Combine output records=175503
12/04/02 19:33:11 INFO mapred.JobClient: Map input records=21578
12/04/02 19:33:11 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/02 19:33:11 INFO mapred.JobClient: Reduce output records=41808
12/04/02 19:33:11 INFO mapred.JobClient: Spilled Records=217311
12/04/02 19:33:11 INFO mapred.JobClient: Map output bytes=18371184
12/04/02 19:33:11 INFO mapred.JobClient: Combine input records=1664627
12/04/02 19:33:11 INFO mapred.JobClient: Map output records=1530932
12/04/02 19:33:11 INFO mapred.JobClient: SPLIT_RAW_BYTES=150
12/04/02 19:33:11 INFO mapred.JobClient: Reduce input records=41808
12/04/02 19:33:12 INFO input.FileInputFormat: Total input paths to process : 1
12/04/02 19:33:12 INFO filecache.TrackerDistributedCacheManager: Creating frequency.file-0 in /tmp/hadoop-hudson/mapred/local/archive/103354343583574224_1334525619_1955329860/file/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans-work-5280379547316864023 with rwxr-xr-x
12/04/02 19:33:12 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-hudson/mapred/local/archive/103354343583574224_1334525619_1955329860/file/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/02 19:33:12 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-hudson/mapred/local/archive/103354343583574224_1334525619_1955329860/file/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/02 19:33:12 INFO mapred.JobClient: Running job: job_local_0006
12/04/02 19:33:12 INFO mapred.MapTask: io.sort.mb = 100
12/04/02 19:33:12 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/02 19:33:12 INFO mapred.MapTask: record buffer = 262144/327680
12/04/02 19:33:13 INFO mapred.JobClient: map 0% reduce 0%
12/04/02 19:33:13 INFO mapred.MapTask: Starting flush of map output
12/04/02 19:33:13 INFO mapred.MapTask: Finished spill 0
12/04/02 19:33:13 INFO mapred.Task: Task:attempt_local_0006_m_000000_0 is done. And is in the process of commiting
12/04/02 19:33:15 INFO mapred.LocalJobRunner:
12/04/02 19:33:15 INFO mapred.Task: Task 'attempt_local_0006_m_000000_0' done.
12/04/02 19:33:15 INFO mapred.LocalJobRunner:
12/04/02 19:33:15 INFO mapred.Merger: Merging 1 sorted segments
12/04/02 19:33:15 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/02 19:33:15 INFO mapred.LocalJobRunner:
12/04/02 19:33:15 INFO mapred.Task: Task:attempt_local_0006_r_000000_0 is done. And is in the process of commiting
12/04/02 19:33:15 INFO mapred.LocalJobRunner:
12/04/02 19:33:15 INFO mapred.Task: Task attempt_local_0006_r_000000_0 is allowed to commit now
12/04/02 19:33:15 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0006_r_000000_0' to /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/02 19:33:16 INFO mapred.JobClient: map 100% reduce 0%
12/04/02 19:33:18 INFO mapred.LocalJobRunner: reduce > reduce
12/04/02 19:33:18 INFO mapred.Task: Task 'attempt_local_0006_r_000000_0' done.
12/04/02 19:33:19 INFO mapred.JobClient: map 100% reduce 100%
12/04/02 19:33:19 INFO mapred.JobClient: Job complete: job_local_0006
12/04/02 19:33:19 INFO mapred.JobClient: Counters: 16
12/04/02 19:33:19 INFO mapred.JobClient: File Output Format Counters
12/04/02 19:33:19 INFO mapred.JobClient: Bytes Written=4914503
12/04/02 19:33:19 INFO mapred.JobClient: FileSystemCounters
12/04/02 19:33:19 INFO mapred.JobClient: FILE_BYTES_READ=604120935
12/04/02 19:33:19 INFO mapred.JobClient: FILE_BYTES_WRITTEN=526912355
12/04/02 19:33:19 INFO mapred.JobClient: File Input Format Counters
12/04/02 19:33:19 INFO mapred.JobClient: Bytes Read=17780621
12/04/02 19:33:19 INFO mapred.JobClient: Map-Reduce Framework
12/04/02 19:33:19 INFO mapred.JobClient: Reduce input groups=21578
12/04/02 19:33:19 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/02 19:33:19 INFO mapred.JobClient: Combine output records=0
12/04/02 19:33:19 INFO mapred.JobClient: Map input records=21578
12/04/02 19:33:19 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/02 19:33:19 INFO mapred.JobClient: Reduce output records=21578
12/04/02 19:33:19 INFO mapred.JobClient: Spilled Records=43156
12/04/02 19:33:19 INFO mapred.JobClient: Map output bytes=17337483
12/04/02 19:33:19 INFO mapred.JobClient: Combine input records=0
12/04/02 19:33:19 INFO mapred.JobClient: Map output records=21578
12/04/02 19:33:19 INFO mapred.JobClient: SPLIT_RAW_BYTES=150
12/04/02 19:33:19 INFO mapred.JobClient: Reduce input records=21578
12/04/02 19:33:19 INFO input.FileInputFormat: Total input paths to process : 1
12/04/02 19:33:19 INFO mapred.JobClient: Running job: job_local_0007
12/04/02 19:33:19 INFO mapred.MapTask: io.sort.mb = 100
12/04/02 19:33:19 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/02 19:33:19 INFO mapred.MapTask: record buffer = 262144/327680
12/04/02 19:33:19 INFO mapred.MapTask: Starting flush of map output
12/04/02 19:33:19 INFO mapred.MapTask: Finished spill 0
12/04/02 19:33:19 INFO mapred.Task: Task:attempt_local_0007_m_000000_0 is done. And is in the process of commiting
12/04/02 19:33:20 INFO mapred.JobClient: map 0% reduce 0%
12/04/02 19:33:22 INFO mapred.LocalJobRunner:
12/04/02 19:33:22 INFO mapred.Task: Task 'attempt_local_0007_m_000000_0' done.
12/04/02 19:33:23 INFO mapred.JobClient: map 100% reduce 0%
12/04/02 19:33:23 INFO mapred.LocalJobRunner:
12/04/02 19:33:23 INFO mapred.Merger: Merging 1 sorted segments
12/04/02 19:33:23 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 4719200 bytes
12/04/02 19:33:23 INFO mapred.LocalJobRunner:
12/04/02 19:33:23 INFO mapred.Task: Task:attempt_local_0007_r_000000_0 is done. And is in the process of commiting
12/04/02 19:33:23 INFO mapred.LocalJobRunner:
12/04/02 19:33:23 INFO mapred.Task: Task attempt_local_0007_r_000000_0 is allowed to commit now
12/04/02 19:33:23 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0007_r_000000_0' to /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/02 19:33:26 INFO mapred.LocalJobRunner: reduce > reduce
12/04/02 19:33:26 INFO mapred.Task: Task 'attempt_local_0007_r_000000_0' done.
12/04/02 19:33:27 INFO mapred.JobClient: map 100% reduce 100%
12/04/02 19:33:27 INFO mapred.JobClient: Job complete: job_local_0007
12/04/02 19:33:27 INFO mapred.JobClient: Counters: 16
12/04/02 19:33:27 INFO mapred.JobClient: File Output Format Counters
12/04/02 19:33:27 INFO mapred.JobClient: Bytes Written=4914503
12/04/02 19:33:27 INFO mapred.JobClient: FileSystemCounters
12/04/02 19:33:27 INFO mapred.JobClient: FILE_BYTES_READ=685210606
12/04/02 19:33:27 INFO mapred.JobClient: FILE_BYTES_WRITTEN=594892585
12/04/02 19:33:27 INFO mapred.JobClient: File Input Format Counters
12/04/02 19:33:27 INFO mapred.JobClient: Bytes Read=4914503
12/04/02 19:33:27 INFO mapred.JobClient: Map-Reduce Framework
12/04/02 19:33:27 INFO mapred.JobClient: Reduce input groups=21578
12/04/02 19:33:27 INFO mapred.JobClient: Map output materialized bytes=4719204
12/04/02 19:33:27 INFO mapred.JobClient: Combine output records=0
12/04/02 19:33:27 INFO mapred.JobClient: Map input records=21578
12/04/02 19:33:27 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/02 19:33:27 INFO mapred.JobClient: Reduce output records=21578
12/04/02 19:33:27 INFO mapred.JobClient: Spilled Records=43156
12/04/02 19:33:27 INFO mapred.JobClient: Map output bytes=4659281
12/04/02 19:33:27 INFO mapred.JobClient: Combine input records=0
12/04/02 19:33:27 INFO mapred.JobClient: Map output records=21578
12/04/02 19:33:27 INFO mapred.JobClient: SPLIT_RAW_BYTES=157
12/04/02 19:33:27 INFO mapred.JobClient: Reduce input records=21578
12/04/02 19:33:27 INFO common.HadoopUtil: Deleting /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/02 19:33:27 INFO driver.MahoutDriver: Program took 80251 ms (Minutes: 1.3375166666666667)
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
no HADOOP_HOME set, running locally
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/mahout-examples-0.7-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-jcl-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
12/04/02 19:33:27 INFO common.AbstractJob: Command line arguments: {--clustering=null, --clusters=[/tmp/mahout-work-hudson/reuters-kmeans-clusters], --convergenceDelta=[0.5], --distanceMeasure=[org.apache.mahout.common.distance.CosineDistanceMeasure], --endPhase=[2147483647], --input=[/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors/], --maxIter=[10], --method=[mapreduce], --numClusters=[20], --output=[/tmp/mahout-work-hudson/reuters-kmeans], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
12/04/02 19:33:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
12/04/02 19:33:28 INFO compress.CodecPool: Got brand-new compressor
12/04/02 19:33:29 INFO kmeans.RandomSeedGenerator: Wrote 20 vectors to /tmp/mahout-work-hudson/reuters-kmeans-clusters/part-randomSeed
12/04/02 19:33:29 INFO kmeans.KMeansDriver: Input: /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors Clusters In: /tmp/mahout-work-hudson/reuters-kmeans-clusters/part-randomSeed Out: /tmp/mahout-work-hudson/reuters-kmeans Distance: org.apache.mahout.common.distance.CosineDistanceMeasure
12/04/02 19:33:29 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10 num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}
12/04/02 19:33:29 INFO compress.CodecPool: Got brand-new decompressor
Exception in thread "main" java.io.IOException: Mkdirs failed to create /tmp/mahout-work-hudson/reuters-kmeans-clusters/part-randomSeed/clusters-0
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:366)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:528)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:843)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:831)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:823)
at org.apache.mahout.clustering.classify.ClusterClassifier.writePolicy(ClusterClassifier.java:232)
at org.apache.mahout.clustering.classify.ClusterClassifier.writeToSeqFiles(ClusterClassifier.java:185)
at org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:254)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:154)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:104)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.mahout.clustering.kmeans.KMeansDriver.main(KMeansDriver.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: Mahout-Examples-Cluster-Reuters #97
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/97/>
------------------------------------------
[...truncated 6192 lines...]
12/04/09 19:18:24 INFO mapred.JobClient: Reduce input records=21578
12/04/09 19:18:24 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/09 19:18:24 INFO input.FileInputFormat: Total input paths to process : 1
12/04/09 19:18:24 INFO mapred.JobClient: Running job: job_local_0004
12/04/09 19:18:24 INFO mapred.MapTask: io.sort.mb = 100
12/04/09 19:18:24 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/09 19:18:24 INFO mapred.MapTask: record buffer = 262144/327680
12/04/09 19:18:25 INFO mapred.JobClient: map 0% reduce 0%
12/04/09 19:18:26 INFO mapred.MapTask: Starting flush of map output
12/04/09 19:18:26 INFO mapred.MapTask: Finished spill 0
12/04/09 19:18:26 INFO mapred.Task: Task:attempt_local_0004_m_000000_0 is done. And is in the process of commiting
12/04/09 19:18:27 INFO mapred.LocalJobRunner:
12/04/09 19:18:27 INFO mapred.Task: Task 'attempt_local_0004_m_000000_0' done.
12/04/09 19:18:27 INFO mapred.LocalJobRunner:
12/04/09 19:18:27 INFO mapred.Merger: Merging 1 sorted segments
12/04/09 19:18:27 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/09 19:18:27 INFO mapred.LocalJobRunner:
12/04/09 19:18:28 INFO mapred.JobClient: map 100% reduce 0%
12/04/09 19:18:28 INFO mapred.Task: Task:attempt_local_0004_r_000000_0 is done. And is in the process of commiting
12/04/09 19:18:28 INFO mapred.LocalJobRunner:
12/04/09 19:18:28 INFO mapred.Task: Task attempt_local_0004_r_000000_0 is allowed to commit now
12/04/09 19:18:28 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0004_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/09 19:18:30 INFO mapred.LocalJobRunner: reduce > reduce
12/04/09 19:18:30 INFO mapred.Task: Task 'attempt_local_0004_r_000000_0' done.
12/04/09 19:18:31 INFO mapred.JobClient: map 100% reduce 100%
12/04/09 19:18:31 INFO mapred.JobClient: Job complete: job_local_0004
12/04/09 19:18:31 INFO mapred.JobClient: Counters: 16
12/04/09 19:18:31 INFO mapred.JobClient: File Output Format Counters
12/04/09 19:18:31 INFO mapred.JobClient: Bytes Written=17780621
12/04/09 19:18:31 INFO mapred.JobClient: FileSystemCounters
12/04/09 19:18:31 INFO mapred.JobClient: FILE_BYTES_READ=392453306
12/04/09 19:18:31 INFO mapred.JobClient: FILE_BYTES_WRITTEN=361932633
12/04/09 19:18:31 INFO mapred.JobClient: File Input Format Counters
12/04/09 19:18:31 INFO mapred.JobClient: Bytes Read=17780621
12/04/09 19:18:31 INFO mapred.JobClient: Map-Reduce Framework
12/04/09 19:18:31 INFO mapred.JobClient: Reduce input groups=21578
12/04/09 19:18:31 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/09 19:18:31 INFO mapred.JobClient: Combine output records=0
12/04/09 19:18:31 INFO mapred.JobClient: Map input records=21578
12/04/09 19:18:31 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/09 19:18:31 INFO mapred.JobClient: Reduce output records=21578
12/04/09 19:18:31 INFO mapred.JobClient: Spilled Records=43156
12/04/09 19:18:31 INFO mapred.JobClient: Map output bytes=17337483
12/04/09 19:18:31 INFO mapred.JobClient: Combine input records=0
12/04/09 19:18:31 INFO mapred.JobClient: Map output records=21578
12/04/09 19:18:31 INFO mapred.JobClient: SPLIT_RAW_BYTES=158
12/04/09 19:18:31 INFO mapred.JobClient: Reduce input records=21578
12/04/09 19:18:31 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/09 19:18:31 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/df-count
12/04/09 19:18:32 INFO input.FileInputFormat: Total input paths to process : 1
12/04/09 19:18:32 INFO mapred.JobClient: Running job: job_local_0005
12/04/09 19:18:32 INFO mapred.MapTask: io.sort.mb = 100
12/04/09 19:18:32 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/09 19:18:32 INFO mapred.MapTask: record buffer = 262144/327680
12/04/09 19:18:32 INFO mapred.MapTask: Spilling map output: record full = true
12/04/09 19:18:32 INFO mapred.MapTask: bufstart = 0; bufend = 3145728; bufvoid = 99614720
12/04/09 19:18:32 INFO mapred.MapTask: kvstart = 0; kvend = 262144; length = 327680
12/04/09 19:18:32 INFO mapred.MapTask: Finished spill 0
12/04/09 19:18:33 INFO mapred.MapTask: Spilling map output: record full = true
12/04/09 19:18:33 INFO mapred.MapTask: bufstart = 3145728; bufend = 6291444; bufvoid = 99614720
12/04/09 19:18:33 INFO mapred.MapTask: kvstart = 262144; kvend = 196607; length = 327680
12/04/09 19:18:33 INFO mapred.JobClient: map 0% reduce 0%
12/04/09 19:18:33 INFO mapred.MapTask: Finished spill 1
12/04/09 19:18:33 INFO mapred.MapTask: Spilling map output: record full = true
12/04/09 19:18:33 INFO mapred.MapTask: bufstart = 6291444; bufend = 9437160; bufvoid = 99614720
12/04/09 19:18:33 INFO mapred.MapTask: kvstart = 196607; kvend = 131070; length = 327680
12/04/09 19:18:33 INFO mapred.MapTask: Finished spill 2
12/04/09 19:18:34 INFO mapred.MapTask: Spilling map output: record full = true
12/04/09 19:18:34 INFO mapred.MapTask: bufstart = 9437160; bufend = 12582876; bufvoid = 99614720
12/04/09 19:18:34 INFO mapred.MapTask: kvstart = 131070; kvend = 65533; length = 327680
12/04/09 19:18:34 INFO mapred.MapTask: Finished spill 3
12/04/09 19:18:34 INFO mapred.MapTask: Spilling map output: record full = true
12/04/09 19:18:34 INFO mapred.MapTask: bufstart = 12582876; bufend = 15728604; bufvoid = 99614720
12/04/09 19:18:34 INFO mapred.MapTask: kvstart = 65533; kvend = 327677; length = 327680
12/04/09 19:18:34 INFO mapred.MapTask: Finished spill 4
12/04/09 19:18:35 INFO mapred.MapTask: Starting flush of map output
12/04/09 19:18:35 INFO mapred.MapTask: Finished spill 5
12/04/09 19:18:35 INFO mapred.Merger: Merging 6 sorted segments
12/04/09 19:18:35 INFO mapred.Merger: Down to the last merge-pass, with 6 segments left of total size: 1871742 bytes
12/04/09 19:18:35 INFO mapred.Task: Task:attempt_local_0005_m_000000_0 is done. And is in the process of commiting
12/04/09 19:18:38 INFO mapred.LocalJobRunner:
12/04/09 19:18:38 INFO mapred.JobClient: map 100% reduce 0%
12/04/09 19:18:38 INFO mapred.LocalJobRunner:
12/04/09 19:18:38 INFO mapred.Task: Task 'attempt_local_0005_m_000000_0' done.
12/04/09 19:18:38 INFO mapred.LocalJobRunner:
12/04/09 19:18:38 INFO mapred.Merger: Merging 1 sorted segments
12/04/09 19:18:38 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 585314 bytes
12/04/09 19:18:38 INFO mapred.LocalJobRunner:
12/04/09 19:18:38 INFO mapred.Task: Task:attempt_local_0005_r_000000_0 is done. And is in the process of commiting
12/04/09 19:18:38 INFO mapred.LocalJobRunner:
12/04/09 19:18:38 INFO mapred.Task: Task attempt_local_0005_r_000000_0 is allowed to commit now
12/04/09 19:18:38 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0005_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/df-count
12/04/09 19:18:41 INFO mapred.LocalJobRunner: reduce > reduce
12/04/09 19:18:41 INFO mapred.Task: Task 'attempt_local_0005_r_000000_0' done.
12/04/09 19:18:42 INFO mapred.JobClient: map 100% reduce 100%
12/04/09 19:18:42 INFO mapred.JobClient: Job complete: job_local_0005
12/04/09 19:18:42 INFO mapred.JobClient: Counters: 16
12/04/09 19:18:42 INFO mapred.JobClient: File Output Format Counters
12/04/09 19:18:42 INFO mapred.JobClient: Bytes Written=851221
12/04/09 19:18:42 INFO mapred.JobClient: FileSystemCounters
12/04/09 19:18:42 INFO mapred.JobClient: FILE_BYTES_READ=498036904
12/04/09 19:18:42 INFO mapred.JobClient: FILE_BYTES_WRITTEN=434194181
12/04/09 19:18:42 INFO mapred.JobClient: File Input Format Counters
12/04/09 19:18:42 INFO mapred.JobClient: Bytes Read=17780621
12/04/09 19:18:42 INFO mapred.JobClient: Map-Reduce Framework
12/04/09 19:18:42 INFO mapred.JobClient: Reduce input groups=41808
12/04/09 19:18:42 INFO mapred.JobClient: Map output materialized bytes=585318
12/04/09 19:18:42 INFO mapred.JobClient: Combine output records=175503
12/04/09 19:18:42 INFO mapred.JobClient: Map input records=21578
12/04/09 19:18:42 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/09 19:18:42 INFO mapred.JobClient: Reduce output records=41808
12/04/09 19:18:42 INFO mapred.JobClient: Spilled Records=217311
12/04/09 19:18:42 INFO mapred.JobClient: Map output bytes=18371184
12/04/09 19:18:42 INFO mapred.JobClient: Combine input records=1664627
12/04/09 19:18:42 INFO mapred.JobClient: Map output records=1530932
12/04/09 19:18:42 INFO mapred.JobClient: SPLIT_RAW_BYTES=151
12/04/09 19:18:42 INFO mapred.JobClient: Reduce input records=41808
12/04/09 19:18:42 INFO input.FileInputFormat: Total input paths to process : 1
12/04/09 19:18:42 INFO filecache.TrackerDistributedCacheManager: Creating frequency.file-0 in /tmp/hadoop-jenkins/mapred/local/archive/-5380933297960546750_-299119410_411777213/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans-work-7352206724486129918 with rwxr-xr-x
12/04/09 19:18:42 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-jenkins/mapred/local/archive/-5380933297960546750_-299119410_411777213/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/09 19:18:42 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-jenkins/mapred/local/archive/-5380933297960546750_-299119410_411777213/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/09 19:18:42 INFO mapred.JobClient: Running job: job_local_0006
12/04/09 19:18:42 INFO mapred.MapTask: io.sort.mb = 100
12/04/09 19:18:42 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/09 19:18:42 INFO mapred.MapTask: record buffer = 262144/327680
12/04/09 19:18:43 INFO mapred.JobClient: map 0% reduce 0%
12/04/09 19:18:43 INFO mapred.MapTask: Starting flush of map output
12/04/09 19:18:43 INFO mapred.MapTask: Finished spill 0
12/04/09 19:18:43 INFO mapred.Task: Task:attempt_local_0006_m_000000_0 is done. And is in the process of commiting
12/04/09 19:18:45 INFO mapred.LocalJobRunner:
12/04/09 19:18:45 INFO mapred.Task: Task 'attempt_local_0006_m_000000_0' done.
12/04/09 19:18:45 INFO mapred.LocalJobRunner:
12/04/09 19:18:45 INFO mapred.Merger: Merging 1 sorted segments
12/04/09 19:18:45 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/09 19:18:45 INFO mapred.LocalJobRunner:
12/04/09 19:18:46 INFO mapred.JobClient: map 100% reduce 0%
12/04/09 19:18:46 INFO mapred.Task: Task:attempt_local_0006_r_000000_0 is done. And is in the process of commiting
12/04/09 19:18:46 INFO mapred.LocalJobRunner:
12/04/09 19:18:46 INFO mapred.Task: Task attempt_local_0006_r_000000_0 is allowed to commit now
12/04/09 19:18:46 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0006_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/09 19:18:48 INFO mapred.LocalJobRunner: reduce > reduce
12/04/09 19:18:48 INFO mapred.Task: Task 'attempt_local_0006_r_000000_0' done.
12/04/09 19:18:49 INFO mapred.JobClient: map 100% reduce 100%
12/04/09 19:18:49 INFO mapred.JobClient: Job complete: job_local_0006
12/04/09 19:18:49 INFO mapred.JobClient: Counters: 16
12/04/09 19:18:49 INFO mapred.JobClient: File Output Format Counters
12/04/09 19:18:49 INFO mapred.JobClient: Bytes Written=17156391
12/04/09 19:18:49 INFO mapred.JobClient: FileSystemCounters
12/04/09 19:18:49 INFO mapred.JobClient: FILE_BYTES_READ=604133015
12/04/09 19:18:49 INFO mapred.JobClient: FILE_BYTES_WRITTEN=539166423
12/04/09 19:18:49 INFO mapred.JobClient: File Input Format Counters
12/04/09 19:18:49 INFO mapred.JobClient: Bytes Read=17780621
12/04/09 19:18:49 INFO mapred.JobClient: Map-Reduce Framework
12/04/09 19:18:49 INFO mapred.JobClient: Reduce input groups=21578
12/04/09 19:18:49 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/09 19:18:49 INFO mapred.JobClient: Combine output records=0
12/04/09 19:18:49 INFO mapred.JobClient: Map input records=21578
12/04/09 19:18:49 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/09 19:18:49 INFO mapred.JobClient: Reduce output records=21578
12/04/09 19:18:49 INFO mapred.JobClient: Spilled Records=43156
12/04/09 19:18:49 INFO mapred.JobClient: Map output bytes=17337483
12/04/09 19:18:49 INFO mapred.JobClient: Combine input records=0
12/04/09 19:18:49 INFO mapred.JobClient: Map output records=21578
12/04/09 19:18:49 INFO mapred.JobClient: SPLIT_RAW_BYTES=151
12/04/09 19:18:49 INFO mapred.JobClient: Reduce input records=21578
12/04/09 19:18:49 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/09 19:18:49 INFO input.FileInputFormat: Total input paths to process : 1
12/04/09 19:18:49 INFO mapred.JobClient: Running job: job_local_0007
12/04/09 19:18:49 INFO mapred.MapTask: io.sort.mb = 100
12/04/09 19:18:49 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/09 19:18:49 INFO mapred.MapTask: record buffer = 262144/327680
12/04/09 19:18:50 INFO mapred.MapTask: Starting flush of map output
12/04/09 19:18:50 INFO mapred.JobClient: map 0% reduce 0%
12/04/09 19:18:50 INFO mapred.MapTask: Finished spill 0
12/04/09 19:18:50 INFO mapred.Task: Task:attempt_local_0007_m_000000_0 is done. And is in the process of commiting
12/04/09 19:18:52 INFO mapred.LocalJobRunner:
12/04/09 19:18:52 INFO mapred.Task: Task 'attempt_local_0007_m_000000_0' done.
12/04/09 19:18:52 INFO mapred.LocalJobRunner:
12/04/09 19:18:52 INFO mapred.Merger: Merging 1 sorted segments
12/04/09 19:18:52 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 16804860 bytes
12/04/09 19:18:52 INFO mapred.LocalJobRunner:
12/04/09 19:18:53 INFO mapred.Task: Task:attempt_local_0007_r_000000_0 is done. And is in the process of commiting
12/04/09 19:18:53 INFO mapred.LocalJobRunner:
12/04/09 19:18:53 INFO mapred.Task: Task attempt_local_0007_r_000000_0 is allowed to commit now
12/04/09 19:18:53 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0007_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/09 19:18:53 INFO mapred.JobClient: map 100% reduce 0%
12/04/09 19:18:55 INFO mapred.LocalJobRunner: reduce > reduce
12/04/09 19:18:55 INFO mapred.Task: Task 'attempt_local_0007_r_000000_0' done.
12/04/09 19:18:56 INFO mapred.JobClient: map 100% reduce 100%
12/04/09 19:18:56 INFO mapred.JobClient: Job complete: job_local_0007
12/04/09 19:18:56 INFO mapred.JobClient: Counters: 16
12/04/09 19:18:56 INFO mapred.JobClient: File Output Format Counters
12/04/09 19:18:56 INFO mapred.JobClient: Bytes Written=17156391
12/04/09 19:18:56 INFO mapred.JobClient: FileSystemCounters
12/04/09 19:18:56 INFO mapred.JobClient: FILE_BYTES_READ=721795382
12/04/09 19:18:56 INFO mapred.JobClient: FILE_BYTES_WRITTEN=655805017
12/04/09 19:18:56 INFO mapred.JobClient: File Input Format Counters
12/04/09 19:18:56 INFO mapred.JobClient: Bytes Read=17156391
12/04/09 19:18:56 INFO mapred.JobClient: Map-Reduce Framework
12/04/09 19:18:56 INFO mapred.JobClient: Reduce input groups=21578
12/04/09 19:18:56 INFO mapred.JobClient: Map output materialized bytes=16804864
12/04/09 19:18:56 INFO mapred.JobClient: Combine output records=0
12/04/09 19:18:56 INFO mapred.JobClient: Map input records=21578
12/04/09 19:18:56 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/09 19:18:56 INFO mapred.JobClient: Reduce output records=21578
12/04/09 19:18:56 INFO mapred.JobClient: Spilled Records=43156
12/04/09 19:18:56 INFO mapred.JobClient: Map output bytes=16722273
12/04/09 19:18:56 INFO mapred.JobClient: Combine input records=0
12/04/09 19:18:56 INFO mapred.JobClient: Map output records=21578
12/04/09 19:18:56 INFO mapred.JobClient: SPLIT_RAW_BYTES=158
12/04/09 19:18:56 INFO mapred.JobClient: Reduce input records=21578
12/04/09 19:18:56 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/09 19:18:56 INFO driver.MahoutDriver: Program took 79139 ms (Minutes: 1.3189833333333334)
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
no HADOOP_HOME set, running locally
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/mahout-examples-0.7-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-jcl-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
12/04/09 19:18:57 INFO common.AbstractJob: Command line arguments: {--clustering=null, --clusters=[/tmp/mahout-work-jenkins/reuters-kmeans-clusters], --convergenceDelta=[0.5], --distanceMeasure=[org.apache.mahout.common.distance.CosineDistanceMeasure], --endPhase=[2147483647], --input=[/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors/], --maxIter=[10], --method=[mapreduce], --numClusters=[20], --output=[/tmp/mahout-work-jenkins/reuters-kmeans], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
12/04/09 19:18:57 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-kmeans-clusters
12/04/09 19:18:57 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
12/04/09 19:18:57 INFO compress.CodecPool: Got brand-new compressor
12/04/09 19:18:59 INFO kmeans.RandomSeedGenerator: Wrote 20 vectors to /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed
12/04/09 19:18:59 INFO kmeans.KMeansDriver: Input: /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors Clusters In: /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed Out: /tmp/mahout-work-jenkins/reuters-kmeans Distance: org.apache.mahout.common.distance.CosineDistanceMeasure
12/04/09 19:18:59 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10 num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}
12/04/09 19:18:59 INFO compress.CodecPool: Got brand-new decompressor
Exception in thread "main" java.io.IOException: Mkdirs failed to create /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed/clusters-0
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:366)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:528)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:843)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:831)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:823)
at org.apache.mahout.clustering.classify.ClusterClassifier.writePolicy(ClusterClassifier.java:232)
at org.apache.mahout.clustering.classify.ClusterClassifier.writeToSeqFiles(ClusterClassifier.java:185)
at org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:254)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:154)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:104)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.mahout.clustering.kmeans.KMeansDriver.main(KMeansDriver.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)
Build step 'Execute shell' marked build as failure
Re: Actual bug (I think) was Re: Build failed in Jenkins: Mahout-Examples-Cluster-Reuters #96
Posted by Grant Ingersoll <gs...@apache.org>.
https://issues.apache.org/jira/browse/MAHOUT-999
On Apr 9, 2012, at 8:35 AM, Grant Ingersoll wrote:
> This appears to be a real, bona-fide bug. I can repro it on my local machine.
>
> Seems that /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed/ is actually just a file, not a directory, so somewhere writing a policy to that area doesn't make sense.
>
>
> On Apr 8, 2012, at 3:19 PM, Apache Jenkins Server wrote:
>
>> See <https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/96/>
>>
>> 12/04/08 19:19:16 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-kmeans-clusters
>> 12/04/08 19:19:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
>> 12/04/08 19:19:16 INFO compress.CodecPool: Got brand-new compressor
>> 12/04/08 19:19:18 INFO kmeans.RandomSeedGenerator: Wrote 20 vectors to /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed
>> 12/04/08 19:19:18 INFO kmeans.KMeansDriver: Input: /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors Clusters In: /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed Out: /tmp/mahout-work-jenkins/reuters-kmeans Distance: org.apache.mahout.common.distance.CosineDistanceMeasure
>> 12/04/08 19:19:18 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10 num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}
>> 12/04/08 19:19:18 INFO compress.CodecPool: Got brand-new decompressor
>> Exception in thread "main" java.io.IOException: Mkdirs failed to create /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed/clusters-0
>> at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:366)
>> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:528)
>> at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:843)
>> at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:831)
>> at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:823)
>> at org.apache.mahout.clustering.classify.ClusterClassifier.writePolicy(ClusterClassifier.java:232)
>> at org.apache.mahout.clustering.classify.ClusterClassifier.writeToSeqFiles(ClusterClassifier.java:185)
>> at org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:254)
>> at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:154)
>> at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:104)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.apache.mahout.clustering.kmeans.KMeansDriver.main(KMeansDriver.java:48)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>> at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)
>> Build step 'Execute shell' marked build as failure
>
>
--------------------------------------------
Grant Ingersoll
http://www.lucidimagination.com
Actual bug (I think) was Re: Build failed in Jenkins: Mahout-Examples-Cluster-Reuters #96
Posted by Grant Ingersoll <gs...@apache.org>.
This appears to be a real, bona-fide bug. I can repro it on my local machine.
Seems that /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed/ is actually just a file, not a directory, so somewhere writing a policy to that area doesn't make sense.
On Apr 8, 2012, at 3:19 PM, Apache Jenkins Server wrote:
> See <https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/96/>
>
> 12/04/08 19:19:16 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-kmeans-clusters
> 12/04/08 19:19:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> 12/04/08 19:19:16 INFO compress.CodecPool: Got brand-new compressor
> 12/04/08 19:19:18 INFO kmeans.RandomSeedGenerator: Wrote 20 vectors to /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed
> 12/04/08 19:19:18 INFO kmeans.KMeansDriver: Input: /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors Clusters In: /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed Out: /tmp/mahout-work-jenkins/reuters-kmeans Distance: org.apache.mahout.common.distance.CosineDistanceMeasure
> 12/04/08 19:19:18 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10 num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}
> 12/04/08 19:19:18 INFO compress.CodecPool: Got brand-new decompressor
> Exception in thread "main" java.io.IOException: Mkdirs failed to create /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed/clusters-0
> at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:366)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:528)
> at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:843)
> at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:831)
> at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:823)
> at org.apache.mahout.clustering.classify.ClusterClassifier.writePolicy(ClusterClassifier.java:232)
> at org.apache.mahout.clustering.classify.ClusterClassifier.writeToSeqFiles(ClusterClassifier.java:185)
> at org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:254)
> at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:154)
> at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:104)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.mahout.clustering.kmeans.KMeansDriver.main(KMeansDriver.java:48)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
> at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)
> Build step 'Execute shell' marked build as failure
Build failed in Jenkins: Mahout-Examples-Cluster-Reuters #96
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/96/>
------------------------------------------
[...truncated 6192 lines...]
12/04/08 19:18:46 INFO mapred.JobClient: SPLIT_RAW_BYTES=640
12/04/08 19:18:46 INFO mapred.JobClient: Reduce input records=21578
12/04/08 19:18:46 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/08 19:18:46 INFO input.FileInputFormat: Total input paths to process : 1
12/04/08 19:18:46 INFO mapred.JobClient: Running job: job_local_0004
12/04/08 19:18:46 INFO mapred.MapTask: io.sort.mb = 100
12/04/08 19:18:46 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/08 19:18:46 INFO mapred.MapTask: record buffer = 262144/327680
12/04/08 19:18:47 INFO mapred.JobClient: map 0% reduce 0%
12/04/08 19:18:47 INFO mapred.MapTask: Starting flush of map output
12/04/08 19:18:47 INFO mapred.MapTask: Finished spill 0
12/04/08 19:18:47 INFO mapred.Task: Task:attempt_local_0004_m_000000_0 is done. And is in the process of commiting
12/04/08 19:18:49 INFO mapred.LocalJobRunner:
12/04/08 19:18:49 INFO mapred.Task: Task 'attempt_local_0004_m_000000_0' done.
12/04/08 19:18:49 INFO mapred.LocalJobRunner:
12/04/08 19:18:49 INFO mapred.Merger: Merging 1 sorted segments
12/04/08 19:18:49 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/08 19:18:49 INFO mapred.LocalJobRunner:
12/04/08 19:18:50 INFO mapred.JobClient: map 100% reduce 0%
12/04/08 19:18:50 INFO mapred.Task: Task:attempt_local_0004_r_000000_0 is done. And is in the process of commiting
12/04/08 19:18:50 INFO mapred.LocalJobRunner:
12/04/08 19:18:50 INFO mapred.Task: Task attempt_local_0004_r_000000_0 is allowed to commit now
12/04/08 19:18:50 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0004_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/08 19:18:52 INFO mapred.LocalJobRunner: reduce > reduce
12/04/08 19:18:52 INFO mapred.Task: Task 'attempt_local_0004_r_000000_0' done.
12/04/08 19:18:53 INFO mapred.JobClient: map 100% reduce 100%
12/04/08 19:18:53 INFO mapred.JobClient: Job complete: job_local_0004
12/04/08 19:18:53 INFO mapred.JobClient: Counters: 16
12/04/08 19:18:53 INFO mapred.JobClient: File Output Format Counters
12/04/08 19:18:53 INFO mapred.JobClient: Bytes Written=17780621
12/04/08 19:18:53 INFO mapred.JobClient: FileSystemCounters
12/04/08 19:18:53 INFO mapred.JobClient: FILE_BYTES_READ=392453306
12/04/08 19:18:53 INFO mapred.JobClient: FILE_BYTES_WRITTEN=361932645
12/04/08 19:18:53 INFO mapred.JobClient: File Input Format Counters
12/04/08 19:18:53 INFO mapred.JobClient: Bytes Read=17780621
12/04/08 19:18:53 INFO mapred.JobClient: Map-Reduce Framework
12/04/08 19:18:53 INFO mapred.JobClient: Reduce input groups=21578
12/04/08 19:18:53 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/08 19:18:53 INFO mapred.JobClient: Combine output records=0
12/04/08 19:18:53 INFO mapred.JobClient: Map input records=21578
12/04/08 19:18:53 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/08 19:18:53 INFO mapred.JobClient: Reduce output records=21578
12/04/08 19:18:53 INFO mapred.JobClient: Spilled Records=43156
12/04/08 19:18:53 INFO mapred.JobClient: Map output bytes=17337483
12/04/08 19:18:53 INFO mapred.JobClient: Combine input records=0
12/04/08 19:18:53 INFO mapred.JobClient: Map output records=21578
12/04/08 19:18:53 INFO mapred.JobClient: SPLIT_RAW_BYTES=158
12/04/08 19:18:53 INFO mapred.JobClient: Reduce input records=21578
12/04/08 19:18:53 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/08 19:18:53 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/df-count
12/04/08 19:18:53 INFO input.FileInputFormat: Total input paths to process : 1
12/04/08 19:18:53 INFO mapred.JobClient: Running job: job_local_0005
12/04/08 19:18:53 INFO mapred.MapTask: io.sort.mb = 100
12/04/08 19:18:53 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/08 19:18:53 INFO mapred.MapTask: record buffer = 262144/327680
12/04/08 19:18:54 INFO mapred.MapTask: Spilling map output: record full = true
12/04/08 19:18:54 INFO mapred.MapTask: bufstart = 0; bufend = 3145728; bufvoid = 99614720
12/04/08 19:18:54 INFO mapred.MapTask: kvstart = 0; kvend = 262144; length = 327680
12/04/08 19:18:54 INFO mapred.MapTask: Finished spill 0
12/04/08 19:18:54 INFO mapred.MapTask: Spilling map output: record full = true
12/04/08 19:18:54 INFO mapred.MapTask: bufstart = 3145728; bufend = 6291444; bufvoid = 99614720
12/04/08 19:18:54 INFO mapred.MapTask: kvstart = 262144; kvend = 196607; length = 327680
12/04/08 19:18:54 INFO mapred.JobClient: map 0% reduce 0%
12/04/08 19:18:54 INFO mapred.MapTask: Finished spill 1
12/04/08 19:18:55 INFO mapred.MapTask: Spilling map output: record full = true
12/04/08 19:18:55 INFO mapred.MapTask: bufstart = 6291444; bufend = 9437160; bufvoid = 99614720
12/04/08 19:18:55 INFO mapred.MapTask: kvstart = 196607; kvend = 131070; length = 327680
12/04/08 19:18:55 INFO mapred.MapTask: Finished spill 2
12/04/08 19:18:55 INFO mapred.MapTask: Spilling map output: record full = true
12/04/08 19:18:55 INFO mapred.MapTask: bufstart = 9437160; bufend = 12582876; bufvoid = 99614720
12/04/08 19:18:55 INFO mapred.MapTask: kvstart = 131070; kvend = 65533; length = 327680
12/04/08 19:18:55 INFO mapred.MapTask: Finished spill 3
12/04/08 19:18:56 INFO mapred.MapTask: Spilling map output: record full = true
12/04/08 19:18:56 INFO mapred.MapTask: bufstart = 12582876; bufend = 15728604; bufvoid = 99614720
12/04/08 19:18:56 INFO mapred.MapTask: kvstart = 65533; kvend = 327677; length = 327680
12/04/08 19:18:56 INFO mapred.MapTask: Finished spill 4
12/04/08 19:18:56 INFO mapred.MapTask: Starting flush of map output
12/04/08 19:18:56 INFO mapred.MapTask: Finished spill 5
12/04/08 19:18:56 INFO mapred.Merger: Merging 6 sorted segments
12/04/08 19:18:56 INFO mapred.Merger: Down to the last merge-pass, with 6 segments left of total size: 1871742 bytes
12/04/08 19:18:56 INFO mapred.Task: Task:attempt_local_0005_m_000000_0 is done. And is in the process of commiting
12/04/08 19:18:56 INFO mapred.LocalJobRunner:
12/04/08 19:18:56 INFO mapred.Task: Task 'attempt_local_0005_m_000000_0' done.
12/04/08 19:18:56 INFO mapred.LocalJobRunner:
12/04/08 19:18:56 INFO mapred.Merger: Merging 1 sorted segments
12/04/08 19:18:56 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 585314 bytes
12/04/08 19:18:56 INFO mapred.LocalJobRunner:
12/04/08 19:18:56 INFO mapred.Task: Task:attempt_local_0005_r_000000_0 is done. And is in the process of commiting
12/04/08 19:18:56 INFO mapred.LocalJobRunner:
12/04/08 19:18:56 INFO mapred.Task: Task attempt_local_0005_r_000000_0 is allowed to commit now
12/04/08 19:18:56 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0005_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/df-count
12/04/08 19:18:57 INFO mapred.JobClient: map 100% reduce 0%
12/04/08 19:18:59 INFO mapred.LocalJobRunner: reduce > reduce
12/04/08 19:18:59 INFO mapred.Task: Task 'attempt_local_0005_r_000000_0' done.
12/04/08 19:19:00 INFO mapred.JobClient: map 100% reduce 100%
12/04/08 19:19:00 INFO mapred.JobClient: Job complete: job_local_0005
12/04/08 19:19:00 INFO mapred.JobClient: Counters: 16
12/04/08 19:19:00 INFO mapred.JobClient: File Output Format Counters
12/04/08 19:19:00 INFO mapred.JobClient: Bytes Written=851221
12/04/08 19:19:00 INFO mapred.JobClient: FileSystemCounters
12/04/08 19:19:00 INFO mapred.JobClient: FILE_BYTES_READ=498036904
12/04/08 19:19:00 INFO mapred.JobClient: FILE_BYTES_WRITTEN=434194185
12/04/08 19:19:00 INFO mapred.JobClient: File Input Format Counters
12/04/08 19:19:00 INFO mapred.JobClient: Bytes Read=17780621
12/04/08 19:19:00 INFO mapred.JobClient: Map-Reduce Framework
12/04/08 19:19:00 INFO mapred.JobClient: Reduce input groups=41808
12/04/08 19:19:00 INFO mapred.JobClient: Map output materialized bytes=585318
12/04/08 19:19:00 INFO mapred.JobClient: Combine output records=175503
12/04/08 19:19:00 INFO mapred.JobClient: Map input records=21578
12/04/08 19:19:00 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/08 19:19:00 INFO mapred.JobClient: Reduce output records=41808
12/04/08 19:19:00 INFO mapred.JobClient: Spilled Records=217311
12/04/08 19:19:00 INFO mapred.JobClient: Map output bytes=18371184
12/04/08 19:19:00 INFO mapred.JobClient: Combine input records=1664627
12/04/08 19:19:00 INFO mapred.JobClient: Map output records=1530932
12/04/08 19:19:00 INFO mapred.JobClient: SPLIT_RAW_BYTES=151
12/04/08 19:19:00 INFO mapred.JobClient: Reduce input records=41808
12/04/08 19:19:01 INFO input.FileInputFormat: Total input paths to process : 1
12/04/08 19:19:01 INFO filecache.TrackerDistributedCacheManager: Creating frequency.file-0 in /tmp/hadoop-jenkins/mapred/local/archive/3516673424516957443_-299119410_325396213/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans-work--7741270144794027612 with rwxr-xr-x
12/04/08 19:19:01 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-jenkins/mapred/local/archive/3516673424516957443_-299119410_325396213/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/08 19:19:01 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-jenkins/mapred/local/archive/3516673424516957443_-299119410_325396213/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/08 19:19:01 INFO mapred.JobClient: Running job: job_local_0006
12/04/08 19:19:01 INFO mapred.MapTask: io.sort.mb = 100
12/04/08 19:19:01 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/08 19:19:01 INFO mapred.MapTask: record buffer = 262144/327680
12/04/08 19:19:02 INFO mapred.JobClient: map 0% reduce 0%
12/04/08 19:19:02 INFO mapred.MapTask: Starting flush of map output
12/04/08 19:19:02 INFO mapred.MapTask: Finished spill 0
12/04/08 19:19:02 INFO mapred.Task: Task:attempt_local_0006_m_000000_0 is done. And is in the process of commiting
12/04/08 19:19:04 INFO mapred.LocalJobRunner:
12/04/08 19:19:04 INFO mapred.Task: Task 'attempt_local_0006_m_000000_0' done.
12/04/08 19:19:04 INFO mapred.LocalJobRunner:
12/04/08 19:19:04 INFO mapred.Merger: Merging 1 sorted segments
12/04/08 19:19:04 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/08 19:19:04 INFO mapred.LocalJobRunner:
12/04/08 19:19:05 INFO mapred.JobClient: map 100% reduce 0%
12/04/08 19:19:05 INFO mapred.Task: Task:attempt_local_0006_r_000000_0 is done. And is in the process of commiting
12/04/08 19:19:05 INFO mapred.LocalJobRunner:
12/04/08 19:19:05 INFO mapred.Task: Task attempt_local_0006_r_000000_0 is allowed to commit now
12/04/08 19:19:05 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0006_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/08 19:19:07 INFO mapred.LocalJobRunner: reduce > reduce
12/04/08 19:19:07 INFO mapred.Task: Task 'attempt_local_0006_r_000000_0' done.
12/04/08 19:19:08 INFO mapred.JobClient: map 100% reduce 100%
12/04/08 19:19:08 INFO mapred.JobClient: Job complete: job_local_0006
12/04/08 19:19:08 INFO mapred.JobClient: Counters: 16
12/04/08 19:19:08 INFO mapred.JobClient: File Output Format Counters
12/04/08 19:19:08 INFO mapred.JobClient: Bytes Written=17156391
12/04/08 19:19:08 INFO mapred.JobClient: FileSystemCounters
12/04/08 19:19:08 INFO mapred.JobClient: FILE_BYTES_READ=604133015
12/04/08 19:19:08 INFO mapred.JobClient: FILE_BYTES_WRITTEN=539166433
12/04/08 19:19:08 INFO mapred.JobClient: File Input Format Counters
12/04/08 19:19:08 INFO mapred.JobClient: Bytes Read=17780621
12/04/08 19:19:08 INFO mapred.JobClient: Map-Reduce Framework
12/04/08 19:19:08 INFO mapred.JobClient: Reduce input groups=21578
12/04/08 19:19:08 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/08 19:19:08 INFO mapred.JobClient: Combine output records=0
12/04/08 19:19:08 INFO mapred.JobClient: Map input records=21578
12/04/08 19:19:08 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/08 19:19:08 INFO mapred.JobClient: Reduce output records=21578
12/04/08 19:19:08 INFO mapred.JobClient: Spilled Records=43156
12/04/08 19:19:08 INFO mapred.JobClient: Map output bytes=17337483
12/04/08 19:19:08 INFO mapred.JobClient: Combine input records=0
12/04/08 19:19:08 INFO mapred.JobClient: Map output records=21578
12/04/08 19:19:08 INFO mapred.JobClient: SPLIT_RAW_BYTES=151
12/04/08 19:19:08 INFO mapred.JobClient: Reduce input records=21578
12/04/08 19:19:08 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/08 19:19:08 INFO input.FileInputFormat: Total input paths to process : 1
12/04/08 19:19:08 INFO mapred.JobClient: Running job: job_local_0007
12/04/08 19:19:08 INFO mapred.MapTask: io.sort.mb = 100
12/04/08 19:19:08 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/08 19:19:08 INFO mapred.MapTask: record buffer = 262144/327680
12/04/08 19:19:09 INFO mapred.MapTask: Starting flush of map output
12/04/08 19:19:09 INFO mapred.JobClient: map 0% reduce 0%
12/04/08 19:19:09 INFO mapred.MapTask: Finished spill 0
12/04/08 19:19:09 INFO mapred.Task: Task:attempt_local_0007_m_000000_0 is done. And is in the process of commiting
12/04/08 19:19:11 INFO mapred.LocalJobRunner:
12/04/08 19:19:11 INFO mapred.Task: Task 'attempt_local_0007_m_000000_0' done.
12/04/08 19:19:11 INFO mapred.LocalJobRunner:
12/04/08 19:19:11 INFO mapred.Merger: Merging 1 sorted segments
12/04/08 19:19:11 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 16804860 bytes
12/04/08 19:19:11 INFO mapred.LocalJobRunner:
12/04/08 19:19:12 INFO mapred.Task: Task:attempt_local_0007_r_000000_0 is done. And is in the process of commiting
12/04/08 19:19:12 INFO mapred.LocalJobRunner:
12/04/08 19:19:12 INFO mapred.Task: Task attempt_local_0007_r_000000_0 is allowed to commit now
12/04/08 19:19:12 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0007_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/08 19:19:12 INFO mapred.JobClient: map 100% reduce 0%
12/04/08 19:19:14 INFO mapred.LocalJobRunner: reduce > reduce
12/04/08 19:19:14 INFO mapred.Task: Task 'attempt_local_0007_r_000000_0' done.
12/04/08 19:19:15 INFO mapred.JobClient: map 100% reduce 100%
12/04/08 19:19:15 INFO mapred.JobClient: Job complete: job_local_0007
12/04/08 19:19:15 INFO mapred.JobClient: Counters: 16
12/04/08 19:19:15 INFO mapred.JobClient: File Output Format Counters
12/04/08 19:19:15 INFO mapred.JobClient: Bytes Written=17156391
12/04/08 19:19:15 INFO mapred.JobClient: FileSystemCounters
12/04/08 19:19:15 INFO mapred.JobClient: FILE_BYTES_READ=721795382
12/04/08 19:19:15 INFO mapred.JobClient: FILE_BYTES_WRITTEN=655805035
12/04/08 19:19:15 INFO mapred.JobClient: File Input Format Counters
12/04/08 19:19:15 INFO mapred.JobClient: Bytes Read=17156391
12/04/08 19:19:15 INFO mapred.JobClient: Map-Reduce Framework
12/04/08 19:19:15 INFO mapred.JobClient: Reduce input groups=21578
12/04/08 19:19:15 INFO mapred.JobClient: Map output materialized bytes=16804864
12/04/08 19:19:15 INFO mapred.JobClient: Combine output records=0
12/04/08 19:19:15 INFO mapred.JobClient: Map input records=21578
12/04/08 19:19:15 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/08 19:19:15 INFO mapred.JobClient: Reduce output records=21578
12/04/08 19:19:15 INFO mapred.JobClient: Spilled Records=43156
12/04/08 19:19:15 INFO mapred.JobClient: Map output bytes=16722273
12/04/08 19:19:15 INFO mapred.JobClient: Combine input records=0
12/04/08 19:19:15 INFO mapred.JobClient: Map output records=21578
12/04/08 19:19:15 INFO mapred.JobClient: SPLIT_RAW_BYTES=158
12/04/08 19:19:15 INFO mapred.JobClient: Reduce input records=21578
12/04/08 19:19:15 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/08 19:19:15 INFO driver.MahoutDriver: Program took 76171 ms (Minutes: 1.2695166666666666)
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
no HADOOP_HOME set, running locally
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/mahout-examples-0.7-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-jcl-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
12/04/08 19:19:16 INFO common.AbstractJob: Command line arguments: {--clustering=null, --clusters=[/tmp/mahout-work-jenkins/reuters-kmeans-clusters], --convergenceDelta=[0.5], --distanceMeasure=[org.apache.mahout.common.distance.CosineDistanceMeasure], --endPhase=[2147483647], --input=[/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors/], --maxIter=[10], --method=[mapreduce], --numClusters=[20], --output=[/tmp/mahout-work-jenkins/reuters-kmeans], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
12/04/08 19:19:16 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-kmeans-clusters
12/04/08 19:19:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
12/04/08 19:19:16 INFO compress.CodecPool: Got brand-new compressor
12/04/08 19:19:18 INFO kmeans.RandomSeedGenerator: Wrote 20 vectors to /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed
12/04/08 19:19:18 INFO kmeans.KMeansDriver: Input: /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors Clusters In: /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed Out: /tmp/mahout-work-jenkins/reuters-kmeans Distance: org.apache.mahout.common.distance.CosineDistanceMeasure
12/04/08 19:19:18 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10 num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}
12/04/08 19:19:18 INFO compress.CodecPool: Got brand-new decompressor
Exception in thread "main" java.io.IOException: Mkdirs failed to create /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed/clusters-0
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:366)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:528)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:843)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:831)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:823)
at org.apache.mahout.clustering.classify.ClusterClassifier.writePolicy(ClusterClassifier.java:232)
at org.apache.mahout.clustering.classify.ClusterClassifier.writeToSeqFiles(ClusterClassifier.java:185)
at org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:254)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:154)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:104)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.mahout.clustering.kmeans.KMeansDriver.main(KMeansDriver.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: Mahout-Examples-Cluster-Reuters #95
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/95/>
------------------------------------------
[...truncated 6201 lines...]
12/04/07 19:18:46 INFO mapred.JobClient: Reduce input records=21578
12/04/07 19:18:46 INFO common.HadoopUtil: Deleting /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/07 19:18:47 INFO input.FileInputFormat: Total input paths to process : 1
12/04/07 19:18:47 INFO mapred.JobClient: Running job: job_local_0004
12/04/07 19:18:47 INFO mapred.MapTask: io.sort.mb = 100
12/04/07 19:18:47 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/07 19:18:47 INFO mapred.MapTask: record buffer = 262144/327680
12/04/07 19:18:48 INFO mapred.JobClient: map 0% reduce 0%
12/04/07 19:18:48 INFO mapred.MapTask: Starting flush of map output
12/04/07 19:18:48 INFO mapred.MapTask: Finished spill 0
12/04/07 19:18:48 INFO mapred.Task: Task:attempt_local_0004_m_000000_0 is done. And is in the process of commiting
12/04/07 19:18:50 INFO mapred.LocalJobRunner:
12/04/07 19:18:50 INFO mapred.Task: Task 'attempt_local_0004_m_000000_0' done.
12/04/07 19:18:50 INFO mapred.LocalJobRunner:
12/04/07 19:18:50 INFO mapred.Merger: Merging 1 sorted segments
12/04/07 19:18:50 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/07 19:18:50 INFO mapred.LocalJobRunner:
12/04/07 19:18:50 INFO mapred.JobClient: map 100% reduce 0%
12/04/07 19:18:51 INFO mapred.Task: Task:attempt_local_0004_r_000000_0 is done. And is in the process of commiting
12/04/07 19:18:51 INFO mapred.LocalJobRunner:
12/04/07 19:18:51 INFO mapred.Task: Task attempt_local_0004_r_000000_0 is allowed to commit now
12/04/07 19:18:51 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0004_r_000000_0' to /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/07 19:18:53 INFO mapred.LocalJobRunner: reduce > reduce
12/04/07 19:18:53 INFO mapred.Task: Task 'attempt_local_0004_r_000000_0' done.
12/04/07 19:18:53 INFO mapred.JobClient: map 100% reduce 100%
12/04/07 19:18:53 INFO mapred.JobClient: Job complete: job_local_0004
12/04/07 19:18:53 INFO mapred.JobClient: Counters: 16
12/04/07 19:18:53 INFO mapred.JobClient: File Output Format Counters
12/04/07 19:18:53 INFO mapred.JobClient: Bytes Written=17780621
12/04/07 19:18:53 INFO mapred.JobClient: FileSystemCounters
12/04/07 19:18:53 INFO mapred.JobClient: FILE_BYTES_READ=392451318
12/04/07 19:18:53 INFO mapred.JobClient: FILE_BYTES_WRITTEN=361930607
12/04/07 19:18:53 INFO mapred.JobClient: File Input Format Counters
12/04/07 19:18:53 INFO mapred.JobClient: Bytes Read=17780621
12/04/07 19:18:53 INFO mapred.JobClient: Map-Reduce Framework
12/04/07 19:18:53 INFO mapred.JobClient: Reduce input groups=21578
12/04/07 19:18:53 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/07 19:18:53 INFO mapred.JobClient: Combine output records=0
12/04/07 19:18:53 INFO mapred.JobClient: Map input records=21578
12/04/07 19:18:53 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/07 19:18:53 INFO mapred.JobClient: Reduce output records=21578
12/04/07 19:18:53 INFO mapred.JobClient: Spilled Records=43156
12/04/07 19:18:53 INFO mapred.JobClient: Map output bytes=17337483
12/04/07 19:18:53 INFO mapred.JobClient: Combine input records=0
12/04/07 19:18:53 INFO mapred.JobClient: Map output records=21578
12/04/07 19:18:53 INFO mapred.JobClient: SPLIT_RAW_BYTES=157
12/04/07 19:18:53 INFO mapred.JobClient: Reduce input records=21578
12/04/07 19:18:53 INFO common.HadoopUtil: Deleting /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/07 19:18:53 INFO common.HadoopUtil: Deleting /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/df-count
12/04/07 19:18:53 INFO input.FileInputFormat: Total input paths to process : 1
12/04/07 19:18:53 INFO mapred.JobClient: Running job: job_local_0005
12/04/07 19:18:53 INFO mapred.MapTask: io.sort.mb = 100
12/04/07 19:18:53 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/07 19:18:53 INFO mapred.MapTask: record buffer = 262144/327680
12/04/07 19:18:53 INFO mapred.MapTask: Spilling map output: record full = true
12/04/07 19:18:53 INFO mapred.MapTask: bufstart = 0; bufend = 3145728; bufvoid = 99614720
12/04/07 19:18:53 INFO mapred.MapTask: kvstart = 0; kvend = 262144; length = 327680
12/04/07 19:18:54 INFO mapred.MapTask: Finished spill 0
12/04/07 19:18:54 INFO mapred.MapTask: Spilling map output: record full = true
12/04/07 19:18:54 INFO mapred.MapTask: bufstart = 3145728; bufend = 6291444; bufvoid = 99614720
12/04/07 19:18:54 INFO mapred.MapTask: kvstart = 262144; kvend = 196607; length = 327680
12/04/07 19:18:54 INFO mapred.JobClient: map 0% reduce 0%
12/04/07 19:18:54 INFO mapred.MapTask: Finished spill 1
12/04/07 19:18:54 INFO mapred.MapTask: Spilling map output: record full = true
12/04/07 19:18:54 INFO mapred.MapTask: bufstart = 6291444; bufend = 9437160; bufvoid = 99614720
12/04/07 19:18:54 INFO mapred.MapTask: kvstart = 196607; kvend = 131070; length = 327680
12/04/07 19:18:55 INFO mapred.MapTask: Finished spill 2
12/04/07 19:18:55 INFO mapred.MapTask: Spilling map output: record full = true
12/04/07 19:18:55 INFO mapred.MapTask: bufstart = 9437160; bufend = 12582876; bufvoid = 99614720
12/04/07 19:18:55 INFO mapred.MapTask: kvstart = 131070; kvend = 65533; length = 327680
12/04/07 19:18:55 INFO mapred.MapTask: Finished spill 3
12/04/07 19:18:55 INFO mapred.MapTask: Spilling map output: record full = true
12/04/07 19:18:55 INFO mapred.MapTask: bufstart = 12582876; bufend = 15728604; bufvoid = 99614720
12/04/07 19:18:55 INFO mapred.MapTask: kvstart = 65533; kvend = 327677; length = 327680
12/04/07 19:18:56 INFO mapred.MapTask: Finished spill 4
12/04/07 19:18:56 INFO mapred.MapTask: Starting flush of map output
12/04/07 19:18:56 INFO mapred.MapTask: Finished spill 5
12/04/07 19:18:56 INFO mapred.Merger: Merging 6 sorted segments
12/04/07 19:18:56 INFO mapred.Merger: Down to the last merge-pass, with 6 segments left of total size: 1871742 bytes
12/04/07 19:18:56 INFO mapred.Task: Task:attempt_local_0005_m_000000_0 is done. And is in the process of commiting
12/04/07 19:18:59 INFO mapred.LocalJobRunner:
12/04/07 19:18:59 INFO mapred.LocalJobRunner:
12/04/07 19:18:59 INFO mapred.Task: Task 'attempt_local_0005_m_000000_0' done.
12/04/07 19:18:59 INFO mapred.LocalJobRunner:
12/04/07 19:18:59 INFO mapred.Merger: Merging 1 sorted segments
12/04/07 19:18:59 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 585314 bytes
12/04/07 19:18:59 INFO mapred.LocalJobRunner:
12/04/07 19:18:59 INFO mapred.Task: Task:attempt_local_0005_r_000000_0 is done. And is in the process of commiting
12/04/07 19:18:59 INFO mapred.LocalJobRunner:
12/04/07 19:18:59 INFO mapred.Task: Task attempt_local_0005_r_000000_0 is allowed to commit now
12/04/07 19:18:59 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0005_r_000000_0' to /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/df-count
12/04/07 19:19:00 INFO mapred.JobClient: map 100% reduce 0%
12/04/07 19:19:02 INFO mapred.LocalJobRunner: reduce > reduce
12/04/07 19:19:02 INFO mapred.Task: Task 'attempt_local_0005_r_000000_0' done.
12/04/07 19:19:03 INFO mapred.JobClient: map 100% reduce 100%
12/04/07 19:19:03 INFO mapred.JobClient: Job complete: job_local_0005
12/04/07 19:19:03 INFO mapred.JobClient: Counters: 16
12/04/07 19:19:03 INFO mapred.JobClient: File Output Format Counters
12/04/07 19:19:03 INFO mapred.JobClient: Bytes Written=851221
12/04/07 19:19:03 INFO mapred.JobClient: FileSystemCounters
12/04/07 19:19:03 INFO mapred.JobClient: FILE_BYTES_READ=498032552
12/04/07 19:19:03 INFO mapred.JobClient: FILE_BYTES_WRITTEN=434189763
12/04/07 19:19:03 INFO mapred.JobClient: File Input Format Counters
12/04/07 19:19:03 INFO mapred.JobClient: Bytes Read=17780621
12/04/07 19:19:03 INFO mapred.JobClient: Map-Reduce Framework
12/04/07 19:19:03 INFO mapred.JobClient: Reduce input groups=41808
12/04/07 19:19:03 INFO mapred.JobClient: Map output materialized bytes=585318
12/04/07 19:19:03 INFO mapred.JobClient: Combine output records=175503
12/04/07 19:19:03 INFO mapred.JobClient: Map input records=21578
12/04/07 19:19:03 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/07 19:19:03 INFO mapred.JobClient: Reduce output records=41808
12/04/07 19:19:03 INFO mapred.JobClient: Spilled Records=217311
12/04/07 19:19:03 INFO mapred.JobClient: Map output bytes=18371184
12/04/07 19:19:03 INFO mapred.JobClient: Combine input records=1664627
12/04/07 19:19:03 INFO mapred.JobClient: Map output records=1530932
12/04/07 19:19:03 INFO mapred.JobClient: SPLIT_RAW_BYTES=150
12/04/07 19:19:03 INFO mapred.JobClient: Reduce input records=41808
12/04/07 19:19:03 INFO input.FileInputFormat: Total input paths to process : 1
12/04/07 19:19:03 INFO filecache.TrackerDistributedCacheManager: Creating frequency.file-0 in /tmp/hadoop-hudson/mapred/local/archive/-7108820634966285364_1334525619_238998213/file/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans-work-5258590934477135848 with rwxr-xr-x
12/04/07 19:19:03 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-hudson/mapred/local/archive/-7108820634966285364_1334525619_238998213/file/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/07 19:19:03 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-hudson/mapred/local/archive/-7108820634966285364_1334525619_238998213/file/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/07 19:19:03 INFO mapred.JobClient: Running job: job_local_0006
12/04/07 19:19:03 INFO mapred.MapTask: io.sort.mb = 100
12/04/07 19:19:03 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/07 19:19:03 INFO mapred.MapTask: record buffer = 262144/327680
12/04/07 19:19:04 INFO mapred.JobClient: map 0% reduce 0%
12/04/07 19:19:04 INFO mapred.MapTask: Starting flush of map output
12/04/07 19:19:05 INFO mapred.MapTask: Finished spill 0
12/04/07 19:19:05 INFO mapred.Task: Task:attempt_local_0006_m_000000_0 is done. And is in the process of commiting
12/04/07 19:19:06 INFO mapred.LocalJobRunner:
12/04/07 19:19:06 INFO mapred.Task: Task 'attempt_local_0006_m_000000_0' done.
12/04/07 19:19:06 INFO mapred.LocalJobRunner:
12/04/07 19:19:06 INFO mapred.Merger: Merging 1 sorted segments
12/04/07 19:19:06 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/07 19:19:06 INFO mapred.LocalJobRunner:
12/04/07 19:19:06 INFO mapred.JobClient: map 100% reduce 0%
12/04/07 19:19:08 INFO mapred.Task: Task:attempt_local_0006_r_000000_0 is done. And is in the process of commiting
12/04/07 19:19:08 INFO mapred.LocalJobRunner:
12/04/07 19:19:08 INFO mapred.Task: Task attempt_local_0006_r_000000_0 is allowed to commit now
12/04/07 19:19:08 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0006_r_000000_0' to /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/07 19:19:09 INFO mapred.LocalJobRunner: reduce > reduce
12/04/07 19:19:09 INFO mapred.Task: Task 'attempt_local_0006_r_000000_0' done.
12/04/07 19:19:09 INFO mapred.JobClient: map 100% reduce 100%
12/04/07 19:19:09 INFO mapred.JobClient: Job complete: job_local_0006
12/04/07 19:19:09 INFO mapred.JobClient: Counters: 16
12/04/07 19:19:09 INFO mapred.JobClient: File Output Format Counters
12/04/07 19:19:09 INFO mapred.JobClient: Bytes Written=17156391
12/04/07 19:19:09 INFO mapred.JobClient: FileSystemCounters
12/04/07 19:19:09 INFO mapred.JobClient: FILE_BYTES_READ=604126299
12/04/07 19:19:09 INFO mapred.JobClient: FILE_BYTES_WRITTEN=539159617
12/04/07 19:19:09 INFO mapred.JobClient: File Input Format Counters
12/04/07 19:19:09 INFO mapred.JobClient: Bytes Read=17780621
12/04/07 19:19:09 INFO mapred.JobClient: Map-Reduce Framework
12/04/07 19:19:09 INFO mapred.JobClient: Reduce input groups=21578
12/04/07 19:19:09 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/07 19:19:09 INFO mapred.JobClient: Combine output records=0
12/04/07 19:19:09 INFO mapred.JobClient: Map input records=21578
12/04/07 19:19:09 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/07 19:19:09 INFO mapred.JobClient: Reduce output records=21578
12/04/07 19:19:09 INFO mapred.JobClient: Spilled Records=43156
12/04/07 19:19:09 INFO mapred.JobClient: Map output bytes=17337483
12/04/07 19:19:09 INFO mapred.JobClient: Combine input records=0
12/04/07 19:19:09 INFO mapred.JobClient: Map output records=21578
12/04/07 19:19:09 INFO mapred.JobClient: SPLIT_RAW_BYTES=150
12/04/07 19:19:09 INFO mapred.JobClient: Reduce input records=21578
12/04/07 19:19:09 INFO common.HadoopUtil: Deleting /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/07 19:19:10 INFO input.FileInputFormat: Total input paths to process : 1
12/04/07 19:19:10 INFO mapred.JobClient: Running job: job_local_0007
12/04/07 19:19:10 INFO mapred.MapTask: io.sort.mb = 100
12/04/07 19:19:10 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/07 19:19:10 INFO mapred.MapTask: record buffer = 262144/327680
12/04/07 19:19:11 INFO mapred.JobClient: map 0% reduce 0%
12/04/07 19:19:11 INFO mapred.MapTask: Starting flush of map output
12/04/07 19:19:11 INFO mapred.MapTask: Finished spill 0
12/04/07 19:19:11 INFO mapred.Task: Task:attempt_local_0007_m_000000_0 is done. And is in the process of commiting
12/04/07 19:19:13 INFO mapred.LocalJobRunner:
12/04/07 19:19:13 INFO mapred.Task: Task 'attempt_local_0007_m_000000_0' done.
12/04/07 19:19:13 INFO mapred.LocalJobRunner:
12/04/07 19:19:13 INFO mapred.Merger: Merging 1 sorted segments
12/04/07 19:19:13 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 16804860 bytes
12/04/07 19:19:13 INFO mapred.LocalJobRunner:
12/04/07 19:19:14 INFO mapred.Task: Task:attempt_local_0007_r_000000_0 is done. And is in the process of commiting
12/04/07 19:19:14 INFO mapred.LocalJobRunner:
12/04/07 19:19:14 INFO mapred.Task: Task attempt_local_0007_r_000000_0 is allowed to commit now
12/04/07 19:19:14 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0007_r_000000_0' to /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/07 19:19:14 INFO mapred.JobClient: map 100% reduce 0%
12/04/07 19:19:16 INFO mapred.LocalJobRunner: reduce > reduce
12/04/07 19:19:16 INFO mapred.Task: Task 'attempt_local_0007_r_000000_0' done.
12/04/07 19:19:17 INFO mapred.JobClient: map 100% reduce 100%
12/04/07 19:19:17 INFO mapred.JobClient: Job complete: job_local_0007
12/04/07 19:19:17 INFO mapred.JobClient: Counters: 16
12/04/07 19:19:17 INFO mapred.JobClient: File Output Format Counters
12/04/07 19:19:17 INFO mapred.JobClient: Bytes Written=17156391
12/04/07 19:19:17 INFO mapred.JobClient: FileSystemCounters
12/04/07 19:19:17 INFO mapred.JobClient: FILE_BYTES_READ=721786302
12/04/07 19:19:17 INFO mapred.JobClient: FILE_BYTES_WRITTEN=655795831
12/04/07 19:19:17 INFO mapred.JobClient: File Input Format Counters
12/04/07 19:19:17 INFO mapred.JobClient: Bytes Read=17156391
12/04/07 19:19:17 INFO mapred.JobClient: Map-Reduce Framework
12/04/07 19:19:17 INFO mapred.JobClient: Reduce input groups=21578
12/04/07 19:19:17 INFO mapred.JobClient: Map output materialized bytes=16804864
12/04/07 19:19:17 INFO mapred.JobClient: Combine output records=0
12/04/07 19:19:17 INFO mapred.JobClient: Map input records=21578
12/04/07 19:19:17 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/07 19:19:17 INFO mapred.JobClient: Reduce output records=21578
12/04/07 19:19:17 INFO mapred.JobClient: Spilled Records=43156
12/04/07 19:19:17 INFO mapred.JobClient: Map output bytes=16722273
12/04/07 19:19:17 INFO mapred.JobClient: Combine input records=0
12/04/07 19:19:17 INFO mapred.JobClient: Map output records=21578
12/04/07 19:19:17 INFO mapred.JobClient: SPLIT_RAW_BYTES=157
12/04/07 19:19:17 INFO mapred.JobClient: Reduce input records=21578
12/04/07 19:19:17 INFO common.HadoopUtil: Deleting /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/07 19:19:17 INFO driver.MahoutDriver: Program took 76892 ms (Minutes: 1.2815333333333334)
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
no HADOOP_HOME set, running locally
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/mahout-examples-0.7-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-jcl-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
12/04/07 19:19:18 INFO common.AbstractJob: Command line arguments: {--clustering=null, --clusters=[/tmp/mahout-work-hudson/reuters-kmeans-clusters], --convergenceDelta=[0.5], --distanceMeasure=[org.apache.mahout.common.distance.CosineDistanceMeasure], --endPhase=[2147483647], --input=[/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors/], --maxIter=[10], --method=[mapreduce], --numClusters=[20], --output=[/tmp/mahout-work-hudson/reuters-kmeans], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
12/04/07 19:19:18 INFO common.HadoopUtil: Deleting /tmp/mahout-work-hudson/reuters-kmeans-clusters
12/04/07 19:19:18 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
12/04/07 19:19:18 INFO compress.CodecPool: Got brand-new compressor
12/04/07 19:19:20 INFO kmeans.RandomSeedGenerator: Wrote 20 vectors to /tmp/mahout-work-hudson/reuters-kmeans-clusters/part-randomSeed
12/04/07 19:19:20 INFO kmeans.KMeansDriver: Input: /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors Clusters In: /tmp/mahout-work-hudson/reuters-kmeans-clusters/part-randomSeed Out: /tmp/mahout-work-hudson/reuters-kmeans Distance: org.apache.mahout.common.distance.CosineDistanceMeasure
12/04/07 19:19:20 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10 num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}
12/04/07 19:19:20 INFO compress.CodecPool: Got brand-new decompressor
Exception in thread "main" java.io.IOException: Mkdirs failed to create /tmp/mahout-work-hudson/reuters-kmeans-clusters/part-randomSeed/clusters-0
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:366)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:528)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:843)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:831)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:823)
at org.apache.mahout.clustering.classify.ClusterClassifier.writePolicy(ClusterClassifier.java:232)
at org.apache.mahout.clustering.classify.ClusterClassifier.writeToSeqFiles(ClusterClassifier.java:185)
at org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:254)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:154)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:104)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.mahout.clustering.kmeans.KMeansDriver.main(KMeansDriver.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: Mahout-Examples-Cluster-Reuters #94
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/94/changes>
Changes:
[srowen] Allow choice of load factor in custom maps
[srowen] MAHOUT-973 one more file needed for fix to compute maxDF as a percent of total count
[srowen] MAHOUT-973 fix treatment of value as percentage
------------------------------------------
[...truncated 6191 lines...]
12/04/06 19:18:03 INFO mapred.JobClient: Reduce input records=21578
12/04/06 19:18:03 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/06 19:18:04 INFO input.FileInputFormat: Total input paths to process : 1
12/04/06 19:18:04 INFO mapred.JobClient: Running job: job_local_0004
12/04/06 19:18:04 INFO mapred.MapTask: io.sort.mb = 100
12/04/06 19:18:04 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/06 19:18:04 INFO mapred.MapTask: record buffer = 262144/327680
12/04/06 19:18:05 INFO mapred.JobClient: map 0% reduce 0%
12/04/06 19:18:05 INFO mapred.MapTask: Starting flush of map output
12/04/06 19:18:05 INFO mapred.MapTask: Finished spill 0
12/04/06 19:18:05 INFO mapred.Task: Task:attempt_local_0004_m_000000_0 is done. And is in the process of commiting
12/04/06 19:18:07 INFO mapred.LocalJobRunner:
12/04/06 19:18:07 INFO mapred.Task: Task 'attempt_local_0004_m_000000_0' done.
12/04/06 19:18:07 INFO mapred.LocalJobRunner:
12/04/06 19:18:07 INFO mapred.Merger: Merging 1 sorted segments
12/04/06 19:18:07 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/06 19:18:07 INFO mapred.LocalJobRunner:
12/04/06 19:18:08 INFO mapred.JobClient: map 100% reduce 0%
12/04/06 19:18:08 INFO mapred.Task: Task:attempt_local_0004_r_000000_0 is done. And is in the process of commiting
12/04/06 19:18:08 INFO mapred.LocalJobRunner:
12/04/06 19:18:08 INFO mapred.Task: Task attempt_local_0004_r_000000_0 is allowed to commit now
12/04/06 19:18:08 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0004_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/06 19:18:10 INFO mapred.LocalJobRunner: reduce > reduce
12/04/06 19:18:10 INFO mapred.Task: Task 'attempt_local_0004_r_000000_0' done.
12/04/06 19:18:11 INFO mapred.JobClient: map 100% reduce 100%
12/04/06 19:18:11 INFO mapred.JobClient: Job complete: job_local_0004
12/04/06 19:18:11 INFO mapred.JobClient: Counters: 16
12/04/06 19:18:11 INFO mapred.JobClient: File Output Format Counters
12/04/06 19:18:11 INFO mapred.JobClient: Bytes Written=17780621
12/04/06 19:18:11 INFO mapred.JobClient: FileSystemCounters
12/04/06 19:18:11 INFO mapred.JobClient: FILE_BYTES_READ=392451502
12/04/06 19:18:11 INFO mapred.JobClient: FILE_BYTES_WRITTEN=361931993
12/04/06 19:18:11 INFO mapred.JobClient: File Input Format Counters
12/04/06 19:18:11 INFO mapred.JobClient: Bytes Read=17780621
12/04/06 19:18:11 INFO mapred.JobClient: Map-Reduce Framework
12/04/06 19:18:11 INFO mapred.JobClient: Reduce input groups=21578
12/04/06 19:18:11 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/06 19:18:11 INFO mapred.JobClient: Combine output records=0
12/04/06 19:18:11 INFO mapred.JobClient: Map input records=21578
12/04/06 19:18:11 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/06 19:18:11 INFO mapred.JobClient: Reduce output records=21578
12/04/06 19:18:11 INFO mapred.JobClient: Spilled Records=43156
12/04/06 19:18:11 INFO mapred.JobClient: Map output bytes=17337483
12/04/06 19:18:11 INFO mapred.JobClient: Combine input records=0
12/04/06 19:18:11 INFO mapred.JobClient: Map output records=21578
12/04/06 19:18:11 INFO mapred.JobClient: SPLIT_RAW_BYTES=158
12/04/06 19:18:11 INFO mapred.JobClient: Reduce input records=21578
12/04/06 19:18:11 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/06 19:18:11 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/df-count
12/04/06 19:18:11 INFO input.FileInputFormat: Total input paths to process : 1
12/04/06 19:18:11 INFO mapred.JobClient: Running job: job_local_0005
12/04/06 19:18:11 INFO mapred.MapTask: io.sort.mb = 100
12/04/06 19:18:11 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/06 19:18:11 INFO mapred.MapTask: record buffer = 262144/327680
12/04/06 19:18:11 INFO mapred.MapTask: Spilling map output: record full = true
12/04/06 19:18:11 INFO mapred.MapTask: bufstart = 0; bufend = 3145728; bufvoid = 99614720
12/04/06 19:18:11 INFO mapred.MapTask: kvstart = 0; kvend = 262144; length = 327680
12/04/06 19:18:12 INFO mapred.MapTask: Finished spill 0
12/04/06 19:18:12 INFO mapred.MapTask: Spilling map output: record full = true
12/04/06 19:18:12 INFO mapred.MapTask: bufstart = 3145728; bufend = 6291444; bufvoid = 99614720
12/04/06 19:18:12 INFO mapred.MapTask: kvstart = 262144; kvend = 196607; length = 327680
12/04/06 19:18:12 INFO mapred.JobClient: map 0% reduce 0%
12/04/06 19:18:12 INFO mapred.MapTask: Finished spill 1
12/04/06 19:18:12 INFO mapred.MapTask: Spilling map output: record full = true
12/04/06 19:18:12 INFO mapred.MapTask: bufstart = 6291444; bufend = 9437160; bufvoid = 99614720
12/04/06 19:18:12 INFO mapred.MapTask: kvstart = 196607; kvend = 131070; length = 327680
12/04/06 19:18:13 INFO mapred.MapTask: Finished spill 2
12/04/06 19:18:13 INFO mapred.MapTask: Spilling map output: record full = true
12/04/06 19:18:13 INFO mapred.MapTask: bufstart = 9437160; bufend = 12582876; bufvoid = 99614720
12/04/06 19:18:13 INFO mapred.MapTask: kvstart = 131070; kvend = 65533; length = 327680
12/04/06 19:18:13 INFO mapred.MapTask: Finished spill 3
12/04/06 19:18:14 INFO mapred.MapTask: Spilling map output: record full = true
12/04/06 19:18:14 INFO mapred.MapTask: bufstart = 12582876; bufend = 15728604; bufvoid = 99614720
12/04/06 19:18:14 INFO mapred.MapTask: kvstart = 65533; kvend = 327677; length = 327680
12/04/06 19:18:14 INFO mapred.MapTask: Finished spill 4
12/04/06 19:18:14 INFO mapred.MapTask: Starting flush of map output
12/04/06 19:18:14 INFO mapred.MapTask: Finished spill 5
12/04/06 19:18:14 INFO mapred.Merger: Merging 6 sorted segments
12/04/06 19:18:14 INFO mapred.Merger: Down to the last merge-pass, with 6 segments left of total size: 1871742 bytes
12/04/06 19:18:14 INFO mapred.Task: Task:attempt_local_0005_m_000000_0 is done. And is in the process of commiting
12/04/06 19:18:17 INFO mapred.LocalJobRunner:
12/04/06 19:18:17 INFO mapred.LocalJobRunner:
12/04/06 19:18:17 INFO mapred.Task: Task 'attempt_local_0005_m_000000_0' done.
12/04/06 19:18:17 INFO mapred.LocalJobRunner:
12/04/06 19:18:17 INFO mapred.Merger: Merging 1 sorted segments
12/04/06 19:18:17 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 585314 bytes
12/04/06 19:18:17 INFO mapred.LocalJobRunner:
12/04/06 19:18:17 INFO mapred.Task: Task:attempt_local_0005_r_000000_0 is done. And is in the process of commiting
12/04/06 19:18:17 INFO mapred.LocalJobRunner:
12/04/06 19:18:17 INFO mapred.Task: Task attempt_local_0005_r_000000_0 is allowed to commit now
12/04/06 19:18:17 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0005_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/df-count
12/04/06 19:18:17 INFO mapred.JobClient: map 100% reduce 0%
12/04/06 19:18:20 INFO mapred.LocalJobRunner: reduce > reduce
12/04/06 19:18:20 INFO mapred.Task: Task 'attempt_local_0005_r_000000_0' done.
12/04/06 19:18:20 INFO mapred.JobClient: map 100% reduce 100%
12/04/06 19:18:20 INFO mapred.JobClient: Job complete: job_local_0005
12/04/06 19:18:20 INFO mapred.JobClient: Counters: 16
12/04/06 19:18:20 INFO mapred.JobClient: File Output Format Counters
12/04/06 19:18:20 INFO mapred.JobClient: Bytes Written=851221
12/04/06 19:18:20 INFO mapred.JobClient: FileSystemCounters
12/04/06 19:18:20 INFO mapred.JobClient: FILE_BYTES_READ=498035098
12/04/06 19:18:20 INFO mapred.JobClient: FILE_BYTES_WRITTEN=434193527
12/04/06 19:18:20 INFO mapred.JobClient: File Input Format Counters
12/04/06 19:18:20 INFO mapred.JobClient: Bytes Read=17780621
12/04/06 19:18:20 INFO mapred.JobClient: Map-Reduce Framework
12/04/06 19:18:20 INFO mapred.JobClient: Reduce input groups=41808
12/04/06 19:18:20 INFO mapred.JobClient: Map output materialized bytes=585318
12/04/06 19:18:20 INFO mapred.JobClient: Combine output records=175503
12/04/06 19:18:20 INFO mapred.JobClient: Map input records=21578
12/04/06 19:18:20 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/06 19:18:20 INFO mapred.JobClient: Reduce output records=41808
12/04/06 19:18:20 INFO mapred.JobClient: Spilled Records=217311
12/04/06 19:18:20 INFO mapred.JobClient: Map output bytes=18371184
12/04/06 19:18:20 INFO mapred.JobClient: Combine input records=1664627
12/04/06 19:18:20 INFO mapred.JobClient: Map output records=1530932
12/04/06 19:18:20 INFO mapred.JobClient: SPLIT_RAW_BYTES=151
12/04/06 19:18:20 INFO mapred.JobClient: Reduce input records=41808
12/04/06 19:18:20 INFO input.FileInputFormat: Total input paths to process : 1
12/04/06 19:18:20 INFO filecache.TrackerDistributedCacheManager: Creating frequency.file-0 in /tmp/hadoop-jenkins/mapred/local/archive/7714946264482490086_-299119410_152555213/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans-work--6528827160256529525 with rwxr-xr-x
12/04/06 19:18:20 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-jenkins/mapred/local/archive/7714946264482490086_-299119410_152555213/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/06 19:18:20 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-jenkins/mapred/local/archive/7714946264482490086_-299119410_152555213/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/06 19:18:20 INFO mapred.JobClient: Running job: job_local_0006
12/04/06 19:18:20 INFO mapred.MapTask: io.sort.mb = 100
12/04/06 19:18:21 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/06 19:18:21 INFO mapred.MapTask: record buffer = 262144/327680
12/04/06 19:18:21 INFO mapred.JobClient: map 0% reduce 0%
12/04/06 19:18:21 INFO mapred.MapTask: Starting flush of map output
12/04/06 19:18:22 INFO mapred.MapTask: Finished spill 0
12/04/06 19:18:22 INFO mapred.Task: Task:attempt_local_0006_m_000000_0 is done. And is in the process of commiting
12/04/06 19:18:23 INFO mapred.LocalJobRunner:
12/04/06 19:18:23 INFO mapred.Task: Task 'attempt_local_0006_m_000000_0' done.
12/04/06 19:18:23 INFO mapred.LocalJobRunner:
12/04/06 19:18:23 INFO mapred.Merger: Merging 1 sorted segments
12/04/06 19:18:23 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/06 19:18:23 INFO mapred.LocalJobRunner:
12/04/06 19:18:24 INFO mapred.JobClient: map 100% reduce 0%
12/04/06 19:18:25 INFO mapred.Task: Task:attempt_local_0006_r_000000_0 is done. And is in the process of commiting
12/04/06 19:18:25 INFO mapred.LocalJobRunner:
12/04/06 19:18:25 INFO mapred.Task: Task attempt_local_0006_r_000000_0 is allowed to commit now
12/04/06 19:18:25 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0006_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/06 19:18:26 INFO mapred.LocalJobRunner: reduce > reduce
12/04/06 19:18:26 INFO mapred.Task: Task 'attempt_local_0006_r_000000_0' done.
12/04/06 19:18:27 INFO mapred.JobClient: map 100% reduce 100%
12/04/06 19:18:27 INFO mapred.JobClient: Job complete: job_local_0006
12/04/06 19:18:27 INFO mapred.JobClient: Counters: 16
12/04/06 19:18:27 INFO mapred.JobClient: File Output Format Counters
12/04/06 19:18:27 INFO mapred.JobClient: Bytes Written=17156391
12/04/06 19:18:27 INFO mapred.JobClient: FileSystemCounters
12/04/06 19:18:27 INFO mapred.JobClient: FILE_BYTES_READ=604131207
12/04/06 19:18:27 INFO mapred.JobClient: FILE_BYTES_WRITTEN=539165753
12/04/06 19:18:27 INFO mapred.JobClient: File Input Format Counters
12/04/06 19:18:27 INFO mapred.JobClient: Bytes Read=17780621
12/04/06 19:18:27 INFO mapred.JobClient: Map-Reduce Framework
12/04/06 19:18:27 INFO mapred.JobClient: Reduce input groups=21578
12/04/06 19:18:27 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/06 19:18:27 INFO mapred.JobClient: Combine output records=0
12/04/06 19:18:27 INFO mapred.JobClient: Map input records=21578
12/04/06 19:18:27 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/06 19:18:27 INFO mapred.JobClient: Reduce output records=21578
12/04/06 19:18:27 INFO mapred.JobClient: Spilled Records=43156
12/04/06 19:18:27 INFO mapred.JobClient: Map output bytes=17337483
12/04/06 19:18:27 INFO mapred.JobClient: Combine input records=0
12/04/06 19:18:27 INFO mapred.JobClient: Map output records=21578
12/04/06 19:18:27 INFO mapred.JobClient: SPLIT_RAW_BYTES=151
12/04/06 19:18:27 INFO mapred.JobClient: Reduce input records=21578
12/04/06 19:18:27 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/06 19:18:28 INFO input.FileInputFormat: Total input paths to process : 1
12/04/06 19:18:28 INFO mapred.JobClient: Running job: job_local_0007
12/04/06 19:18:28 INFO mapred.MapTask: io.sort.mb = 100
12/04/06 19:18:28 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/06 19:18:28 INFO mapred.MapTask: record buffer = 262144/327680
12/04/06 19:18:29 INFO mapred.MapTask: Starting flush of map output
12/04/06 19:18:29 INFO mapred.JobClient: map 0% reduce 0%
12/04/06 19:18:29 INFO mapred.MapTask: Finished spill 0
12/04/06 19:18:29 INFO mapred.Task: Task:attempt_local_0007_m_000000_0 is done. And is in the process of commiting
12/04/06 19:18:31 INFO mapred.LocalJobRunner:
12/04/06 19:18:31 INFO mapred.Task: Task 'attempt_local_0007_m_000000_0' done.
12/04/06 19:18:31 INFO mapred.LocalJobRunner:
12/04/06 19:18:31 INFO mapred.Merger: Merging 1 sorted segments
12/04/06 19:18:31 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 16804860 bytes
12/04/06 19:18:31 INFO mapred.LocalJobRunner:
12/04/06 19:18:32 INFO mapred.Task: Task:attempt_local_0007_r_000000_0 is done. And is in the process of commiting
12/04/06 19:18:32 INFO mapred.LocalJobRunner:
12/04/06 19:18:32 INFO mapred.Task: Task attempt_local_0007_r_000000_0 is allowed to commit now
12/04/06 19:18:32 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0007_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/06 19:18:32 INFO mapred.JobClient: map 100% reduce 0%
12/04/06 19:18:34 INFO mapred.LocalJobRunner: reduce > reduce
12/04/06 19:18:34 INFO mapred.Task: Task 'attempt_local_0007_r_000000_0' done.
12/04/06 19:18:35 INFO mapred.JobClient: map 100% reduce 100%
12/04/06 19:18:35 INFO mapred.JobClient: Job complete: job_local_0007
12/04/06 19:18:35 INFO mapred.JobClient: Counters: 16
12/04/06 19:18:35 INFO mapred.JobClient: File Output Format Counters
12/04/06 19:18:35 INFO mapred.JobClient: Bytes Written=17156391
12/04/06 19:18:35 INFO mapred.JobClient: FileSystemCounters
12/04/06 19:18:35 INFO mapred.JobClient: FILE_BYTES_READ=721793572
12/04/06 19:18:35 INFO mapred.JobClient: FILE_BYTES_WRITTEN=655804341
12/04/06 19:18:35 INFO mapred.JobClient: File Input Format Counters
12/04/06 19:18:35 INFO mapred.JobClient: Bytes Read=17156391
12/04/06 19:18:35 INFO mapred.JobClient: Map-Reduce Framework
12/04/06 19:18:35 INFO mapred.JobClient: Reduce input groups=21578
12/04/06 19:18:35 INFO mapred.JobClient: Map output materialized bytes=16804864
12/04/06 19:18:35 INFO mapred.JobClient: Combine output records=0
12/04/06 19:18:35 INFO mapred.JobClient: Map input records=21578
12/04/06 19:18:35 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/06 19:18:35 INFO mapred.JobClient: Reduce output records=21578
12/04/06 19:18:35 INFO mapred.JobClient: Spilled Records=43156
12/04/06 19:18:35 INFO mapred.JobClient: Map output bytes=16722273
12/04/06 19:18:35 INFO mapred.JobClient: Combine input records=0
12/04/06 19:18:35 INFO mapred.JobClient: Map output records=21578
12/04/06 19:18:35 INFO mapred.JobClient: SPLIT_RAW_BYTES=158
12/04/06 19:18:35 INFO mapred.JobClient: Reduce input records=21578
12/04/06 19:18:35 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/06 19:18:35 INFO driver.MahoutDriver: Program took 78084 ms (Minutes: 1.3014)
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
no HADOOP_HOME set, running locally
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/mahout-examples-0.7-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-jcl-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
12/04/06 19:18:35 INFO common.AbstractJob: Command line arguments: {--clustering=null, --clusters=[/tmp/mahout-work-jenkins/reuters-kmeans-clusters], --convergenceDelta=[0.5], --distanceMeasure=[org.apache.mahout.common.distance.CosineDistanceMeasure], --endPhase=[2147483647], --input=[/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors/], --maxIter=[10], --method=[mapreduce], --numClusters=[20], --output=[/tmp/mahout-work-jenkins/reuters-kmeans], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
12/04/06 19:18:36 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-kmeans-clusters
12/04/06 19:18:36 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
12/04/06 19:18:36 INFO compress.CodecPool: Got brand-new compressor
12/04/06 19:18:38 INFO kmeans.RandomSeedGenerator: Wrote 20 vectors to /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed
12/04/06 19:18:38 INFO kmeans.KMeansDriver: Input: /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors Clusters In: /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed Out: /tmp/mahout-work-jenkins/reuters-kmeans Distance: org.apache.mahout.common.distance.CosineDistanceMeasure
12/04/06 19:18:38 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10 num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}
12/04/06 19:18:38 INFO compress.CodecPool: Got brand-new decompressor
Exception in thread "main" java.io.IOException: Mkdirs failed to create /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed/clusters-0
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:366)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:528)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:843)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:831)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:823)
at org.apache.mahout.clustering.classify.ClusterClassifier.writePolicy(ClusterClassifier.java:232)
at org.apache.mahout.clustering.classify.ClusterClassifier.writeToSeqFiles(ClusterClassifier.java:185)
at org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:254)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:154)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:104)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.mahout.clustering.kmeans.KMeansDriver.main(KMeansDriver.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: Mahout-Examples-Cluster-Reuters #93
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/93/>
------------------------------------------
[...truncated 6202 lines...]
12/04/05 19:20:00 INFO common.HadoopUtil: Deleting /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/05 19:20:00 INFO input.FileInputFormat: Total input paths to process : 1
12/04/05 19:20:00 INFO mapred.JobClient: Running job: job_local_0004
12/04/05 19:20:00 INFO mapred.MapTask: io.sort.mb = 100
12/04/05 19:20:00 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/05 19:20:00 INFO mapred.MapTask: record buffer = 262144/327680
12/04/05 19:20:01 INFO mapred.JobClient: map 0% reduce 0%
12/04/05 19:20:01 INFO mapred.MapTask: Starting flush of map output
12/04/05 19:20:01 INFO mapred.MapTask: Finished spill 0
12/04/05 19:20:01 INFO mapred.Task: Task:attempt_local_0004_m_000000_0 is done. And is in the process of commiting
12/04/05 19:20:03 INFO mapred.LocalJobRunner:
12/04/05 19:20:03 INFO mapred.Task: Task 'attempt_local_0004_m_000000_0' done.
12/04/05 19:20:03 INFO mapred.LocalJobRunner:
12/04/05 19:20:03 INFO mapred.Merger: Merging 1 sorted segments
12/04/05 19:20:03 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/05 19:20:03 INFO mapred.LocalJobRunner:
12/04/05 19:20:04 INFO mapred.JobClient: map 100% reduce 0%
12/04/05 19:20:06 INFO mapred.Task: Task:attempt_local_0004_r_000000_0 is done. And is in the process of commiting
12/04/05 19:20:06 INFO mapred.LocalJobRunner:
12/04/05 19:20:06 INFO mapred.Task: Task attempt_local_0004_r_000000_0 is allowed to commit now
12/04/05 19:20:06 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0004_r_000000_0' to /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/05 19:20:09 INFO mapred.LocalJobRunner: reduce > reduce
12/04/05 19:20:09 INFO mapred.LocalJobRunner: reduce > reduce
12/04/05 19:20:09 INFO mapred.Task: Task 'attempt_local_0004_r_000000_0' done.
12/04/05 19:20:10 INFO mapred.JobClient: map 100% reduce 100%
12/04/05 19:20:10 INFO mapred.JobClient: Job complete: job_local_0004
12/04/05 19:20:10 INFO mapred.JobClient: Counters: 16
12/04/05 19:20:10 INFO mapred.JobClient: File Output Format Counters
12/04/05 19:20:10 INFO mapred.JobClient: Bytes Written=17780621
12/04/05 19:20:10 INFO mapred.JobClient: FileSystemCounters
12/04/05 19:20:10 INFO mapred.JobClient: FILE_BYTES_READ=392447742
12/04/05 19:20:10 INFO mapred.JobClient: FILE_BYTES_WRITTEN=361927027
12/04/05 19:20:10 INFO mapred.JobClient: File Input Format Counters
12/04/05 19:20:10 INFO mapred.JobClient: Bytes Read=17780621
12/04/05 19:20:10 INFO mapred.JobClient: Map-Reduce Framework
12/04/05 19:20:10 INFO mapred.JobClient: Reduce input groups=21578
12/04/05 19:20:10 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/05 19:20:10 INFO mapred.JobClient: Combine output records=0
12/04/05 19:20:10 INFO mapred.JobClient: Map input records=21578
12/04/05 19:20:10 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/05 19:20:10 INFO mapred.JobClient: Reduce output records=21578
12/04/05 19:20:10 INFO mapred.JobClient: Spilled Records=43156
12/04/05 19:20:10 INFO mapred.JobClient: Map output bytes=17337483
12/04/05 19:20:10 INFO mapred.JobClient: Combine input records=0
12/04/05 19:20:10 INFO mapred.JobClient: Map output records=21578
12/04/05 19:20:10 INFO mapred.JobClient: SPLIT_RAW_BYTES=157
12/04/05 19:20:10 INFO mapred.JobClient: Reduce input records=21578
12/04/05 19:20:10 INFO common.HadoopUtil: Deleting /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/05 19:20:10 INFO common.HadoopUtil: Deleting /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/df-count
12/04/05 19:20:10 INFO input.FileInputFormat: Total input paths to process : 1
12/04/05 19:20:10 INFO mapred.JobClient: Running job: job_local_0005
12/04/05 19:20:10 INFO mapred.MapTask: io.sort.mb = 100
12/04/05 19:20:10 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/05 19:20:10 INFO mapred.MapTask: record buffer = 262144/327680
12/04/05 19:20:11 INFO mapred.MapTask: Spilling map output: record full = true
12/04/05 19:20:11 INFO mapred.MapTask: bufstart = 0; bufend = 3145728; bufvoid = 99614720
12/04/05 19:20:11 INFO mapred.MapTask: kvstart = 0; kvend = 262144; length = 327680
12/04/05 19:20:11 INFO mapred.MapTask: Finished spill 0
12/04/05 19:20:11 INFO mapred.MapTask: Spilling map output: record full = true
12/04/05 19:20:11 INFO mapred.MapTask: bufstart = 3145728; bufend = 6291444; bufvoid = 99614720
12/04/05 19:20:11 INFO mapred.MapTask: kvstart = 262144; kvend = 196607; length = 327680
12/04/05 19:20:11 INFO mapred.JobClient: map 0% reduce 0%
12/04/05 19:20:11 INFO mapred.MapTask: Finished spill 1
12/04/05 19:20:12 INFO mapred.MapTask: Spilling map output: record full = true
12/04/05 19:20:12 INFO mapred.MapTask: bufstart = 6291444; bufend = 9437160; bufvoid = 99614720
12/04/05 19:20:12 INFO mapred.MapTask: kvstart = 196607; kvend = 131070; length = 327680
12/04/05 19:20:12 INFO mapred.MapTask: Finished spill 2
12/04/05 19:20:12 INFO mapred.MapTask: Spilling map output: record full = true
12/04/05 19:20:12 INFO mapred.MapTask: bufstart = 9437160; bufend = 12582876; bufvoid = 99614720
12/04/05 19:20:12 INFO mapred.MapTask: kvstart = 131070; kvend = 65533; length = 327680
12/04/05 19:20:12 INFO mapred.MapTask: Finished spill 3
12/04/05 19:20:13 INFO mapred.MapTask: Spilling map output: record full = true
12/04/05 19:20:13 INFO mapred.MapTask: bufstart = 12582876; bufend = 15728604; bufvoid = 99614720
12/04/05 19:20:13 INFO mapred.MapTask: kvstart = 65533; kvend = 327677; length = 327680
12/04/05 19:20:13 INFO mapred.MapTask: Finished spill 4
12/04/05 19:20:13 INFO mapred.MapTask: Starting flush of map output
12/04/05 19:20:13 INFO mapred.MapTask: Finished spill 5
12/04/05 19:20:13 INFO mapred.Merger: Merging 6 sorted segments
12/04/05 19:20:13 INFO mapred.Merger: Down to the last merge-pass, with 6 segments left of total size: 1871742 bytes
12/04/05 19:20:13 INFO mapred.Task: Task:attempt_local_0005_m_000000_0 is done. And is in the process of commiting
12/04/05 19:20:16 INFO mapred.LocalJobRunner:
12/04/05 19:20:16 INFO mapred.LocalJobRunner:
12/04/05 19:20:16 INFO mapred.Task: Task 'attempt_local_0005_m_000000_0' done.
12/04/05 19:20:16 INFO mapred.LocalJobRunner:
12/04/05 19:20:16 INFO mapred.Merger: Merging 1 sorted segments
12/04/05 19:20:16 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 585314 bytes
12/04/05 19:20:16 INFO mapred.LocalJobRunner:
12/04/05 19:20:16 INFO mapred.Task: Task:attempt_local_0005_r_000000_0 is done. And is in the process of commiting
12/04/05 19:20:16 INFO mapred.LocalJobRunner:
12/04/05 19:20:16 INFO mapred.Task: Task attempt_local_0005_r_000000_0 is allowed to commit now
12/04/05 19:20:16 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0005_r_000000_0' to /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/df-count
12/04/05 19:20:17 INFO mapred.JobClient: map 100% reduce 0%
12/04/05 19:20:19 INFO mapred.LocalJobRunner: reduce > reduce
12/04/05 19:20:19 INFO mapred.Task: Task 'attempt_local_0005_r_000000_0' done.
12/04/05 19:20:20 INFO mapred.JobClient: map 100% reduce 100%
12/04/05 19:20:20 INFO mapred.JobClient: Job complete: job_local_0005
12/04/05 19:20:20 INFO mapred.JobClient: Counters: 16
12/04/05 19:20:20 INFO mapred.JobClient: File Output Format Counters
12/04/05 19:20:20 INFO mapred.JobClient: Bytes Written=851221
12/04/05 19:20:20 INFO mapred.JobClient: FileSystemCounters
12/04/05 19:20:20 INFO mapred.JobClient: FILE_BYTES_READ=498028082
12/04/05 19:20:20 INFO mapred.JobClient: FILE_BYTES_WRITTEN=434185289
12/04/05 19:20:20 INFO mapred.JobClient: File Input Format Counters
12/04/05 19:20:20 INFO mapred.JobClient: Bytes Read=17780621
12/04/05 19:20:20 INFO mapred.JobClient: Map-Reduce Framework
12/04/05 19:20:20 INFO mapred.JobClient: Reduce input groups=41808
12/04/05 19:20:20 INFO mapred.JobClient: Map output materialized bytes=585318
12/04/05 19:20:20 INFO mapred.JobClient: Combine output records=175503
12/04/05 19:20:20 INFO mapred.JobClient: Map input records=21578
12/04/05 19:20:20 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/05 19:20:20 INFO mapred.JobClient: Reduce output records=41808
12/04/05 19:20:20 INFO mapred.JobClient: Spilled Records=217311
12/04/05 19:20:20 INFO mapred.JobClient: Map output bytes=18371184
12/04/05 19:20:20 INFO mapred.JobClient: Combine input records=1664627
12/04/05 19:20:20 INFO mapred.JobClient: Map output records=1530932
12/04/05 19:20:20 INFO mapred.JobClient: SPLIT_RAW_BYTES=150
12/04/05 19:20:20 INFO mapred.JobClient: Reduce input records=41808
12/04/05 19:20:21 INFO input.FileInputFormat: Total input paths to process : 1
12/04/05 19:20:21 INFO filecache.TrackerDistributedCacheManager: Creating frequency.file-0 in /tmp/hadoop-hudson/mapred/local/archive/7210292023373271358_1334525619_66276213/file/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans-work-4919838161411799932 with rwxr-xr-x
12/04/05 19:20:21 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-hudson/mapred/local/archive/7210292023373271358_1334525619_66276213/file/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/05 19:20:21 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-hudson/mapred/local/archive/7210292023373271358_1334525619_66276213/file/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/05 19:20:21 INFO mapred.JobClient: Running job: job_local_0006
12/04/05 19:20:21 INFO mapred.MapTask: io.sort.mb = 100
12/04/05 19:20:21 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/05 19:20:21 INFO mapred.MapTask: record buffer = 262144/327680
12/04/05 19:20:22 INFO mapred.JobClient: map 0% reduce 0%
12/04/05 19:20:22 INFO mapred.MapTask: Starting flush of map output
12/04/05 19:20:22 INFO mapred.MapTask: Finished spill 0
12/04/05 19:20:22 INFO mapred.Task: Task:attempt_local_0006_m_000000_0 is done. And is in the process of commiting
12/04/05 19:20:24 INFO mapred.LocalJobRunner:
12/04/05 19:20:24 INFO mapred.Task: Task 'attempt_local_0006_m_000000_0' done.
12/04/05 19:20:24 INFO mapred.LocalJobRunner:
12/04/05 19:20:24 INFO mapred.Merger: Merging 1 sorted segments
12/04/05 19:20:24 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/05 19:20:24 INFO mapred.LocalJobRunner:
12/04/05 19:20:25 INFO mapred.Task: Task:attempt_local_0006_r_000000_0 is done. And is in the process of commiting
12/04/05 19:20:25 INFO mapred.LocalJobRunner:
12/04/05 19:20:25 INFO mapred.Task: Task attempt_local_0006_r_000000_0 is allowed to commit now
12/04/05 19:20:25 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0006_r_000000_0' to /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/05 19:20:25 INFO mapred.JobClient: map 100% reduce 0%
12/04/05 19:20:27 INFO mapred.LocalJobRunner: reduce > reduce
12/04/05 19:20:27 INFO mapred.Task: Task 'attempt_local_0006_r_000000_0' done.
12/04/05 19:20:28 INFO mapred.JobClient: map 100% reduce 100%
12/04/05 19:20:28 INFO mapred.JobClient: Job complete: job_local_0006
12/04/05 19:20:28 INFO mapred.JobClient: Counters: 16
12/04/05 19:20:28 INFO mapred.JobClient: File Output Format Counters
12/04/05 19:20:28 INFO mapred.JobClient: Bytes Written=4914503
12/04/05 19:20:28 INFO mapred.JobClient: FileSystemCounters
12/04/05 19:20:28 INFO mapred.JobClient: FILE_BYTES_READ=604120935
12/04/05 19:20:28 INFO mapred.JobClient: FILE_BYTES_WRITTEN=526912341
12/04/05 19:20:28 INFO mapred.JobClient: File Input Format Counters
12/04/05 19:20:28 INFO mapred.JobClient: Bytes Read=17780621
12/04/05 19:20:28 INFO mapred.JobClient: Map-Reduce Framework
12/04/05 19:20:28 INFO mapred.JobClient: Reduce input groups=21578
12/04/05 19:20:28 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/05 19:20:28 INFO mapred.JobClient: Combine output records=0
12/04/05 19:20:28 INFO mapred.JobClient: Map input records=21578
12/04/05 19:20:28 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/05 19:20:28 INFO mapred.JobClient: Reduce output records=21578
12/04/05 19:20:28 INFO mapred.JobClient: Spilled Records=43156
12/04/05 19:20:28 INFO mapred.JobClient: Map output bytes=17337483
12/04/05 19:20:28 INFO mapred.JobClient: Combine input records=0
12/04/05 19:20:28 INFO mapred.JobClient: Map output records=21578
12/04/05 19:20:28 INFO mapred.JobClient: SPLIT_RAW_BYTES=150
12/04/05 19:20:28 INFO mapred.JobClient: Reduce input records=21578
12/04/05 19:20:28 INFO common.HadoopUtil: Deleting /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/05 19:20:28 INFO input.FileInputFormat: Total input paths to process : 1
12/04/05 19:20:28 INFO mapred.JobClient: Running job: job_local_0007
12/04/05 19:20:28 INFO mapred.MapTask: io.sort.mb = 100
12/04/05 19:20:28 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/05 19:20:28 INFO mapred.MapTask: record buffer = 262144/327680
12/04/05 19:20:28 INFO mapred.MapTask: Starting flush of map output
12/04/05 19:20:29 INFO mapred.MapTask: Finished spill 0
12/04/05 19:20:29 INFO mapred.Task: Task:attempt_local_0007_m_000000_0 is done. And is in the process of commiting
12/04/05 19:20:29 INFO mapred.JobClient: map 0% reduce 0%
12/04/05 19:20:31 INFO mapred.LocalJobRunner:
12/04/05 19:20:31 INFO mapred.Task: Task 'attempt_local_0007_m_000000_0' done.
12/04/05 19:20:31 INFO mapred.LocalJobRunner:
12/04/05 19:20:31 INFO mapred.Merger: Merging 1 sorted segments
12/04/05 19:20:31 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 4719200 bytes
12/04/05 19:20:31 INFO mapred.LocalJobRunner:
12/04/05 19:20:31 INFO mapred.Task: Task:attempt_local_0007_r_000000_0 is done. And is in the process of commiting
12/04/05 19:20:31 INFO mapred.LocalJobRunner:
12/04/05 19:20:31 INFO mapred.Task: Task attempt_local_0007_r_000000_0 is allowed to commit now
12/04/05 19:20:31 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0007_r_000000_0' to /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/05 19:20:32 INFO mapred.JobClient: map 100% reduce 0%
12/04/05 19:20:34 INFO mapred.LocalJobRunner: reduce > reduce
12/04/05 19:20:34 INFO mapred.Task: Task 'attempt_local_0007_r_000000_0' done.
12/04/05 19:20:35 INFO mapred.JobClient: map 100% reduce 100%
12/04/05 19:20:35 INFO mapred.JobClient: Job complete: job_local_0007
12/04/05 19:20:35 INFO mapred.JobClient: Counters: 16
12/04/05 19:20:35 INFO mapred.JobClient: File Output Format Counters
12/04/05 19:20:35 INFO mapred.JobClient: Bytes Written=4914503
12/04/05 19:20:35 INFO mapred.JobClient: FileSystemCounters
12/04/05 19:20:35 INFO mapred.JobClient: FILE_BYTES_READ=685210606
12/04/05 19:20:35 INFO mapred.JobClient: FILE_BYTES_WRITTEN=594892579
12/04/05 19:20:35 INFO mapred.JobClient: File Input Format Counters
12/04/05 19:20:35 INFO mapred.JobClient: Bytes Read=4914503
12/04/05 19:20:35 INFO mapred.JobClient: Map-Reduce Framework
12/04/05 19:20:35 INFO mapred.JobClient: Reduce input groups=21578
12/04/05 19:20:35 INFO mapred.JobClient: Map output materialized bytes=4719204
12/04/05 19:20:35 INFO mapred.JobClient: Combine output records=0
12/04/05 19:20:35 INFO mapred.JobClient: Map input records=21578
12/04/05 19:20:35 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/05 19:20:35 INFO mapred.JobClient: Reduce output records=21578
12/04/05 19:20:35 INFO mapred.JobClient: Spilled Records=43156
12/04/05 19:20:35 INFO mapred.JobClient: Map output bytes=4659281
12/04/05 19:20:35 INFO mapred.JobClient: Combine input records=0
12/04/05 19:20:35 INFO mapred.JobClient: Map output records=21578
12/04/05 19:20:35 INFO mapred.JobClient: SPLIT_RAW_BYTES=157
12/04/05 19:20:35 INFO mapred.JobClient: Reduce input records=21578
12/04/05 19:20:35 INFO common.HadoopUtil: Deleting /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/05 19:20:35 INFO driver.MahoutDriver: Program took 86861 ms (Minutes: 1.4476833333333334)
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
no HADOOP_HOME set, running locally
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/mahout-examples-0.7-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-jcl-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
12/04/05 19:20:36 INFO common.AbstractJob: Command line arguments: {--clustering=null, --clusters=[/tmp/mahout-work-hudson/reuters-kmeans-clusters], --convergenceDelta=[0.5], --distanceMeasure=[org.apache.mahout.common.distance.CosineDistanceMeasure], --endPhase=[2147483647], --input=[/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors/], --maxIter=[10], --method=[mapreduce], --numClusters=[20], --output=[/tmp/mahout-work-hudson/reuters-kmeans], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
12/04/05 19:20:36 INFO common.HadoopUtil: Deleting /tmp/mahout-work-hudson/reuters-kmeans-clusters
12/04/05 19:20:36 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
12/04/05 19:20:36 INFO compress.CodecPool: Got brand-new compressor
12/04/05 19:20:37 INFO kmeans.RandomSeedGenerator: Wrote 20 vectors to /tmp/mahout-work-hudson/reuters-kmeans-clusters/part-randomSeed
12/04/05 19:20:37 INFO kmeans.KMeansDriver: Input: /tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors Clusters In: /tmp/mahout-work-hudson/reuters-kmeans-clusters/part-randomSeed Out: /tmp/mahout-work-hudson/reuters-kmeans Distance: org.apache.mahout.common.distance.CosineDistanceMeasure
12/04/05 19:20:37 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10 num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}
12/04/05 19:20:37 INFO compress.CodecPool: Got brand-new decompressor
Exception in thread "main" java.io.IOException: Mkdirs failed to create /tmp/mahout-work-hudson/reuters-kmeans-clusters/part-randomSeed/clusters-0
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:366)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:528)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:843)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:831)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:823)
at org.apache.mahout.clustering.classify.ClusterClassifier.writePolicy(ClusterClassifier.java:232)
at org.apache.mahout.clustering.classify.ClusterClassifier.writeToSeqFiles(ClusterClassifier.java:185)
at org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:254)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:154)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:104)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.mahout.clustering.kmeans.KMeansDriver.main(KMeansDriver.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: Mahout-Examples-Cluster-Reuters #92
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/92/>
------------------------------------------
[...truncated 6193 lines...]
12/04/04 19:18:04 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/04 19:18:04 INFO input.FileInputFormat: Total input paths to process : 1
12/04/04 19:18:05 INFO mapred.JobClient: Running job: job_local_0004
12/04/04 19:18:05 INFO mapred.MapTask: io.sort.mb = 100
12/04/04 19:18:05 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/04 19:18:05 INFO mapred.MapTask: record buffer = 262144/327680
12/04/04 19:18:06 INFO mapred.JobClient: map 0% reduce 0%
12/04/04 19:18:06 INFO mapred.MapTask: Starting flush of map output
12/04/04 19:18:06 INFO mapred.MapTask: Finished spill 0
12/04/04 19:18:06 INFO mapred.Task: Task:attempt_local_0004_m_000000_0 is done. And is in the process of commiting
12/04/04 19:18:08 INFO mapred.LocalJobRunner:
12/04/04 19:18:08 INFO mapred.Task: Task 'attempt_local_0004_m_000000_0' done.
12/04/04 19:18:08 INFO mapred.LocalJobRunner:
12/04/04 19:18:08 INFO mapred.Merger: Merging 1 sorted segments
12/04/04 19:18:08 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/04 19:18:08 INFO mapred.LocalJobRunner:
12/04/04 19:18:09 INFO mapred.JobClient: map 100% reduce 0%
12/04/04 19:18:09 INFO mapred.Task: Task:attempt_local_0004_r_000000_0 is done. And is in the process of commiting
12/04/04 19:18:09 INFO mapred.LocalJobRunner:
12/04/04 19:18:09 INFO mapred.Task: Task attempt_local_0004_r_000000_0 is allowed to commit now
12/04/04 19:18:09 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0004_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/04 19:18:11 INFO mapred.LocalJobRunner: reduce > reduce
12/04/04 19:18:11 INFO mapred.Task: Task 'attempt_local_0004_r_000000_0' done.
12/04/04 19:18:12 INFO mapred.JobClient: map 100% reduce 100%
12/04/04 19:18:12 INFO mapred.JobClient: Job complete: job_local_0004
12/04/04 19:18:12 INFO mapred.JobClient: Counters: 16
12/04/04 19:18:12 INFO mapred.JobClient: File Output Format Counters
12/04/04 19:18:12 INFO mapred.JobClient: Bytes Written=17780621
12/04/04 19:18:12 INFO mapred.JobClient: FileSystemCounters
12/04/04 19:18:12 INFO mapred.JobClient: FILE_BYTES_READ=392447894
12/04/04 19:18:12 INFO mapred.JobClient: FILE_BYTES_WRITTEN=361928347
12/04/04 19:18:12 INFO mapred.JobClient: File Input Format Counters
12/04/04 19:18:12 INFO mapred.JobClient: Bytes Read=17780621
12/04/04 19:18:12 INFO mapred.JobClient: Map-Reduce Framework
12/04/04 19:18:12 INFO mapred.JobClient: Reduce input groups=21578
12/04/04 19:18:12 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/04 19:18:12 INFO mapred.JobClient: Combine output records=0
12/04/04 19:18:12 INFO mapred.JobClient: Map input records=21578
12/04/04 19:18:12 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/04 19:18:12 INFO mapred.JobClient: Reduce output records=21578
12/04/04 19:18:12 INFO mapred.JobClient: Spilled Records=43156
12/04/04 19:18:12 INFO mapred.JobClient: Map output bytes=17337483
12/04/04 19:18:12 INFO mapred.JobClient: Combine input records=0
12/04/04 19:18:12 INFO mapred.JobClient: Map output records=21578
12/04/04 19:18:12 INFO mapred.JobClient: SPLIT_RAW_BYTES=158
12/04/04 19:18:12 INFO mapred.JobClient: Reduce input records=21578
12/04/04 19:18:12 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/04 19:18:12 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/df-count
12/04/04 19:18:12 INFO input.FileInputFormat: Total input paths to process : 1
12/04/04 19:18:12 INFO mapred.JobClient: Running job: job_local_0005
12/04/04 19:18:12 INFO mapred.MapTask: io.sort.mb = 100
12/04/04 19:18:12 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/04 19:18:12 INFO mapred.MapTask: record buffer = 262144/327680
12/04/04 19:18:12 INFO mapred.MapTask: Spilling map output: record full = true
12/04/04 19:18:12 INFO mapred.MapTask: bufstart = 0; bufend = 3145728; bufvoid = 99614720
12/04/04 19:18:12 INFO mapred.MapTask: kvstart = 0; kvend = 262144; length = 327680
12/04/04 19:18:12 INFO mapred.MapTask: Finished spill 0
12/04/04 19:18:13 INFO mapred.MapTask: Spilling map output: record full = true
12/04/04 19:18:13 INFO mapred.MapTask: bufstart = 3145728; bufend = 6291444; bufvoid = 99614720
12/04/04 19:18:13 INFO mapred.MapTask: kvstart = 262144; kvend = 196607; length = 327680
12/04/04 19:18:13 INFO mapred.JobClient: map 0% reduce 0%
12/04/04 19:18:13 INFO mapred.MapTask: Finished spill 1
12/04/04 19:18:13 INFO mapred.MapTask: Spilling map output: record full = true
12/04/04 19:18:13 INFO mapred.MapTask: bufstart = 6291444; bufend = 9437160; bufvoid = 99614720
12/04/04 19:18:13 INFO mapred.MapTask: kvstart = 196607; kvend = 131070; length = 327680
12/04/04 19:18:13 INFO mapred.MapTask: Finished spill 2
12/04/04 19:18:14 INFO mapred.MapTask: Spilling map output: record full = true
12/04/04 19:18:14 INFO mapred.MapTask: bufstart = 9437160; bufend = 12582876; bufvoid = 99614720
12/04/04 19:18:14 INFO mapred.MapTask: kvstart = 131070; kvend = 65533; length = 327680
12/04/04 19:18:14 INFO mapred.MapTask: Finished spill 3
12/04/04 19:18:14 INFO mapred.MapTask: Spilling map output: record full = true
12/04/04 19:18:14 INFO mapred.MapTask: bufstart = 12582876; bufend = 15728604; bufvoid = 99614720
12/04/04 19:18:14 INFO mapred.MapTask: kvstart = 65533; kvend = 327677; length = 327680
12/04/04 19:18:14 INFO mapred.MapTask: Finished spill 4
12/04/04 19:18:15 INFO mapred.MapTask: Starting flush of map output
12/04/04 19:18:15 INFO mapred.MapTask: Finished spill 5
12/04/04 19:18:15 INFO mapred.Merger: Merging 6 sorted segments
12/04/04 19:18:15 INFO mapred.Merger: Down to the last merge-pass, with 6 segments left of total size: 1871742 bytes
12/04/04 19:18:15 INFO mapred.Task: Task:attempt_local_0005_m_000000_0 is done. And is in the process of commiting
12/04/04 19:18:18 INFO mapred.LocalJobRunner:
12/04/04 19:18:18 INFO mapred.LocalJobRunner:
12/04/04 19:18:18 INFO mapred.Task: Task 'attempt_local_0005_m_000000_0' done.
12/04/04 19:18:18 INFO mapred.LocalJobRunner:
12/04/04 19:18:18 INFO mapred.Merger: Merging 1 sorted segments
12/04/04 19:18:18 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 585314 bytes
12/04/04 19:18:18 INFO mapred.LocalJobRunner:
12/04/04 19:18:18 INFO mapred.Task: Task:attempt_local_0005_r_000000_0 is done. And is in the process of commiting
12/04/04 19:18:18 INFO mapred.LocalJobRunner:
12/04/04 19:18:18 INFO mapred.Task: Task attempt_local_0005_r_000000_0 is allowed to commit now
12/04/04 19:18:18 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0005_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/df-count
12/04/04 19:18:19 INFO mapred.JobClient: map 100% reduce 0%
12/04/04 19:18:21 INFO mapred.LocalJobRunner: reduce > reduce
12/04/04 19:18:21 INFO mapred.Task: Task 'attempt_local_0005_r_000000_0' done.
12/04/04 19:18:22 INFO mapred.JobClient: map 100% reduce 100%
12/04/04 19:18:22 INFO mapred.JobClient: Job complete: job_local_0005
12/04/04 19:18:22 INFO mapred.JobClient: Counters: 16
12/04/04 19:18:22 INFO mapred.JobClient: File Output Format Counters
12/04/04 19:18:22 INFO mapred.JobClient: Bytes Written=851221
12/04/04 19:18:22 INFO mapred.JobClient: FileSystemCounters
12/04/04 19:18:22 INFO mapred.JobClient: FILE_BYTES_READ=498030588
12/04/04 19:18:22 INFO mapred.JobClient: FILE_BYTES_WRITTEN=434188971
12/04/04 19:18:22 INFO mapred.JobClient: File Input Format Counters
12/04/04 19:18:22 INFO mapred.JobClient: Bytes Read=17780621
12/04/04 19:18:22 INFO mapred.JobClient: Map-Reduce Framework
12/04/04 19:18:22 INFO mapred.JobClient: Reduce input groups=41808
12/04/04 19:18:22 INFO mapred.JobClient: Map output materialized bytes=585318
12/04/04 19:18:22 INFO mapred.JobClient: Combine output records=175503
12/04/04 19:18:22 INFO mapred.JobClient: Map input records=21578
12/04/04 19:18:22 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/04 19:18:22 INFO mapred.JobClient: Reduce output records=41808
12/04/04 19:18:22 INFO mapred.JobClient: Spilled Records=217311
12/04/04 19:18:22 INFO mapred.JobClient: Map output bytes=18371184
12/04/04 19:18:22 INFO mapred.JobClient: Combine input records=1664627
12/04/04 19:18:22 INFO mapred.JobClient: Map output records=1530932
12/04/04 19:18:22 INFO mapred.JobClient: SPLIT_RAW_BYTES=151
12/04/04 19:18:22 INFO mapred.JobClient: Reduce input records=41808
12/04/04 19:18:22 INFO input.FileInputFormat: Total input paths to process : 1
12/04/04 19:18:22 INFO filecache.TrackerDistributedCacheManager: Creating frequency.file-0 in /tmp/hadoop-jenkins/mapred/local/archive/-3059237245597021655_-299119410_2127240860/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans-work-6911088201205305321 with rwxr-xr-x
12/04/04 19:18:22 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-jenkins/mapred/local/archive/-3059237245597021655_-299119410_2127240860/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/04 19:18:22 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-jenkins/mapred/local/archive/-3059237245597021655_-299119410_2127240860/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/04 19:18:22 INFO mapred.JobClient: Running job: job_local_0006
12/04/04 19:18:22 INFO mapred.MapTask: io.sort.mb = 100
12/04/04 19:18:22 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/04 19:18:22 INFO mapred.MapTask: record buffer = 262144/327680
12/04/04 19:18:23 INFO mapred.MapTask: Starting flush of map output
12/04/04 19:18:23 INFO mapred.JobClient: map 0% reduce 0%
12/04/04 19:18:23 INFO mapred.MapTask: Finished spill 0
12/04/04 19:18:23 INFO mapred.Task: Task:attempt_local_0006_m_000000_0 is done. And is in the process of commiting
12/04/04 19:18:25 INFO mapred.LocalJobRunner:
12/04/04 19:18:25 INFO mapred.Task: Task 'attempt_local_0006_m_000000_0' done.
12/04/04 19:18:25 INFO mapred.LocalJobRunner:
12/04/04 19:18:25 INFO mapred.Merger: Merging 1 sorted segments
12/04/04 19:18:25 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/04 19:18:25 INFO mapred.LocalJobRunner:
12/04/04 19:18:26 INFO mapred.Task: Task:attempt_local_0006_r_000000_0 is done. And is in the process of commiting
12/04/04 19:18:26 INFO mapred.LocalJobRunner:
12/04/04 19:18:26 INFO mapred.Task: Task attempt_local_0006_r_000000_0 is allowed to commit now
12/04/04 19:18:26 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0006_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/04 19:18:26 INFO mapred.JobClient: map 100% reduce 0%
12/04/04 19:18:28 INFO mapred.LocalJobRunner: reduce > reduce
12/04/04 19:18:28 INFO mapred.Task: Task 'attempt_local_0006_r_000000_0' done.
12/04/04 19:18:29 INFO mapred.JobClient: map 100% reduce 100%
12/04/04 19:18:29 INFO mapred.JobClient: Job complete: job_local_0006
12/04/04 19:18:29 INFO mapred.JobClient: Counters: 16
12/04/04 19:18:29 INFO mapred.JobClient: File Output Format Counters
12/04/04 19:18:29 INFO mapred.JobClient: Bytes Written=4914503
12/04/04 19:18:29 INFO mapred.JobClient: FileSystemCounters
12/04/04 19:18:29 INFO mapred.JobClient: FILE_BYTES_READ=604125795
12/04/04 19:18:29 INFO mapred.JobClient: FILE_BYTES_WRITTEN=526918411
12/04/04 19:18:29 INFO mapred.JobClient: File Input Format Counters
12/04/04 19:18:29 INFO mapred.JobClient: Bytes Read=17780621
12/04/04 19:18:29 INFO mapred.JobClient: Map-Reduce Framework
12/04/04 19:18:29 INFO mapred.JobClient: Reduce input groups=21578
12/04/04 19:18:29 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/04 19:18:29 INFO mapred.JobClient: Combine output records=0
12/04/04 19:18:29 INFO mapred.JobClient: Map input records=21578
12/04/04 19:18:29 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/04 19:18:29 INFO mapred.JobClient: Reduce output records=21578
12/04/04 19:18:29 INFO mapred.JobClient: Spilled Records=43156
12/04/04 19:18:29 INFO mapred.JobClient: Map output bytes=17337483
12/04/04 19:18:29 INFO mapred.JobClient: Combine input records=0
12/04/04 19:18:29 INFO mapred.JobClient: Map output records=21578
12/04/04 19:18:29 INFO mapred.JobClient: SPLIT_RAW_BYTES=151
12/04/04 19:18:29 INFO mapred.JobClient: Reduce input records=21578
12/04/04 19:18:29 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/04 19:18:29 INFO input.FileInputFormat: Total input paths to process : 1
12/04/04 19:18:29 INFO mapred.JobClient: Running job: job_local_0007
12/04/04 19:18:29 INFO mapred.MapTask: io.sort.mb = 100
12/04/04 19:18:29 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/04 19:18:29 INFO mapred.MapTask: record buffer = 262144/327680
12/04/04 19:18:30 INFO mapred.MapTask: Starting flush of map output
12/04/04 19:18:30 INFO mapred.MapTask: Finished spill 0
12/04/04 19:18:30 INFO mapred.Task: Task:attempt_local_0007_m_000000_0 is done. And is in the process of commiting
12/04/04 19:18:30 INFO mapred.JobClient: map 0% reduce 0%
12/04/04 19:18:32 INFO mapred.LocalJobRunner:
12/04/04 19:18:32 INFO mapred.Task: Task 'attempt_local_0007_m_000000_0' done.
12/04/04 19:18:32 INFO mapred.LocalJobRunner:
12/04/04 19:18:32 INFO mapred.Merger: Merging 1 sorted segments
12/04/04 19:18:32 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 4719200 bytes
12/04/04 19:18:32 INFO mapred.LocalJobRunner:
12/04/04 19:18:33 INFO mapred.Task: Task:attempt_local_0007_r_000000_0 is done. And is in the process of commiting
12/04/04 19:18:33 INFO mapred.LocalJobRunner:
12/04/04 19:18:33 INFO mapred.Task: Task attempt_local_0007_r_000000_0 is allowed to commit now
12/04/04 19:18:33 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0007_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/04 19:18:33 INFO mapred.JobClient: map 100% reduce 0%
12/04/04 19:18:35 INFO mapred.LocalJobRunner: reduce > reduce
12/04/04 19:18:35 INFO mapred.Task: Task 'attempt_local_0007_r_000000_0' done.
12/04/04 19:18:36 INFO mapred.JobClient: map 100% reduce 100%
12/04/04 19:18:36 INFO mapred.JobClient: Job complete: job_local_0007
12/04/04 19:18:36 INFO mapred.JobClient: Counters: 16
12/04/04 19:18:36 INFO mapred.JobClient: File Output Format Counters
12/04/04 19:18:36 INFO mapred.JobClient: Bytes Written=4914503
12/04/04 19:18:36 INFO mapred.JobClient: FileSystemCounters
12/04/04 19:18:36 INFO mapred.JobClient: FILE_BYTES_READ=685217820
12/04/04 19:18:36 INFO mapred.JobClient: FILE_BYTES_WRITTEN=594900999
12/04/04 19:18:36 INFO mapred.JobClient: File Input Format Counters
12/04/04 19:18:36 INFO mapred.JobClient: Bytes Read=4914503
12/04/04 19:18:36 INFO mapred.JobClient: Map-Reduce Framework
12/04/04 19:18:36 INFO mapred.JobClient: Reduce input groups=21578
12/04/04 19:18:36 INFO mapred.JobClient: Map output materialized bytes=4719204
12/04/04 19:18:36 INFO mapred.JobClient: Combine output records=0
12/04/04 19:18:36 INFO mapred.JobClient: Map input records=21578
12/04/04 19:18:36 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/04 19:18:36 INFO mapred.JobClient: Reduce output records=21578
12/04/04 19:18:36 INFO mapred.JobClient: Spilled Records=43156
12/04/04 19:18:36 INFO mapred.JobClient: Map output bytes=4659281
12/04/04 19:18:36 INFO mapred.JobClient: Combine input records=0
12/04/04 19:18:36 INFO mapred.JobClient: Map output records=21578
12/04/04 19:18:36 INFO mapred.JobClient: SPLIT_RAW_BYTES=158
12/04/04 19:18:36 INFO mapred.JobClient: Reduce input records=21578
12/04/04 19:18:36 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/04 19:18:36 INFO driver.MahoutDriver: Program took 78939 ms (Minutes: 1.31565)
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
no HADOOP_HOME set, running locally
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/mahout-examples-0.7-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-jcl-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
12/04/04 19:18:37 INFO common.AbstractJob: Command line arguments: {--clustering=null, --clusters=[/tmp/mahout-work-jenkins/reuters-kmeans-clusters], --convergenceDelta=[0.5], --distanceMeasure=[org.apache.mahout.common.distance.CosineDistanceMeasure], --endPhase=[2147483647], --input=[/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors/], --maxIter=[10], --method=[mapreduce], --numClusters=[20], --output=[/tmp/mahout-work-jenkins/reuters-kmeans], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
12/04/04 19:18:37 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-kmeans
12/04/04 19:18:37 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-kmeans-clusters
12/04/04 19:18:37 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
12/04/04 19:18:37 INFO compress.CodecPool: Got brand-new compressor
12/04/04 19:18:38 INFO kmeans.RandomSeedGenerator: Wrote 20 vectors to /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed
12/04/04 19:18:38 INFO kmeans.KMeansDriver: Input: /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors Clusters In: /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed Out: /tmp/mahout-work-jenkins/reuters-kmeans Distance: org.apache.mahout.common.distance.CosineDistanceMeasure
12/04/04 19:18:38 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10 num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}
12/04/04 19:18:38 INFO compress.CodecPool: Got brand-new decompressor
Exception in thread "main" java.io.IOException: Mkdirs failed to create /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed/clusters-0
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:366)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:528)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:843)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:831)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:823)
at org.apache.mahout.clustering.classify.ClusterClassifier.writePolicy(ClusterClassifier.java:232)
at org.apache.mahout.clustering.classify.ClusterClassifier.writeToSeqFiles(ClusterClassifier.java:185)
at org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:254)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:154)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:104)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.mahout.clustering.kmeans.KMeansDriver.main(KMeansDriver.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: Mahout-Examples-Cluster-Reuters #91
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/91/>
------------------------------------------
[...truncated 6193 lines...]
12/04/03 20:01:29 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/03 20:01:29 INFO input.FileInputFormat: Total input paths to process : 1
12/04/03 20:01:29 INFO mapred.JobClient: Running job: job_local_0004
12/04/03 20:01:29 INFO mapred.MapTask: io.sort.mb = 100
12/04/03 20:01:29 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/03 20:01:29 INFO mapred.MapTask: record buffer = 262144/327680
12/04/03 20:01:30 INFO mapred.JobClient: map 0% reduce 0%
12/04/03 20:01:30 INFO mapred.MapTask: Starting flush of map output
12/04/03 20:01:31 INFO mapred.MapTask: Finished spill 0
12/04/03 20:01:31 INFO mapred.Task: Task:attempt_local_0004_m_000000_0 is done. And is in the process of commiting
12/04/03 20:01:32 INFO mapred.LocalJobRunner:
12/04/03 20:01:32 INFO mapred.Task: Task 'attempt_local_0004_m_000000_0' done.
12/04/03 20:01:32 INFO mapred.LocalJobRunner:
12/04/03 20:01:32 INFO mapred.Merger: Merging 1 sorted segments
12/04/03 20:01:32 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/03 20:01:32 INFO mapred.LocalJobRunner:
12/04/03 20:01:33 INFO mapred.JobClient: map 100% reduce 0%
12/04/03 20:01:33 INFO mapred.Task: Task:attempt_local_0004_r_000000_0 is done. And is in the process of commiting
12/04/03 20:01:33 INFO mapred.LocalJobRunner:
12/04/03 20:01:33 INFO mapred.Task: Task attempt_local_0004_r_000000_0 is allowed to commit now
12/04/03 20:01:33 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0004_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tf-vectors
12/04/03 20:01:35 INFO mapred.LocalJobRunner: reduce > reduce
12/04/03 20:01:35 INFO mapred.Task: Task 'attempt_local_0004_r_000000_0' done.
12/04/03 20:01:36 INFO mapred.JobClient: map 100% reduce 100%
12/04/03 20:01:36 INFO mapred.JobClient: Job complete: job_local_0004
12/04/03 20:01:36 INFO mapred.JobClient: Counters: 16
12/04/03 20:01:36 INFO mapred.JobClient: File Output Format Counters
12/04/03 20:01:36 INFO mapred.JobClient: Bytes Written=17780621
12/04/03 20:01:36 INFO mapred.JobClient: FileSystemCounters
12/04/03 20:01:36 INFO mapred.JobClient: FILE_BYTES_READ=392464502
12/04/03 20:01:36 INFO mapred.JobClient: FILE_BYTES_WRITTEN=361943185
12/04/03 20:01:36 INFO mapred.JobClient: File Input Format Counters
12/04/03 20:01:36 INFO mapred.JobClient: Bytes Read=17780621
12/04/03 20:01:36 INFO mapred.JobClient: Map-Reduce Framework
12/04/03 20:01:36 INFO mapred.JobClient: Reduce input groups=21578
12/04/03 20:01:36 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/03 20:01:36 INFO mapred.JobClient: Combine output records=0
12/04/03 20:01:36 INFO mapred.JobClient: Map input records=21578
12/04/03 20:01:36 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/03 20:01:36 INFO mapred.JobClient: Reduce output records=21578
12/04/03 20:01:36 INFO mapred.JobClient: Spilled Records=43156
12/04/03 20:01:36 INFO mapred.JobClient: Map output bytes=17337483
12/04/03 20:01:36 INFO mapred.JobClient: Combine input records=0
12/04/03 20:01:36 INFO mapred.JobClient: Map output records=21578
12/04/03 20:01:36 INFO mapred.JobClient: SPLIT_RAW_BYTES=158
12/04/03 20:01:36 INFO mapred.JobClient: Reduce input records=21578
12/04/03 20:01:36 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/03 20:01:36 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/df-count
12/04/03 20:01:37 INFO input.FileInputFormat: Total input paths to process : 1
12/04/03 20:01:37 INFO mapred.JobClient: Running job: job_local_0005
12/04/03 20:01:37 INFO mapred.MapTask: io.sort.mb = 100
12/04/03 20:01:37 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/03 20:01:37 INFO mapred.MapTask: record buffer = 262144/327680
12/04/03 20:01:37 INFO mapred.MapTask: Spilling map output: record full = true
12/04/03 20:01:37 INFO mapred.MapTask: bufstart = 0; bufend = 3145728; bufvoid = 99614720
12/04/03 20:01:37 INFO mapred.MapTask: kvstart = 0; kvend = 262144; length = 327680
12/04/03 20:01:37 INFO mapred.MapTask: Finished spill 0
12/04/03 20:01:38 INFO mapred.MapTask: Spilling map output: record full = true
12/04/03 20:01:38 INFO mapred.MapTask: bufstart = 3145728; bufend = 6291444; bufvoid = 99614720
12/04/03 20:01:38 INFO mapred.MapTask: kvstart = 262144; kvend = 196607; length = 327680
12/04/03 20:01:38 INFO mapred.JobClient: map 0% reduce 0%
12/04/03 20:01:38 INFO mapred.MapTask: Finished spill 1
12/04/03 20:01:38 INFO mapred.MapTask: Spilling map output: record full = true
12/04/03 20:01:38 INFO mapred.MapTask: bufstart = 6291444; bufend = 9437160; bufvoid = 99614720
12/04/03 20:01:38 INFO mapred.MapTask: kvstart = 196607; kvend = 131070; length = 327680
12/04/03 20:01:38 INFO mapred.MapTask: Finished spill 2
12/04/03 20:01:38 INFO mapred.MapTask: Spilling map output: record full = true
12/04/03 20:01:38 INFO mapred.MapTask: bufstart = 9437160; bufend = 12582876; bufvoid = 99614720
12/04/03 20:01:38 INFO mapred.MapTask: kvstart = 131070; kvend = 65533; length = 327680
12/04/03 20:01:39 INFO mapred.MapTask: Finished spill 3
12/04/03 20:01:39 INFO mapred.MapTask: Spilling map output: record full = true
12/04/03 20:01:39 INFO mapred.MapTask: bufstart = 12582876; bufend = 15728604; bufvoid = 99614720
12/04/03 20:01:39 INFO mapred.MapTask: kvstart = 65533; kvend = 327677; length = 327680
12/04/03 20:01:39 INFO mapred.MapTask: Finished spill 4
12/04/03 20:01:39 INFO mapred.MapTask: Starting flush of map output
12/04/03 20:01:39 INFO mapred.MapTask: Finished spill 5
12/04/03 20:01:40 INFO mapred.Merger: Merging 6 sorted segments
12/04/03 20:01:40 INFO mapred.Merger: Down to the last merge-pass, with 6 segments left of total size: 1871742 bytes
12/04/03 20:01:40 INFO mapred.Task: Task:attempt_local_0005_m_000000_0 is done. And is in the process of commiting
12/04/03 20:01:43 INFO mapred.LocalJobRunner:
12/04/03 20:01:43 INFO mapred.LocalJobRunner:
12/04/03 20:01:43 INFO mapred.Task: Task 'attempt_local_0005_m_000000_0' done.
12/04/03 20:01:43 INFO mapred.LocalJobRunner:
12/04/03 20:01:43 INFO mapred.Merger: Merging 1 sorted segments
12/04/03 20:01:43 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 585314 bytes
12/04/03 20:01:43 INFO mapred.LocalJobRunner:
12/04/03 20:01:43 INFO mapred.Task: Task:attempt_local_0005_r_000000_0 is done. And is in the process of commiting
12/04/03 20:01:43 INFO mapred.LocalJobRunner:
12/04/03 20:01:43 INFO mapred.Task: Task attempt_local_0005_r_000000_0 is allowed to commit now
12/04/03 20:01:43 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0005_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/df-count
12/04/03 20:01:44 INFO mapred.JobClient: map 100% reduce 0%
12/04/03 20:01:46 INFO mapred.LocalJobRunner: reduce > reduce
12/04/03 20:01:46 INFO mapred.Task: Task 'attempt_local_0005_r_000000_0' done.
12/04/03 20:01:47 INFO mapred.JobClient: map 100% reduce 100%
12/04/03 20:01:47 INFO mapred.JobClient: Job complete: job_local_0005
12/04/03 20:01:47 INFO mapred.JobClient: Counters: 16
12/04/03 20:01:47 INFO mapred.JobClient: File Output Format Counters
12/04/03 20:01:47 INFO mapred.JobClient: Bytes Written=851221
12/04/03 20:01:47 INFO mapred.JobClient: FileSystemCounters
12/04/03 20:01:47 INFO mapred.JobClient: FILE_BYTES_READ=498047194
12/04/03 20:01:47 INFO mapred.JobClient: FILE_BYTES_WRITTEN=434203807
12/04/03 20:01:47 INFO mapred.JobClient: File Input Format Counters
12/04/03 20:01:47 INFO mapred.JobClient: Bytes Read=17780621
12/04/03 20:01:47 INFO mapred.JobClient: Map-Reduce Framework
12/04/03 20:01:47 INFO mapred.JobClient: Reduce input groups=41808
12/04/03 20:01:47 INFO mapred.JobClient: Map output materialized bytes=585318
12/04/03 20:01:47 INFO mapred.JobClient: Combine output records=175503
12/04/03 20:01:47 INFO mapred.JobClient: Map input records=21578
12/04/03 20:01:47 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/03 20:01:47 INFO mapred.JobClient: Reduce output records=41808
12/04/03 20:01:47 INFO mapred.JobClient: Spilled Records=217311
12/04/03 20:01:47 INFO mapred.JobClient: Map output bytes=18371184
12/04/03 20:01:47 INFO mapred.JobClient: Combine input records=1664627
12/04/03 20:01:47 INFO mapred.JobClient: Map output records=1530932
12/04/03 20:01:47 INFO mapred.JobClient: SPLIT_RAW_BYTES=151
12/04/03 20:01:47 INFO mapred.JobClient: Reduce input records=41808
12/04/03 20:01:47 INFO input.FileInputFormat: Total input paths to process : 1
12/04/03 20:01:47 INFO filecache.TrackerDistributedCacheManager: Creating frequency.file-0 in /tmp/hadoop-jenkins/mapred/local/archive/-8321305666425987227_-299119410_2043445860/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans-work--1332694924927851244 with rwxr-xr-x
12/04/03 20:01:47 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-jenkins/mapred/local/archive/-8321305666425987227_-299119410_2043445860/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/03 20:01:47 INFO filecache.TrackerDistributedCacheManager: Cached /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as /tmp/hadoop-jenkins/mapred/local/archive/-8321305666425987227_-299119410_2043445860/file/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/frequency.file-0
12/04/03 20:01:47 INFO mapred.JobClient: Running job: job_local_0006
12/04/03 20:01:47 INFO mapred.MapTask: io.sort.mb = 100
12/04/03 20:01:47 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/03 20:01:47 INFO mapred.MapTask: record buffer = 262144/327680
12/04/03 20:01:48 INFO mapred.MapTask: Starting flush of map output
12/04/03 20:01:48 INFO mapred.JobClient: map 0% reduce 0%
12/04/03 20:01:48 INFO mapred.MapTask: Finished spill 0
12/04/03 20:01:48 INFO mapred.Task: Task:attempt_local_0006_m_000000_0 is done. And is in the process of commiting
12/04/03 20:01:50 INFO mapred.LocalJobRunner:
12/04/03 20:01:50 INFO mapred.Task: Task 'attempt_local_0006_m_000000_0' done.
12/04/03 20:01:50 INFO mapred.LocalJobRunner:
12/04/03 20:01:50 INFO mapred.Merger: Merging 1 sorted segments
12/04/03 20:01:50 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
12/04/03 20:01:50 INFO mapred.LocalJobRunner:
12/04/03 20:01:51 INFO mapred.Task: Task:attempt_local_0006_r_000000_0 is done. And is in the process of commiting
12/04/03 20:01:51 INFO mapred.LocalJobRunner:
12/04/03 20:01:51 INFO mapred.Task: Task attempt_local_0006_r_000000_0 is allowed to commit now
12/04/03 20:01:51 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0006_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/03 20:01:51 INFO mapred.JobClient: map 100% reduce 0%
12/04/03 20:01:53 INFO mapred.LocalJobRunner: reduce > reduce
12/04/03 20:01:53 INFO mapred.Task: Task 'attempt_local_0006_r_000000_0' done.
12/04/03 20:01:54 INFO mapred.JobClient: map 100% reduce 100%
12/04/03 20:01:54 INFO mapred.JobClient: Job complete: job_local_0006
12/04/03 20:01:54 INFO mapred.JobClient: Counters: 16
12/04/03 20:01:54 INFO mapred.JobClient: File Output Format Counters
12/04/03 20:01:54 INFO mapred.JobClient: Bytes Written=4914503
12/04/03 20:01:54 INFO mapred.JobClient: FileSystemCounters
12/04/03 20:01:54 INFO mapred.JobClient: FILE_BYTES_READ=604142399
12/04/03 20:01:54 INFO mapred.JobClient: FILE_BYTES_WRITTEN=526933253
12/04/03 20:01:54 INFO mapred.JobClient: File Input Format Counters
12/04/03 20:01:54 INFO mapred.JobClient: Bytes Read=17780621
12/04/03 20:01:54 INFO mapred.JobClient: Map-Reduce Framework
12/04/03 20:01:54 INFO mapred.JobClient: Reduce input groups=21578
12/04/03 20:01:54 INFO mapred.JobClient: Map output materialized bytes=17420438
12/04/03 20:01:54 INFO mapred.JobClient: Combine output records=0
12/04/03 20:01:54 INFO mapred.JobClient: Map input records=21578
12/04/03 20:01:54 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/03 20:01:54 INFO mapred.JobClient: Reduce output records=21578
12/04/03 20:01:54 INFO mapred.JobClient: Spilled Records=43156
12/04/03 20:01:54 INFO mapred.JobClient: Map output bytes=17337483
12/04/03 20:01:54 INFO mapred.JobClient: Combine input records=0
12/04/03 20:01:54 INFO mapred.JobClient: Map output records=21578
12/04/03 20:01:54 INFO mapred.JobClient: SPLIT_RAW_BYTES=151
12/04/03 20:01:54 INFO mapred.JobClient: Reduce input records=21578
12/04/03 20:01:54 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/03 20:01:54 INFO input.FileInputFormat: Total input paths to process : 1
12/04/03 20:01:54 INFO mapred.JobClient: Running job: job_local_0007
12/04/03 20:01:54 INFO mapred.MapTask: io.sort.mb = 100
12/04/03 20:01:54 INFO mapred.MapTask: data buffer = 79691776/99614720
12/04/03 20:01:54 INFO mapred.MapTask: record buffer = 262144/327680
12/04/03 20:01:55 INFO mapred.MapTask: Starting flush of map output
12/04/03 20:01:55 INFO mapred.MapTask: Finished spill 0
12/04/03 20:01:55 INFO mapred.Task: Task:attempt_local_0007_m_000000_0 is done. And is in the process of commiting
12/04/03 20:01:55 INFO mapred.JobClient: map 0% reduce 0%
12/04/03 20:01:57 INFO mapred.LocalJobRunner:
12/04/03 20:01:57 INFO mapred.Task: Task 'attempt_local_0007_m_000000_0' done.
12/04/03 20:01:57 INFO mapred.LocalJobRunner:
12/04/03 20:01:57 INFO mapred.Merger: Merging 1 sorted segments
12/04/03 20:01:57 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 4719200 bytes
12/04/03 20:01:57 INFO mapred.LocalJobRunner:
12/04/03 20:01:58 INFO mapred.Task: Task:attempt_local_0007_r_000000_0 is done. And is in the process of commiting
12/04/03 20:01:58 INFO mapred.LocalJobRunner:
12/04/03 20:01:58 INFO mapred.Task: Task attempt_local_0007_r_000000_0 is allowed to commit now
12/04/03 20:01:58 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0007_r_000000_0' to /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
12/04/03 20:01:58 INFO mapred.JobClient: map 100% reduce 0%
12/04/03 20:02:00 INFO mapred.LocalJobRunner: reduce > reduce
12/04/03 20:02:00 INFO mapred.Task: Task 'attempt_local_0007_r_000000_0' done.
12/04/03 20:02:01 INFO mapred.JobClient: map 100% reduce 100%
12/04/03 20:02:01 INFO mapred.JobClient: Job complete: job_local_0007
12/04/03 20:02:01 INFO mapred.JobClient: Counters: 16
12/04/03 20:02:01 INFO mapred.JobClient: File Output Format Counters
12/04/03 20:02:01 INFO mapred.JobClient: Bytes Written=4914503
12/04/03 20:02:01 INFO mapred.JobClient: FileSystemCounters
12/04/03 20:02:01 INFO mapred.JobClient: FILE_BYTES_READ=685234422
12/04/03 20:02:01 INFO mapred.JobClient: FILE_BYTES_WRITTEN=594915847
12/04/03 20:02:01 INFO mapred.JobClient: File Input Format Counters
12/04/03 20:02:01 INFO mapred.JobClient: Bytes Read=4914503
12/04/03 20:02:01 INFO mapred.JobClient: Map-Reduce Framework
12/04/03 20:02:01 INFO mapred.JobClient: Reduce input groups=21578
12/04/03 20:02:01 INFO mapred.JobClient: Map output materialized bytes=4719204
12/04/03 20:02:01 INFO mapred.JobClient: Combine output records=0
12/04/03 20:02:01 INFO mapred.JobClient: Map input records=21578
12/04/03 20:02:01 INFO mapred.JobClient: Reduce shuffle bytes=0
12/04/03 20:02:01 INFO mapred.JobClient: Reduce output records=21578
12/04/03 20:02:01 INFO mapred.JobClient: Spilled Records=43156
12/04/03 20:02:01 INFO mapred.JobClient: Map output bytes=4659281
12/04/03 20:02:01 INFO mapred.JobClient: Combine input records=0
12/04/03 20:02:01 INFO mapred.JobClient: Map output records=21578
12/04/03 20:02:01 INFO mapred.JobClient: SPLIT_RAW_BYTES=158
12/04/03 20:02:01 INFO mapred.JobClient: Reduce input records=21578
12/04/03 20:02:01 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
12/04/03 20:02:01 INFO driver.MahoutDriver: Program took 78844 ms (Minutes: 1.3140666666666667)
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
no HADOOP_HOME set, running locally
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/mahout-examples-0.7-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-jcl-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/ws/trunk/examples/target/dependency/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
12/04/03 20:02:02 INFO common.AbstractJob: Command line arguments: {--clustering=null, --clusters=[/tmp/mahout-work-jenkins/reuters-kmeans-clusters], --convergenceDelta=[0.5], --distanceMeasure=[org.apache.mahout.common.distance.CosineDistanceMeasure], --endPhase=[2147483647], --input=[/tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors/], --maxIter=[10], --method=[mapreduce], --numClusters=[20], --output=[/tmp/mahout-work-jenkins/reuters-kmeans], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
12/04/03 20:02:02 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-kmeans
12/04/03 20:02:02 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jenkins/reuters-kmeans-clusters
12/04/03 20:02:02 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
12/04/03 20:02:02 INFO compress.CodecPool: Got brand-new compressor
12/04/03 20:02:03 INFO kmeans.RandomSeedGenerator: Wrote 20 vectors to /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed
12/04/03 20:02:03 INFO kmeans.KMeansDriver: Input: /tmp/mahout-work-jenkins/reuters-out-seqdir-sparse-kmeans/tfidf-vectors Clusters In: /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed Out: /tmp/mahout-work-jenkins/reuters-kmeans Distance: org.apache.mahout.common.distance.CosineDistanceMeasure
12/04/03 20:02:03 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10 num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}
12/04/03 20:02:03 INFO compress.CodecPool: Got brand-new decompressor
Exception in thread "main" java.io.IOException: Mkdirs failed to create /tmp/mahout-work-jenkins/reuters-kmeans-clusters/part-randomSeed/clusters-0
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:366)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:528)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:843)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:831)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:823)
at org.apache.mahout.clustering.classify.ClusterClassifier.writePolicy(ClusterClassifier.java:232)
at org.apache.mahout.clustering.classify.ClusterClassifier.writeToSeqFiles(ClusterClassifier.java:185)
at org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:254)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:154)
at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:104)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.mahout.clustering.kmeans.KMeansDriver.main(KMeansDriver.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)
Build step 'Execute shell' marked build as failure