You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Thomas Rummler <th...@gmail.com> on 2018/09/08 15:54:00 UTC
Are Giraph Stats written in log only using map-reduce (hadoop2 mvn
profile) ?
It seems that Giraph Stas are written in log only using map-reduce (
giraph-1.3.0-snapshot built with -Phadoop2 mvn profile). Follows an example
of Giraph Stats.
Is there a way to write Giraph Stats in logs using yarn too
(giraph-1.3.0-snapshot built with -Phadoop_yarn mvn profile) ?
2018-09-08 16:03:01,604 INFO org.apache.hadoop.mapreduce.Job: Counters: 42
File System Counters
FILE: Number of bytes read=74623213
FILE: Number of bytes written=75518013
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=1
Map output records=0
Input split bytes=44
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=100
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
Total committed heap usage (bytes)=3087007744
Giraph Stats
Aggregate bytes loaded from local disks (out-of-core)=0
Aggregate bytes stored to local disks (out-of-core)=0
Aggregate edges=12
Aggregate finished vertices=0
Aggregate sent message bytes=3154
Aggregate sent messages=24
Aggregate vertices=5
Current master task partition=0
Current workers=1
Last checkpointed superstep=0
Lowest percentage of graph in memory so far (out-of-core)=100
Sent message bytes=1701
Sent messages=12
Superstep=2
Giraph Timers
Initialize (ms)=653
Input superstep (ms)=599
Setup (ms)=58
Shutdown (ms)=8967
Superstep 0 Worker_Superstep0 (ms)=295
Superstep 1 Worker_Superstep1 (ms)=605
Total (ms)=10538
Zookeeper base path
/_hadoopBsp/job_local881103217_0001=0
Zookeeper halt node
/_hadoopBsp/job_local881103217_0001/_haltComputation=0
Zookeeper server:port
cri:22181=0
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=0
Re: Are Giraph Stats written in log only using map-reduce (hadoop2
mvn profile) ?
Posted by Francesco Sclano <sc...@gmail.com>.
Use
-ca giraph.metrics.enable=true
as explained in http://giraph.apache.org/options.html
Il giorno sab 8 set 2018 alle ore 17:54 Thomas Rummler <
thomas.rummler.gm@gmail.com> ha scritto:
> It seems that Giraph Stas are written in log only using map-reduce (
> giraph-1.3.0-snapshot built with -Phadoop2 mvn profile). Follows an example
> of Giraph Stats.
>
> Is there a way to write Giraph Stats in logs using yarn too
> (giraph-1.3.0-snapshot built with -Phadoop_yarn mvn profile) ?
>
>
> 2018-09-08 16:03:01,604 INFO org.apache.hadoop.mapreduce.Job: Counters: 42
> File System Counters
> FILE: Number of bytes read=74623213
> FILE: Number of bytes written=75518013
> FILE: Number of read operations=0
> FILE: Number of large read operations=0
> FILE: Number of write operations=0
> Map-Reduce Framework
> Map input records=1
> Map output records=0
> Input split bytes=44
> Spilled Records=0
> Failed Shuffles=0
> Merged Map outputs=0
> GC time elapsed (ms)=100
> CPU time spent (ms)=0
> Physical memory (bytes) snapshot=0
> Virtual memory (bytes) snapshot=0
> Total committed heap usage (bytes)=3087007744
> Giraph Stats
> Aggregate bytes loaded from local disks (out-of-core)=0
> Aggregate bytes stored to local disks (out-of-core)=0
> Aggregate edges=12
> Aggregate finished vertices=0
> Aggregate sent message bytes=3154
> Aggregate sent messages=24
> Aggregate vertices=5
> Current master task partition=0
> Current workers=1
> Last checkpointed superstep=0
> Lowest percentage of graph in memory so far (out-of-core)=100
> Sent message bytes=1701
> Sent messages=12
> Superstep=2
> Giraph Timers
> Initialize (ms)=653
> Input superstep (ms)=599
> Setup (ms)=58
> Shutdown (ms)=8967
> Superstep 0 Worker_Superstep0 (ms)=295
> Superstep 1 Worker_Superstep1 (ms)=605
> Total (ms)=10538
> Zookeeper base path
> /_hadoopBsp/job_local881103217_0001=0
> Zookeeper halt node
> /_hadoopBsp/job_local881103217_0001/_haltComputation=0
> Zookeeper server:port
> cri:22181=0
> File Input Format Counters
> Bytes Read=0
> File Output Format Counters
> Bytes Written=0
>
--
Francesco Sclano