You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Thomas Rummler <th...@gmail.com> on 2018/09/08 15:54:00 UTC

Are Giraph Stats written in log only using map-reduce (hadoop2 mvn profile) ?

It seems that Giraph Stas are written in log only using map-reduce (
giraph-1.3.0-snapshot built with -Phadoop2 mvn profile). Follows an example
of Giraph Stats.

Is there a way to write Giraph Stats in logs using yarn too
(giraph-1.3.0-snapshot built with -Phadoop_yarn mvn profile) ?


2018-09-08 16:03:01,604 INFO org.apache.hadoop.mapreduce.Job: Counters: 42
    File System Counters
        FILE: Number of bytes read=74623213
        FILE: Number of bytes written=75518013
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
    Map-Reduce Framework
        Map input records=1
        Map output records=0
        Input split bytes=44
        Spilled Records=0
        Failed Shuffles=0
        Merged Map outputs=0
        GC time elapsed (ms)=100
        CPU time spent (ms)=0
        Physical memory (bytes) snapshot=0
        Virtual memory (bytes) snapshot=0
        Total committed heap usage (bytes)=3087007744
    Giraph Stats
        Aggregate bytes loaded from local disks (out-of-core)=0
        Aggregate bytes stored to local disks (out-of-core)=0
        Aggregate edges=12
        Aggregate finished vertices=0
        Aggregate sent message bytes=3154
        Aggregate sent messages=24
        Aggregate vertices=5
        Current master task partition=0
        Current workers=1
        Last checkpointed superstep=0
        Lowest percentage of graph in memory so far (out-of-core)=100
        Sent message bytes=1701
        Sent messages=12
        Superstep=2
    Giraph Timers
        Initialize (ms)=653
        Input superstep (ms)=599
        Setup (ms)=58
        Shutdown (ms)=8967
        Superstep 0 Worker_Superstep0 (ms)=295
        Superstep 1 Worker_Superstep1 (ms)=605
        Total (ms)=10538
    Zookeeper base path
        /_hadoopBsp/job_local881103217_0001=0
    Zookeeper halt node
        /_hadoopBsp/job_local881103217_0001/_haltComputation=0
    Zookeeper server:port
        cri:22181=0
    File Input Format Counters
        Bytes Read=0
    File Output Format Counters
        Bytes Written=0

Re: Are Giraph Stats written in log only using map-reduce (hadoop2 mvn profile) ?

Posted by Francesco Sclano <sc...@gmail.com>.
Use

-ca giraph.metrics.enable=true

as explained in http://giraph.apache.org/options.html

Il giorno sab 8 set 2018 alle ore 17:54 Thomas Rummler <
thomas.rummler.gm@gmail.com> ha scritto:

> It seems that Giraph Stas are written in log only using map-reduce (
> giraph-1.3.0-snapshot built with -Phadoop2 mvn profile). Follows an example
> of Giraph Stats.
>
> Is there a way to write Giraph Stats in logs using yarn too
> (giraph-1.3.0-snapshot built with -Phadoop_yarn mvn profile) ?
>
>
> 2018-09-08 16:03:01,604 INFO org.apache.hadoop.mapreduce.Job: Counters: 42
>     File System Counters
>         FILE: Number of bytes read=74623213
>         FILE: Number of bytes written=75518013
>         FILE: Number of read operations=0
>         FILE: Number of large read operations=0
>         FILE: Number of write operations=0
>     Map-Reduce Framework
>         Map input records=1
>         Map output records=0
>         Input split bytes=44
>         Spilled Records=0
>         Failed Shuffles=0
>         Merged Map outputs=0
>         GC time elapsed (ms)=100
>         CPU time spent (ms)=0
>         Physical memory (bytes) snapshot=0
>         Virtual memory (bytes) snapshot=0
>         Total committed heap usage (bytes)=3087007744
>     Giraph Stats
>         Aggregate bytes loaded from local disks (out-of-core)=0
>         Aggregate bytes stored to local disks (out-of-core)=0
>         Aggregate edges=12
>         Aggregate finished vertices=0
>         Aggregate sent message bytes=3154
>         Aggregate sent messages=24
>         Aggregate vertices=5
>         Current master task partition=0
>         Current workers=1
>         Last checkpointed superstep=0
>         Lowest percentage of graph in memory so far (out-of-core)=100
>         Sent message bytes=1701
>         Sent messages=12
>         Superstep=2
>     Giraph Timers
>         Initialize (ms)=653
>         Input superstep (ms)=599
>         Setup (ms)=58
>         Shutdown (ms)=8967
>         Superstep 0 Worker_Superstep0 (ms)=295
>         Superstep 1 Worker_Superstep1 (ms)=605
>         Total (ms)=10538
>     Zookeeper base path
>         /_hadoopBsp/job_local881103217_0001=0
>     Zookeeper halt node
>         /_hadoopBsp/job_local881103217_0001/_haltComputation=0
>     Zookeeper server:port
>         cri:22181=0
>     File Input Format Counters
>         Bytes Read=0
>     File Output Format Counters
>         Bytes Written=0
>


-- 
Francesco Sclano