Hadoop example code (preferred Pagerank and Shortest Path)

[Shuai Zheng <sh...@kaust.edu.sa>]

Hi all,

I am student working on Hadoop 0.20.203.0.

I want to monitor the system resource utilization of pagerank and/or
shortest path hadoop program. I only need to monitor the system, so I did
not write any hadoop programs.

I want to ask for some example codes (with many iterations) and dataset
from you. Algorithms such as:
1. Pagerank
2. Shortest path

(This is not necessary) it is better that the program can output log info
for iteration start time and stop time.

If you know any public example code, or you have your own code, and would
like to share with me (in case any related publication using your code in
the future, your contribution will be acknowledged.), please send me a copy.

Thank you and best regards,

Shuai

Re: Hadoop example code (preferred Pagerank and Shortest Path)

[Ravi Prakash <ra...@gmail.com>]

Shuai,

I'm afraid I don't know if there are open-sourced implementations of those
two algorithms, but there are a bunch of example programs in the
hadoop-mapreduce-examples*.jar that gets built / distributed.

Valid program names are:
  aggregatewordcount: An Aggregate based map/reduce program that counts the
words in the input files.
  aggregatewordhist: An Aggregate based map/reduce program that computes
the histogram of the words in the input files.
  dbcount: An example job that count the pageview counts from a database.
  grep: A map/reduce program that counts the matches of a regex in the
input.
  join: A job that effects a join over sorted, equally partitioned datasets
  multifilewc: A job that counts words from several files.
  pentomino: A map/reduce tile laying program to find solutions to
pentomino problems.
  pi: A map/reduce program that estimates Pi using monte-carlo method.
  randomtextwriter: A map/reduce program that writes 10GB of random textual
data per node.
  randomwriter: A map/reduce program that writes 10GB of random data per
node.
  secondarysort: An example defining a secondary sort to the reduce.
  sleep: A job that sleeps at each map and reduce task.
  sort: A map/reduce program that sorts the data written by the random
writer.
  sudoku: A sudoku solver.
  teragen: Generate data for the terasort
  terasort: Run the terasort
  teravalidate: Checking results of terasort
  wordcount: A map/reduce program that counts the words in the input files.

HTH
Ravi

On Mon, May 7, 2012 at 3:56 PM, Shuai Zheng <sh...@kaust.edu.sa>wrote:

> Hi all,
>
> I am student working on Hadoop 0.20.203.0.
>
> I want to monitor the system resource utilization of pagerank and/or
> shortest path hadoop program. I only need to monitor the system, so I did
> not write any hadoop programs.
>
> I want to ask for some example codes (with many iterations) and dataset
> from you. Algorithms such as:
> 1. Pagerank
> 2. Shortest path
>
> (This is not necessary) it is better that the program can output log info
> for iteration start time and stop time.
>
> If you know any public example code, or you have your own code, and would
> like to share with me (in case any related publication using your code in
> the future, your contribution will be acknowledged.), please send me a
> copy.
>
> Thank you and best regards,
>
> Shuai
>