2 questions about hadoopifying

[Hua Wei <si...@gmail.com>]

Hi,

I got 2 questions when hadoopifying my old programs.

1. How to access an exterior configuration file (not an input file) in
mapper class? Every mapper need to read this XML file for parameters.
Is there a solution to this?
2. Is there any limitation about what kind of data structures can be
used in a mapper or reducer class? Say, can hashMap or self-defined
class be used within a mapper class?

I am new to hadoop. Any response would be appreciated.

RE: 2 questions about hadoopifying

[Nathan Wang <wa...@yahoo-inc.com>]

1. You can if you copy the cfg file into HDFS.  Otherwise, it's local to
one node, and can't be accessed by map/reduce jobs running on other
nodes.

2. You can write your own RecordReader/InputFormat classes and handle
input files in any formats of your own.

Nathan

-----Original Message-----
From: Hua Wei [mailto:sinic.wei@gmail.com] 
Sent: Wednesday, March 19, 2008 7:45 AM
To: core-user@hadoop.apache.org
Subject: 2 questions about hadoopifying

Hi,

I got 2 questions when hadoopifying my old programs.

1. How to access an exterior configuration file (not an input file) in
mapper class? Every mapper need to read this XML file for parameters.
Is there a solution to this?
2. Is there any limitation about what kind of data structures can be
used in a mapper or reducer class? Say, can hashMap or self-defined
class be used within a mapper class?

I am new to hadoop. Any response would be appreciated.