You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by 张茂森 <ma...@alibaba-inc.com> on 2006/09/25 12:04:18 UTC

Can Hadoop deal with several files on DFS as input?

Hi all:

I want to process several logs files (which are collected from several
severs) on hadoop, nor combine those files before import them into DFS
system. Could it be carried out?

Best Regards, Zhang Maosen

SDS

Email: maosen.zhang <ma...@alibaba-inc.com> @alibaba-inc.com
Tel: 86-10-6598 5796

Re: Can Hadoop deal with several files on DFS as input?

Posted by Vetle Roeim <ve...@opera.com>.

On Mon, 25 Sep 2006 12:04:18 +0200, 张茂森 <ma...@alibaba-inc.com>  
wrote:

> Hi all:
>
> I want to process several logs files (which are collected from several
> severs) on hadoop, nor combine those files before import them into DFS
> system. Could it be carried out?

Works fine for me. I add logs from multiple servers into the DFS, and then  
use  JobConf.addInputPath(Path) when configuring the job to add all  
relevant log files.

-- 
Vetle Roeim
Team Manager, Information Systems
Opera Software ASA <URL: http://www.opera.com/ >