Error related Hadoop while running Amazon EMR Job

[Bhavesh Shah <bh...@gmail.com>]

Hello all,

I launched my instance on Amazon EMR. And I run the job yesterday on it and
I got the following error in log fies (Log Files of JobTracker)

DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
java.io.IOException: File /mnt/var/lib/hadoop/tmp/mapred/system/
jobtracker.info could only be replicated to 0 nodes, instead of 1

And due to this my job get terminated. Is anything wrong while
configuration/running jobs?

Can someone suggest me what could be the reason while SHUTTING DOWN the Job
Flow in AMazon EMR?

Many Thanks.


-- 
Regards,
Bhavesh Shah

Re: Error related Hadoop while running Amazon EMR Job

[shashwat shriparv <dw...@gmail.com>]

Another reason that may be the cause this problem is that there may not be
much space on the namenode for its operation which was precisely the
problem which I faced. Clear some space for hadoop to launch its operations
and you are done.

On Fri, Apr 27, 2012 at 2:22 AM, shashwat shriparv <
dwivedishashwat@gmail.com> wrote:

> Check out this thread too :
>
> http://comments.gmane.org/gmane.comp.java.hadoop.hbase.user/24098
>
>
> On Fri, Apr 27, 2012 at 2:16 AM, shashwat shriparv <
> dwivedishashwat@gmail.com> wrote:
>
>> The most common is that you have reformatted the namenode leaving it in
>> an inconsistent state. The most common solution is to stop dfs, remove the
>> contents of the dfs directories on all the machines, run “hadoop namenode
>> -format” on the controller, then restart dfs. That consistently fixes the
>> problem for me. This may be serious overkill but it works.
>>
>>
>> On Thu, Apr 26, 2012 at 9:25 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>
>>> are there any datanodes running ?
>>>
>>>
>>> On Thu, Apr 26, 2012 at 7:36 PM, Bhavesh Shah <bh...@gmail.com>wrote:
>>>
>>>> Hello all,
>>>>
>>>> I launched my instance on Amazon EMR. And I run the job yesterday on it
>>>> and I got the following error in log fies (Log Files of JobTracker)
>>>>
>>>> DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
>>>> java.io.IOException: File /mnt/var/lib/hadoop/tmp/mapred/system/
>>>> jobtracker.info could only be replicated to 0 nodes, instead of 1
>>>>
>>>> And due to this my job get terminated. Is anything wrong while
>>>> configuration/running jobs?
>>>>
>>>> Can someone suggest me what could be the reason while SHUTTING DOWN the
>>>> Job Flow in AMazon EMR?
>>>>
>>>> Many Thanks.
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Bhavesh Shah
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>>
>>
>>
>> --
>>
>>
>> ∞
>> Shashwat Shriparv
>>
>>
>>
>
>
> --
>
>
> ∞
> Shashwat Shriparv
>
>
>


-- 


∞
Shashwat Shriparv

Re: Error related Hadoop while running Amazon EMR Job

[shashwat shriparv <dw...@gmail.com>]

Check out this thread too :

http://comments.gmane.org/gmane.comp.java.hadoop.hbase.user/24098


On Fri, Apr 27, 2012 at 2:16 AM, shashwat shriparv <
dwivedishashwat@gmail.com> wrote:

> The most common is that you have reformatted the namenode leaving it in an
> inconsistent state. The most common solution is to stop dfs, remove the
> contents of the dfs directories on all the machines, run “hadoop namenode
> -format” on the controller, then restart dfs. That consistently fixes the
> problem for me. This may be serious overkill but it works.
>
>
> On Thu, Apr 26, 2012 at 9:25 PM, Nitin Pawar <ni...@gmail.com>wrote:
>
>> are there any datanodes running ?
>>
>>
>> On Thu, Apr 26, 2012 at 7:36 PM, Bhavesh Shah <bh...@gmail.com>wrote:
>>
>>> Hello all,
>>>
>>> I launched my instance on Amazon EMR. And I run the job yesterday on it
>>> and I got the following error in log fies (Log Files of JobTracker)
>>>
>>> DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
>>> java.io.IOException: File /mnt/var/lib/hadoop/tmp/mapred/system/
>>> jobtracker.info could only be replicated to 0 nodes, instead of 1
>>>
>>> And due to this my job get terminated. Is anything wrong while
>>> configuration/running jobs?
>>>
>>> Can someone suggest me what could be the reason while SHUTTING DOWN the
>>> Job Flow in AMazon EMR?
>>>
>>> Many Thanks.
>>>
>>>
>>> --
>>> Regards,
>>> Bhavesh Shah
>>>
>>>
>>
>>
>> --
>> Nitin Pawar
>>
>>
>
>
> --
>
>
> ∞
> Shashwat Shriparv
>
>
>


-- 


∞
Shashwat Shriparv

Re: Error related Hadoop while running Amazon EMR Job

[shashwat shriparv <dw...@gmail.com>]

The most common is that you have reformatted the namenode leaving it in an
inconsistent state. The most common solution is to stop dfs, remove the
contents of the dfs directories on all the machines, run “hadoop namenode
-format” on the controller, then restart dfs. That consistently fixes the
problem for me. This may be serious overkill but it works.

On Thu, Apr 26, 2012 at 9:25 PM, Nitin Pawar <ni...@gmail.com>wrote:

> are there any datanodes running ?
>
>
> On Thu, Apr 26, 2012 at 7:36 PM, Bhavesh Shah <bh...@gmail.com>wrote:
>
>> Hello all,
>>
>> I launched my instance on Amazon EMR. And I run the job yesterday on it
>> and I got the following error in log fies (Log Files of JobTracker)
>>
>> DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
>> java.io.IOException: File /mnt/var/lib/hadoop/tmp/mapred/system/
>> jobtracker.info could only be replicated to 0 nodes, instead of 1
>>
>> And due to this my job get terminated. Is anything wrong while
>> configuration/running jobs?
>>
>> Can someone suggest me what could be the reason while SHUTTING DOWN the
>> Job Flow in AMazon EMR?
>>
>> Many Thanks.
>>
>>
>> --
>> Regards,
>> Bhavesh Shah
>>
>>
>
>
> --
> Nitin Pawar
>
>


-- 


∞
Shashwat Shriparv

Re: Error related Hadoop while running Amazon EMR Job

[Nitin Pawar <ni...@gmail.com>]

are there any datanodes running ?

On Thu, Apr 26, 2012 at 7:36 PM, Bhavesh Shah <bh...@gmail.com>wrote:

> Hello all,
>
> I launched my instance on Amazon EMR. And I run the job yesterday on it
> and I got the following error in log fies (Log Files of JobTracker)
>
> DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
> java.io.IOException: File /mnt/var/lib/hadoop/tmp/mapred/system/
> jobtracker.info could only be replicated to 0 nodes, instead of 1
>
> And due to this my job get terminated. Is anything wrong while
> configuration/running jobs?
>
> Can someone suggest me what could be the reason while SHUTTING DOWN the
> Job Flow in AMazon EMR?
>
> Many Thanks.
>
>
> --
> Regards,
> Bhavesh Shah
>
>


-- 
Nitin Pawar