You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by "Kartashov, Andy" <An...@mpac.ca> on 2012/11/16 22:13:16 UTC
notorious ClassNotFoundError
Guys,
The notorious error. Used all possible clues to resolve this.
Ran sqoop --import.... at command line without problems. But whenever am trying to execute MR job get Error: java.lang.ClassNotFoundException: com.cloudera.sqoop.lib.SqoopRecord
This exact same job ran before.
I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside sqoop-1.4.1-cdh4.1.2.jar...
I copied this jar file to every possible directory:
/usr/lib/hadoop
/usr/lib/hadoop/lib
As well as mapreduce-0.20 libs
Even tried to include this jar at command line -libjars which is added to the job...
But MR still doesn't pick it up, throwing ClassNotFoundError error.
Rgds,
AK
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le pr?sent courriel et toute pi?ce jointe qui l'accompagne sont confidentiels, prot?g?s par le droit d'auteur et peuvent ?tre couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autoris?e est interdite. Si vous n'?tes pas le destinataire pr?vu de ce courriel, supprimez-le et contactez imm?diatement l'exp?diteur. Veuillez penser ? l'environnement avant d'imprimer le pr?sent courriel
RE: notorious ClassNotFoundError
Posted by "Kartashov, Andy" <An...@mpac.ca>.
Oh my Gosh!!!
This exact error happened to me before, the very first time I installed hadoop. Spent days solving that exact problem. It got resolved on its own so I had no idea what exactly I did.
Today, I was installing hadoop on a brand new node from afresh, and... exact the same ClassNotfoundError no matter where to I copied this jar file to.
I did it again, accidentally resolved this issue .. but THIS TIME I took a note what I did.
Ready?
So, no matter where the jar file was located, no matter if it was indicated in CLASSPATH or HADOOP_HOME or -libjars for that matter, it would not pick up the .jar file until I restarted the TaskTracker. Heheheh
You know the saying that apple a day keeps doctors away? Well, in the future, as the VERY FIRST possible solution I will always try to restart the nodes first.
For those who find this post ever useful, "you're welcome in advance".
Happy hadooping..
AK-47
From: Kartashov, Andy [mailto:Andy.Kartashov@mpac.ca]
Sent: Friday, November 16, 2012 4:13 PM
To: user@hadoop.apache.org
Subject: notorious ClassNotFoundError
Guys,
The notorious error. Used all possible clues to resolve this.
Ran sqoop --import.... at command line without problems. But whenever am trying to execute MR job get Error: java.lang.ClassNotFoundException: com.cloudera.sqoop.lib.SqoopRecord
This exact same job ran before.
I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside sqoop-1.4.1-cdh4.1.2.jar...
I copied this jar file to every possible directory:
/usr/lib/hadoop
/usr/lib/hadoop/lib
As well as mapreduce-0.20 libs
Even tried to include this jar at command line -libjars which is added to the job...
But MR still doesn't pick it up, throwing ClassNotFoundError error.
Rgds,
AK
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
RE: notorious ClassNotFoundError
Posted by "Kartashov, Andy" <An...@mpac.ca>.
I am running MR in pseudo distributed mode - so it is only one Node involced.
From: Joey Krabacher [mailto:jkrabacher@gmail.com]
Sent: Friday, November 16, 2012 4:16 PM
To: user@hadoop.apache.org
Subject: Re: notorious ClassNotFoundError
External libs must exist on all task nodes, in case you haven't already done that.
/* Joey */
On Fri, Nov 16, 2012 at 3:13 PM, Kartashov, Andy <An...@mpac.ca>> wrote:
Guys,
The notorious error. Used all possible clues to resolve this.
Ran sqoop --import.... at command line without problems. But whenever am trying to execute MR job get Error: java.lang.ClassNotFoundException: com.cloudera.sqoop.lib.SqoopRecord
This exact same job ran before.
I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside sqoop-1.4.1-cdh4.1.2.jar...
I copied this jar file to every possible directory:
/usr/lib/hadoop
/usr/lib/hadoop/lib
As well as mapreduce-0.20 libs
Even tried to include this jar at command line -libjars which is added to the job...
But MR still doesn't pick it up, throwing ClassNotFoundError error.
Rgds,
AK
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
RE: notorious ClassNotFoundError
Posted by "Kartashov, Andy" <An...@mpac.ca>.
I am running MR in pseudo distributed mode - so it is only one Node involced.
From: Joey Krabacher [mailto:jkrabacher@gmail.com]
Sent: Friday, November 16, 2012 4:16 PM
To: user@hadoop.apache.org
Subject: Re: notorious ClassNotFoundError
External libs must exist on all task nodes, in case you haven't already done that.
/* Joey */
On Fri, Nov 16, 2012 at 3:13 PM, Kartashov, Andy <An...@mpac.ca>> wrote:
Guys,
The notorious error. Used all possible clues to resolve this.
Ran sqoop --import.... at command line without problems. But whenever am trying to execute MR job get Error: java.lang.ClassNotFoundException: com.cloudera.sqoop.lib.SqoopRecord
This exact same job ran before.
I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside sqoop-1.4.1-cdh4.1.2.jar...
I copied this jar file to every possible directory:
/usr/lib/hadoop
/usr/lib/hadoop/lib
As well as mapreduce-0.20 libs
Even tried to include this jar at command line -libjars which is added to the job...
But MR still doesn't pick it up, throwing ClassNotFoundError error.
Rgds,
AK
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
RE: notorious ClassNotFoundError
Posted by "Kartashov, Andy" <An...@mpac.ca>.
I am running MR in pseudo distributed mode - so it is only one Node involced.
From: Joey Krabacher [mailto:jkrabacher@gmail.com]
Sent: Friday, November 16, 2012 4:16 PM
To: user@hadoop.apache.org
Subject: Re: notorious ClassNotFoundError
External libs must exist on all task nodes, in case you haven't already done that.
/* Joey */
On Fri, Nov 16, 2012 at 3:13 PM, Kartashov, Andy <An...@mpac.ca>> wrote:
Guys,
The notorious error. Used all possible clues to resolve this.
Ran sqoop --import.... at command line without problems. But whenever am trying to execute MR job get Error: java.lang.ClassNotFoundException: com.cloudera.sqoop.lib.SqoopRecord
This exact same job ran before.
I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside sqoop-1.4.1-cdh4.1.2.jar...
I copied this jar file to every possible directory:
/usr/lib/hadoop
/usr/lib/hadoop/lib
As well as mapreduce-0.20 libs
Even tried to include this jar at command line -libjars which is added to the job...
But MR still doesn't pick it up, throwing ClassNotFoundError error.
Rgds,
AK
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
RE: notorious ClassNotFoundError
Posted by "Kartashov, Andy" <An...@mpac.ca>.
I am running MR in pseudo distributed mode - so it is only one Node involced.
From: Joey Krabacher [mailto:jkrabacher@gmail.com]
Sent: Friday, November 16, 2012 4:16 PM
To: user@hadoop.apache.org
Subject: Re: notorious ClassNotFoundError
External libs must exist on all task nodes, in case you haven't already done that.
/* Joey */
On Fri, Nov 16, 2012 at 3:13 PM, Kartashov, Andy <An...@mpac.ca>> wrote:
Guys,
The notorious error. Used all possible clues to resolve this.
Ran sqoop --import.... at command line without problems. But whenever am trying to execute MR job get Error: java.lang.ClassNotFoundException: com.cloudera.sqoop.lib.SqoopRecord
This exact same job ran before.
I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside sqoop-1.4.1-cdh4.1.2.jar...
I copied this jar file to every possible directory:
/usr/lib/hadoop
/usr/lib/hadoop/lib
As well as mapreduce-0.20 libs
Even tried to include this jar at command line -libjars which is added to the job...
But MR still doesn't pick it up, throwing ClassNotFoundError error.
Rgds,
AK
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
Re: notorious ClassNotFoundError
Posted by Joey Krabacher <jk...@gmail.com>.
External libs must exist on all task nodes, in case you haven't already
done that.
/* Joey */
On Fri, Nov 16, 2012 at 3:13 PM, Kartashov, Andy <An...@mpac.ca>wrote:
> Guys,
>
>
>
> The notorious error. Used all possible clues to resolve this.
>
>
>
> Ran sqoop --import.... at command line without problems. But whenever am
> trying to execute MR job get Error: java.lang.ClassNotFoundException:
> com.cloudera.sqoop.lib.SqoopRecord
>
> This exact same job ran before.
>
>
>
> I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside
> sqoop-1.4.1-cdh4.1.2.jar...
>
>
>
> I copied this jar file to every possible directory:
>
> /usr/lib/hadoop
>
> /usr/lib/hadoop/lib
>
> As well as mapreduce-0.20 libs
>
>
>
> Even tried to include this jar at command line -libjars which is added to
> the job...
>
>
>
> But MR still doesn't pick it up, throwing ClassNotFoundError error.
>
>
>
> Rgds,
>
> AK
>
>
> NOTICE: This e-mail message and any attachments are confidential, subject
> to copyright and may be privileged. Any unauthorized use, copying or
> disclosure is prohibited. If you are not the intended recipient, please
> delete and contact the sender immediately. Please consider the environment
> before printing this e-mail. AVIS : le présent courriel et toute pièce
> jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur
> et peuvent être couverts par le secret professionnel. Toute utilisation,
> copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le
> destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
> l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent
> courriel
>
Re: notorious ClassNotFoundError
Posted by Joey Krabacher <jk...@gmail.com>.
External libs must exist on all task nodes, in case you haven't already
done that.
/* Joey */
On Fri, Nov 16, 2012 at 3:13 PM, Kartashov, Andy <An...@mpac.ca>wrote:
> Guys,
>
>
>
> The notorious error. Used all possible clues to resolve this.
>
>
>
> Ran sqoop --import.... at command line without problems. But whenever am
> trying to execute MR job get Error: java.lang.ClassNotFoundException:
> com.cloudera.sqoop.lib.SqoopRecord
>
> This exact same job ran before.
>
>
>
> I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside
> sqoop-1.4.1-cdh4.1.2.jar...
>
>
>
> I copied this jar file to every possible directory:
>
> /usr/lib/hadoop
>
> /usr/lib/hadoop/lib
>
> As well as mapreduce-0.20 libs
>
>
>
> Even tried to include this jar at command line -libjars which is added to
> the job...
>
>
>
> But MR still doesn't pick it up, throwing ClassNotFoundError error.
>
>
>
> Rgds,
>
> AK
>
>
> NOTICE: This e-mail message and any attachments are confidential, subject
> to copyright and may be privileged. Any unauthorized use, copying or
> disclosure is prohibited. If you are not the intended recipient, please
> delete and contact the sender immediately. Please consider the environment
> before printing this e-mail. AVIS : le présent courriel et toute pièce
> jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur
> et peuvent être couverts par le secret professionnel. Toute utilisation,
> copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le
> destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
> l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent
> courriel
>
RE: notorious ClassNotFoundError
Posted by "Kartashov, Andy" <An...@mpac.ca>.
Oh my Gosh!!!
This exact error happened to me before, the very first time I installed hadoop. Spent days solving that exact problem. It got resolved on its own so I had no idea what exactly I did.
Today, I was installing hadoop on a brand new node from afresh, and... exact the same ClassNotfoundError no matter where to I copied this jar file to.
I did it again, accidentally resolved this issue .. but THIS TIME I took a note what I did.
Ready?
So, no matter where the jar file was located, no matter if it was indicated in CLASSPATH or HADOOP_HOME or -libjars for that matter, it would not pick up the .jar file until I restarted the TaskTracker. Heheheh
You know the saying that apple a day keeps doctors away? Well, in the future, as the VERY FIRST possible solution I will always try to restart the nodes first.
For those who find this post ever useful, "you're welcome in advance".
Happy hadooping..
AK-47
From: Kartashov, Andy [mailto:Andy.Kartashov@mpac.ca]
Sent: Friday, November 16, 2012 4:13 PM
To: user@hadoop.apache.org
Subject: notorious ClassNotFoundError
Guys,
The notorious error. Used all possible clues to resolve this.
Ran sqoop --import.... at command line without problems. But whenever am trying to execute MR job get Error: java.lang.ClassNotFoundException: com.cloudera.sqoop.lib.SqoopRecord
This exact same job ran before.
I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside sqoop-1.4.1-cdh4.1.2.jar...
I copied this jar file to every possible directory:
/usr/lib/hadoop
/usr/lib/hadoop/lib
As well as mapreduce-0.20 libs
Even tried to include this jar at command line -libjars which is added to the job...
But MR still doesn't pick it up, throwing ClassNotFoundError error.
Rgds,
AK
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
Re: notorious ClassNotFoundError
Posted by Joey Krabacher <jk...@gmail.com>.
External libs must exist on all task nodes, in case you haven't already
done that.
/* Joey */
On Fri, Nov 16, 2012 at 3:13 PM, Kartashov, Andy <An...@mpac.ca>wrote:
> Guys,
>
>
>
> The notorious error. Used all possible clues to resolve this.
>
>
>
> Ran sqoop --import.... at command line without problems. But whenever am
> trying to execute MR job get Error: java.lang.ClassNotFoundException:
> com.cloudera.sqoop.lib.SqoopRecord
>
> This exact same job ran before.
>
>
>
> I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside
> sqoop-1.4.1-cdh4.1.2.jar...
>
>
>
> I copied this jar file to every possible directory:
>
> /usr/lib/hadoop
>
> /usr/lib/hadoop/lib
>
> As well as mapreduce-0.20 libs
>
>
>
> Even tried to include this jar at command line -libjars which is added to
> the job...
>
>
>
> But MR still doesn't pick it up, throwing ClassNotFoundError error.
>
>
>
> Rgds,
>
> AK
>
>
> NOTICE: This e-mail message and any attachments are confidential, subject
> to copyright and may be privileged. Any unauthorized use, copying or
> disclosure is prohibited. If you are not the intended recipient, please
> delete and contact the sender immediately. Please consider the environment
> before printing this e-mail. AVIS : le présent courriel et toute pièce
> jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur
> et peuvent être couverts par le secret professionnel. Toute utilisation,
> copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le
> destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
> l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent
> courriel
>
RE: notorious ClassNotFoundError
Posted by "Kartashov, Andy" <An...@mpac.ca>.
Oh my Gosh!!!
This exact error happened to me before, the very first time I installed hadoop. Spent days solving that exact problem. It got resolved on its own so I had no idea what exactly I did.
Today, I was installing hadoop on a brand new node from afresh, and... exact the same ClassNotfoundError no matter where to I copied this jar file to.
I did it again, accidentally resolved this issue .. but THIS TIME I took a note what I did.
Ready?
So, no matter where the jar file was located, no matter if it was indicated in CLASSPATH or HADOOP_HOME or -libjars for that matter, it would not pick up the .jar file until I restarted the TaskTracker. Heheheh
You know the saying that apple a day keeps doctors away? Well, in the future, as the VERY FIRST possible solution I will always try to restart the nodes first.
For those who find this post ever useful, "you're welcome in advance".
Happy hadooping..
AK-47
From: Kartashov, Andy [mailto:Andy.Kartashov@mpac.ca]
Sent: Friday, November 16, 2012 4:13 PM
To: user@hadoop.apache.org
Subject: notorious ClassNotFoundError
Guys,
The notorious error. Used all possible clues to resolve this.
Ran sqoop --import.... at command line without problems. But whenever am trying to execute MR job get Error: java.lang.ClassNotFoundException: com.cloudera.sqoop.lib.SqoopRecord
This exact same job ran before.
I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside sqoop-1.4.1-cdh4.1.2.jar...
I copied this jar file to every possible directory:
/usr/lib/hadoop
/usr/lib/hadoop/lib
As well as mapreduce-0.20 libs
Even tried to include this jar at command line -libjars which is added to the job...
But MR still doesn't pick it up, throwing ClassNotFoundError error.
Rgds,
AK
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
Re: notorious ClassNotFoundError
Posted by Joey Krabacher <jk...@gmail.com>.
External libs must exist on all task nodes, in case you haven't already
done that.
/* Joey */
On Fri, Nov 16, 2012 at 3:13 PM, Kartashov, Andy <An...@mpac.ca>wrote:
> Guys,
>
>
>
> The notorious error. Used all possible clues to resolve this.
>
>
>
> Ran sqoop --import.... at command line without problems. But whenever am
> trying to execute MR job get Error: java.lang.ClassNotFoundException:
> com.cloudera.sqoop.lib.SqoopRecord
>
> This exact same job ran before.
>
>
>
> I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside
> sqoop-1.4.1-cdh4.1.2.jar...
>
>
>
> I copied this jar file to every possible directory:
>
> /usr/lib/hadoop
>
> /usr/lib/hadoop/lib
>
> As well as mapreduce-0.20 libs
>
>
>
> Even tried to include this jar at command line -libjars which is added to
> the job...
>
>
>
> But MR still doesn't pick it up, throwing ClassNotFoundError error.
>
>
>
> Rgds,
>
> AK
>
>
> NOTICE: This e-mail message and any attachments are confidential, subject
> to copyright and may be privileged. Any unauthorized use, copying or
> disclosure is prohibited. If you are not the intended recipient, please
> delete and contact the sender immediately. Please consider the environment
> before printing this e-mail. AVIS : le présent courriel et toute pièce
> jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur
> et peuvent être couverts par le secret professionnel. Toute utilisation,
> copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le
> destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
> l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent
> courriel
>
RE: notorious ClassNotFoundError
Posted by "Kartashov, Andy" <An...@mpac.ca>.
Oh my Gosh!!!
This exact error happened to me before, the very first time I installed hadoop. Spent days solving that exact problem. It got resolved on its own so I had no idea what exactly I did.
Today, I was installing hadoop on a brand new node from afresh, and... exact the same ClassNotfoundError no matter where to I copied this jar file to.
I did it again, accidentally resolved this issue .. but THIS TIME I took a note what I did.
Ready?
So, no matter where the jar file was located, no matter if it was indicated in CLASSPATH or HADOOP_HOME or -libjars for that matter, it would not pick up the .jar file until I restarted the TaskTracker. Heheheh
You know the saying that apple a day keeps doctors away? Well, in the future, as the VERY FIRST possible solution I will always try to restart the nodes first.
For those who find this post ever useful, "you're welcome in advance".
Happy hadooping..
AK-47
From: Kartashov, Andy [mailto:Andy.Kartashov@mpac.ca]
Sent: Friday, November 16, 2012 4:13 PM
To: user@hadoop.apache.org
Subject: notorious ClassNotFoundError
Guys,
The notorious error. Used all possible clues to resolve this.
Ran sqoop --import.... at command line without problems. But whenever am trying to execute MR job get Error: java.lang.ClassNotFoundException: com.cloudera.sqoop.lib.SqoopRecord
This exact same job ran before.
I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside sqoop-1.4.1-cdh4.1.2.jar...
I copied this jar file to every possible directory:
/usr/lib/hadoop
/usr/lib/hadoop/lib
As well as mapreduce-0.20 libs
Even tried to include this jar at command line -libjars which is added to the job...
But MR still doesn't pick it up, throwing ClassNotFoundError error.
Rgds,
AK
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel