You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Shashi Vishwakarma <sh...@gmail.com> on 2015/12/04 06:33:56 UTC
Unable to run Pig jar using java command
Hi
I am trying to simple pig code but its giving me some permission. Here is
my sample pig code.
import java.io.IOException;import org.apache.pig.PigServer;import
org.apache.pig.ExecType;import
org.apache.hadoop.conf.Configuration;public class PigTest{
public static void main(String[] args) {
try {
Configuration conf = new Configuration();
conf.set("hdp.version","2.3.0.0-2557");
//PigServer pigServer = new PigServer("mapreduce",conf);
PigServer pigServer = new PigServer(ExecType.MAPREDUCE, conf);
runIdQuery(pigServer, "myfile.txt");
}
catch(Exception e) {
}}public static void runIdQuery(PigServer pigServer, String
inputFile) throws IOException {
pigServer.registerQuery("A = load '" + inputFile + "' using
PigStorage(':');");
pigServer.registerQuery("B = foreach A generate $0 as id;");
pigServer.store("B", "idout");
}}
I set my classpath as below
echo $CLASSPATH.:/usr/hdp/2.3.0.0-2557/hadoop/client/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/pig/lib/*:/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar
When i am running my code with java command it is working perfectly but
when created a jar of my code and trying to run , it is giving me below
permission error.
*java -cp /usr/hdp/2.3.0.0-2557/hadoop/client**/*;*/usr/hdp/2.3.0.0-2557/hado*op-hdfs/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*;/usr/hdp/2.3.0.0-2557/pig/lib/*;/usr/hdp/2.3.0.0-2557/hadoop/conf;/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar
-jar PigTest.jar*
*Error: Could not find or load main class
.usr.hdp.2.3.0.0-2557.hadoop.client.activation.jar
-bash: /usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/asm-3.2.jar: Permission denied
-bash: /usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/aopalliance-1.0.jar:
Permission denied
-bash: /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/activation-1.1.jar:
Permission denied
-bash: /usr/hdp/2.3.0.0-2557/pig/lib/accumulo-core-1.5.0.jar: Permission denied
-bash: /usr/hdp/2.3.0.0-2557/hadoop/conf: is a directory
-bash: /usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar:
Permission denied*
Any clue on this?
Thanks
Shashi
Re: Unable to run Pig jar using java command
Posted by Shashikant K <sh...@gmail.com>.
Hi Shashi,
Try the following
1. Instead of /usr/hdp/2.3.0.0-2557/hadoop/client/* use
/usr/hdp/2.3.0.0-2557/hadoop/client/*.jar (See .jar at the end. Replace all
* with *.jar)
2. Instead of semi-colon ( ; ) try with colon ( : )
3. If step 1 above does not work then try putting individual JAR in the
path instead of *.jar
Regards,
Shashikant
On 5 December 2015 at 22:24, Shashi Vishwakarma <sh...@gmail.com>
wrote:
> Still getting same error
>
> java -cp
>
> "/usr/hdp/2.3.0.0-2557/hadoop/client/*;/usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*;/usr/hdp/2.3.0.0-2557/pig/lib/*;/usr/hdp/2.3.0.0-2557/hadoop/conf;/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar;$HADOOP_CONF;
> */usr/hdp/2.3.0.0-2557/pig/**" -jar PigTest.jar
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/pig/PigServer
> at java.lang.Class.getDeclaredMethods0(Native Method)
> at java.lang.Class.privateGetDeclaredMethods(Class.java:2625)
> at java.lang.Class.getMethod0(Class.java:2866)
> at java.lang.Class.getMethod(Class.java:1676)
> at
> sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
> at
> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
> Caused by: java.lang.ClassNotFoundException: org.apache.pig.PigServer
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 6 more
>
>
> Thanks
> Shashi
>
>
>
> On Sat, Dec 5, 2015 at 5:26 PM, Artem Ervits <ar...@gmail.com>
> wrote:
>
> > Instead of adding just one pig jar
> > usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core- Add the whole
> > directory. You're missing pig lib on classpath.
> > On Dec 5, 2015 6:06 AM, "Shashi Vishwakarma" <sh...@gmail.com>
> > wrote:
> >
> > > Hi Mike,
> > >
> > > I tried running code using "" in command and i got below error.
> > >
> > > java -cp
> > >
> > >
> >
> "/usr/hdp/2.3.0.0-2557/hadoop/client/*;/usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*;/usr/hdp/2.3.0.0-2557/pig/lib/*;/usr/hdp/2.3.0.0-2557/hadoop/conf;/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar:$HADOOP_CONF"
> > > -jar PigTest.jar
> > > Exception in thread "main" java.lang.NoClassDefFoundError:
> > > org/apache/pig/PigServer
> > > at java.lang.Class.getDeclaredMethods0(Native Method)
> > > at java.lang.Class.privateGetDeclaredMethods(Class.java:2625)
> > > at java.lang.Class.getMethod0(Class.java:2866)
> > > at java.lang.Class.getMethod(Class.java:1676)
> > > at
> > > sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
> > > at
> > > sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
> > > Caused by: java.lang.ClassNotFoundException: org.apache.pig.PigServer
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > > at java.security.AccessController.doPrivileged(Native Method)
> > > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > ... 6 more
> > >
> > > It says class not found. Looks like it is looking for some jar. Can you
> > > please tell me whats going wrong here?
> > >
> > > Thanks
> > > Shashi
> > >
> > > On Fri, Dec 4, 2015 at 4:38 PM, Michael Spertus <mi...@spertus.com>
> > wrote:
> > >
> > > > The shell is interpreting the semicolons as meaning a new command is
> > > being
> > > > entered. You might need to use some quotation marks around the
> > classpath
> > > > On Dec 3, 2015 11:34 PM, "Shashi Vishwakarma" <
> > shashi.vish123@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi
> > > > > I am trying to simple pig code but its giving me some permission.
> > Here
> > > is
> > > > > my sample pig code.
> > > > >
> > > > > import java.io.IOException;import org.apache.pig.PigServer;import
> > > > > org.apache.pig.ExecType;import
> > > > > org.apache.hadoop.conf.Configuration;public class PigTest{
> > > > > public static void main(String[] args) {
> > > > > try {
> > > > > Configuration conf = new Configuration();
> > > > > conf.set("hdp.version","2.3.0.0-2557");
> > > > > //PigServer pigServer = new PigServer("mapreduce",conf);
> > > > > PigServer pigServer = new PigServer(ExecType.MAPREDUCE,
> > conf);
> > > > > runIdQuery(pigServer, "myfile.txt");
> > > > > }
> > > > > catch(Exception e) {
> > > > > }}public static void runIdQuery(PigServer pigServer, String
> > > > > inputFile) throws IOException {
> > > > > pigServer.registerQuery("A = load '" + inputFile + "' using
> > > > > PigStorage(':');");
> > > > > pigServer.registerQuery("B = foreach A generate $0 as id;");
> > > > > pigServer.store("B", "idout");
> > > > > }}
> > > > >
> > > > > I set my classpath as below
> > > > >
> > > > > echo
> > > > >
> > > >
> > >
> >
> $CLASSPATH.:/usr/hdp/2.3.0.0-2557/hadoop/client/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/pig/lib/*:/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar
> > > > >
> > > > > When i am running my code with java command it is working perfectly
> > but
> > > > > when created a jar of my code and trying to run , it is giving me
> > below
> > > > > permission error.
> > > > >
> > > > > *java -cp
> > > > >
> > > >
> > >
> >
> /usr/hdp/2.3.0.0-2557/hadoop/client**/*;*/usr/hdp/2.3.0.0-2557/hado*op-hdfs/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*;/usr/hdp/2.3.0.0-2557/pig/lib/*;/usr/hdp/2.3.0.0-2557/hadoop/conf;/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar
> > > > > -jar PigTest.jar*
> > > > >
> > > > > *Error: Could not find or load main class
> > > > > .usr.hdp.2.3.0.0-2557.hadoop.client.activation.jar
> > > > > -bash: /usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/asm-3.2.jar:
> Permission
> > > > denied
> > > > > -bash:
> > /usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/aopalliance-1.0.jar:
> > > > > Permission denied
> > > > > -bash: /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/activation-1.1.jar:
> > > > > Permission denied
> > > > > -bash: /usr/hdp/2.3.0.0-2557/pig/lib/accumulo-core-1.5.0.jar:
> > > Permission
> > > > > denied
> > > > > -bash: /usr/hdp/2.3.0.0-2557/hadoop/conf: is a directory
> > > > > -bash:
> /usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar:
> > > > > Permission denied*
> > > > >
> > > > >
> > > > > Any clue on this?
> > > > >
> > > > > Thanks
> > > > > Shashi
> > > > >
> > > >
> > >
> >
>
Re: Unable to run Pig jar using java command
Posted by Shashi Vishwakarma <sh...@gmail.com>.
Still getting same error
java -cp
"/usr/hdp/2.3.0.0-2557/hadoop/client/*;/usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*;/usr/hdp/2.3.0.0-2557/pig/lib/*;/usr/hdp/2.3.0.0-2557/hadoop/conf;/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar;$HADOOP_CONF;
*/usr/hdp/2.3.0.0-2557/pig/**" -jar PigTest.jar
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/pig/PigServer
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2625)
at java.lang.Class.getMethod0(Class.java:2866)
at java.lang.Class.getMethod(Class.java:1676)
at
sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
at
sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: org.apache.pig.PigServer
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 6 more
Thanks
Shashi
On Sat, Dec 5, 2015 at 5:26 PM, Artem Ervits <ar...@gmail.com> wrote:
> Instead of adding just one pig jar
> usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core- Add the whole
> directory. You're missing pig lib on classpath.
> On Dec 5, 2015 6:06 AM, "Shashi Vishwakarma" <sh...@gmail.com>
> wrote:
>
> > Hi Mike,
> >
> > I tried running code using "" in command and i got below error.
> >
> > java -cp
> >
> >
> "/usr/hdp/2.3.0.0-2557/hadoop/client/*;/usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*;/usr/hdp/2.3.0.0-2557/pig/lib/*;/usr/hdp/2.3.0.0-2557/hadoop/conf;/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar:$HADOOP_CONF"
> > -jar PigTest.jar
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> > org/apache/pig/PigServer
> > at java.lang.Class.getDeclaredMethods0(Native Method)
> > at java.lang.Class.privateGetDeclaredMethods(Class.java:2625)
> > at java.lang.Class.getMethod0(Class.java:2866)
> > at java.lang.Class.getMethod(Class.java:1676)
> > at
> > sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
> > at
> > sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
> > Caused by: java.lang.ClassNotFoundException: org.apache.pig.PigServer
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > ... 6 more
> >
> > It says class not found. Looks like it is looking for some jar. Can you
> > please tell me whats going wrong here?
> >
> > Thanks
> > Shashi
> >
> > On Fri, Dec 4, 2015 at 4:38 PM, Michael Spertus <mi...@spertus.com>
> wrote:
> >
> > > The shell is interpreting the semicolons as meaning a new command is
> > being
> > > entered. You might need to use some quotation marks around the
> classpath
> > > On Dec 3, 2015 11:34 PM, "Shashi Vishwakarma" <
> shashi.vish123@gmail.com>
> > > wrote:
> > >
> > > > Hi
> > > > I am trying to simple pig code but its giving me some permission.
> Here
> > is
> > > > my sample pig code.
> > > >
> > > > import java.io.IOException;import org.apache.pig.PigServer;import
> > > > org.apache.pig.ExecType;import
> > > > org.apache.hadoop.conf.Configuration;public class PigTest{
> > > > public static void main(String[] args) {
> > > > try {
> > > > Configuration conf = new Configuration();
> > > > conf.set("hdp.version","2.3.0.0-2557");
> > > > //PigServer pigServer = new PigServer("mapreduce",conf);
> > > > PigServer pigServer = new PigServer(ExecType.MAPREDUCE,
> conf);
> > > > runIdQuery(pigServer, "myfile.txt");
> > > > }
> > > > catch(Exception e) {
> > > > }}public static void runIdQuery(PigServer pigServer, String
> > > > inputFile) throws IOException {
> > > > pigServer.registerQuery("A = load '" + inputFile + "' using
> > > > PigStorage(':');");
> > > > pigServer.registerQuery("B = foreach A generate $0 as id;");
> > > > pigServer.store("B", "idout");
> > > > }}
> > > >
> > > > I set my classpath as below
> > > >
> > > > echo
> > > >
> > >
> >
> $CLASSPATH.:/usr/hdp/2.3.0.0-2557/hadoop/client/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/pig/lib/*:/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar
> > > >
> > > > When i am running my code with java command it is working perfectly
> but
> > > > when created a jar of my code and trying to run , it is giving me
> below
> > > > permission error.
> > > >
> > > > *java -cp
> > > >
> > >
> >
> /usr/hdp/2.3.0.0-2557/hadoop/client**/*;*/usr/hdp/2.3.0.0-2557/hado*op-hdfs/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*;/usr/hdp/2.3.0.0-2557/pig/lib/*;/usr/hdp/2.3.0.0-2557/hadoop/conf;/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar
> > > > -jar PigTest.jar*
> > > >
> > > > *Error: Could not find or load main class
> > > > .usr.hdp.2.3.0.0-2557.hadoop.client.activation.jar
> > > > -bash: /usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/asm-3.2.jar: Permission
> > > denied
> > > > -bash:
> /usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/aopalliance-1.0.jar:
> > > > Permission denied
> > > > -bash: /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/activation-1.1.jar:
> > > > Permission denied
> > > > -bash: /usr/hdp/2.3.0.0-2557/pig/lib/accumulo-core-1.5.0.jar:
> > Permission
> > > > denied
> > > > -bash: /usr/hdp/2.3.0.0-2557/hadoop/conf: is a directory
> > > > -bash: /usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar:
> > > > Permission denied*
> > > >
> > > >
> > > > Any clue on this?
> > > >
> > > > Thanks
> > > > Shashi
> > > >
> > >
> >
>
Re: Unable to run Pig jar using java command
Posted by Artem Ervits <ar...@gmail.com>.
Instead of adding just one pig jar
usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core- Add the whole
directory. You're missing pig lib on classpath.
On Dec 5, 2015 6:06 AM, "Shashi Vishwakarma" <sh...@gmail.com>
wrote:
> Hi Mike,
>
> I tried running code using "" in command and i got below error.
>
> java -cp
>
> "/usr/hdp/2.3.0.0-2557/hadoop/client/*;/usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*;/usr/hdp/2.3.0.0-2557/pig/lib/*;/usr/hdp/2.3.0.0-2557/hadoop/conf;/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar:$HADOOP_CONF"
> -jar PigTest.jar
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/pig/PigServer
> at java.lang.Class.getDeclaredMethods0(Native Method)
> at java.lang.Class.privateGetDeclaredMethods(Class.java:2625)
> at java.lang.Class.getMethod0(Class.java:2866)
> at java.lang.Class.getMethod(Class.java:1676)
> at
> sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
> at
> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
> Caused by: java.lang.ClassNotFoundException: org.apache.pig.PigServer
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 6 more
>
> It says class not found. Looks like it is looking for some jar. Can you
> please tell me whats going wrong here?
>
> Thanks
> Shashi
>
> On Fri, Dec 4, 2015 at 4:38 PM, Michael Spertus <mi...@spertus.com> wrote:
>
> > The shell is interpreting the semicolons as meaning a new command is
> being
> > entered. You might need to use some quotation marks around the classpath
> > On Dec 3, 2015 11:34 PM, "Shashi Vishwakarma" <sh...@gmail.com>
> > wrote:
> >
> > > Hi
> > > I am trying to simple pig code but its giving me some permission. Here
> is
> > > my sample pig code.
> > >
> > > import java.io.IOException;import org.apache.pig.PigServer;import
> > > org.apache.pig.ExecType;import
> > > org.apache.hadoop.conf.Configuration;public class PigTest{
> > > public static void main(String[] args) {
> > > try {
> > > Configuration conf = new Configuration();
> > > conf.set("hdp.version","2.3.0.0-2557");
> > > //PigServer pigServer = new PigServer("mapreduce",conf);
> > > PigServer pigServer = new PigServer(ExecType.MAPREDUCE, conf);
> > > runIdQuery(pigServer, "myfile.txt");
> > > }
> > > catch(Exception e) {
> > > }}public static void runIdQuery(PigServer pigServer, String
> > > inputFile) throws IOException {
> > > pigServer.registerQuery("A = load '" + inputFile + "' using
> > > PigStorage(':');");
> > > pigServer.registerQuery("B = foreach A generate $0 as id;");
> > > pigServer.store("B", "idout");
> > > }}
> > >
> > > I set my classpath as below
> > >
> > > echo
> > >
> >
> $CLASSPATH.:/usr/hdp/2.3.0.0-2557/hadoop/client/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/pig/lib/*:/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar
> > >
> > > When i am running my code with java command it is working perfectly but
> > > when created a jar of my code and trying to run , it is giving me below
> > > permission error.
> > >
> > > *java -cp
> > >
> >
> /usr/hdp/2.3.0.0-2557/hadoop/client**/*;*/usr/hdp/2.3.0.0-2557/hado*op-hdfs/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*;/usr/hdp/2.3.0.0-2557/pig/lib/*;/usr/hdp/2.3.0.0-2557/hadoop/conf;/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar
> > > -jar PigTest.jar*
> > >
> > > *Error: Could not find or load main class
> > > .usr.hdp.2.3.0.0-2557.hadoop.client.activation.jar
> > > -bash: /usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/asm-3.2.jar: Permission
> > denied
> > > -bash: /usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/aopalliance-1.0.jar:
> > > Permission denied
> > > -bash: /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/activation-1.1.jar:
> > > Permission denied
> > > -bash: /usr/hdp/2.3.0.0-2557/pig/lib/accumulo-core-1.5.0.jar:
> Permission
> > > denied
> > > -bash: /usr/hdp/2.3.0.0-2557/hadoop/conf: is a directory
> > > -bash: /usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar:
> > > Permission denied*
> > >
> > >
> > > Any clue on this?
> > >
> > > Thanks
> > > Shashi
> > >
> >
>
Re: Unable to run Pig jar using java command
Posted by Shashi Vishwakarma <sh...@gmail.com>.
Hi Mike,
I tried running code using "" in command and i got below error.
java -cp
"/usr/hdp/2.3.0.0-2557/hadoop/client/*;/usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*;/usr/hdp/2.3.0.0-2557/pig/lib/*;/usr/hdp/2.3.0.0-2557/hadoop/conf;/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar:$HADOOP_CONF"
-jar PigTest.jar
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/pig/PigServer
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2625)
at java.lang.Class.getMethod0(Class.java:2866)
at java.lang.Class.getMethod(Class.java:1676)
at
sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
at
sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: org.apache.pig.PigServer
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 6 more
It says class not found. Looks like it is looking for some jar. Can you
please tell me whats going wrong here?
Thanks
Shashi
On Fri, Dec 4, 2015 at 4:38 PM, Michael Spertus <mi...@spertus.com> wrote:
> The shell is interpreting the semicolons as meaning a new command is being
> entered. You might need to use some quotation marks around the classpath
> On Dec 3, 2015 11:34 PM, "Shashi Vishwakarma" <sh...@gmail.com>
> wrote:
>
> > Hi
> > I am trying to simple pig code but its giving me some permission. Here is
> > my sample pig code.
> >
> > import java.io.IOException;import org.apache.pig.PigServer;import
> > org.apache.pig.ExecType;import
> > org.apache.hadoop.conf.Configuration;public class PigTest{
> > public static void main(String[] args) {
> > try {
> > Configuration conf = new Configuration();
> > conf.set("hdp.version","2.3.0.0-2557");
> > //PigServer pigServer = new PigServer("mapreduce",conf);
> > PigServer pigServer = new PigServer(ExecType.MAPREDUCE, conf);
> > runIdQuery(pigServer, "myfile.txt");
> > }
> > catch(Exception e) {
> > }}public static void runIdQuery(PigServer pigServer, String
> > inputFile) throws IOException {
> > pigServer.registerQuery("A = load '" + inputFile + "' using
> > PigStorage(':');");
> > pigServer.registerQuery("B = foreach A generate $0 as id;");
> > pigServer.store("B", "idout");
> > }}
> >
> > I set my classpath as below
> >
> > echo
> >
> $CLASSPATH.:/usr/hdp/2.3.0.0-2557/hadoop/client/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/pig/lib/*:/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar
> >
> > When i am running my code with java command it is working perfectly but
> > when created a jar of my code and trying to run , it is giving me below
> > permission error.
> >
> > *java -cp
> >
> /usr/hdp/2.3.0.0-2557/hadoop/client**/*;*/usr/hdp/2.3.0.0-2557/hado*op-hdfs/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*;/usr/hdp/2.3.0.0-2557/pig/lib/*;/usr/hdp/2.3.0.0-2557/hadoop/conf;/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar
> > -jar PigTest.jar*
> >
> > *Error: Could not find or load main class
> > .usr.hdp.2.3.0.0-2557.hadoop.client.activation.jar
> > -bash: /usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/asm-3.2.jar: Permission
> denied
> > -bash: /usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/aopalliance-1.0.jar:
> > Permission denied
> > -bash: /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/activation-1.1.jar:
> > Permission denied
> > -bash: /usr/hdp/2.3.0.0-2557/pig/lib/accumulo-core-1.5.0.jar: Permission
> > denied
> > -bash: /usr/hdp/2.3.0.0-2557/hadoop/conf: is a directory
> > -bash: /usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar:
> > Permission denied*
> >
> >
> > Any clue on this?
> >
> > Thanks
> > Shashi
> >
>
Re: Unable to run Pig jar using java command
Posted by Michael Spertus <mi...@spertus.com>.
The shell is interpreting the semicolons as meaning a new command is being
entered. You might need to use some quotation marks around the classpath
On Dec 3, 2015 11:34 PM, "Shashi Vishwakarma" <sh...@gmail.com>
wrote:
> Hi
> I am trying to simple pig code but its giving me some permission. Here is
> my sample pig code.
>
> import java.io.IOException;import org.apache.pig.PigServer;import
> org.apache.pig.ExecType;import
> org.apache.hadoop.conf.Configuration;public class PigTest{
> public static void main(String[] args) {
> try {
> Configuration conf = new Configuration();
> conf.set("hdp.version","2.3.0.0-2557");
> //PigServer pigServer = new PigServer("mapreduce",conf);
> PigServer pigServer = new PigServer(ExecType.MAPREDUCE, conf);
> runIdQuery(pigServer, "myfile.txt");
> }
> catch(Exception e) {
> }}public static void runIdQuery(PigServer pigServer, String
> inputFile) throws IOException {
> pigServer.registerQuery("A = load '" + inputFile + "' using
> PigStorage(':');");
> pigServer.registerQuery("B = foreach A generate $0 as id;");
> pigServer.store("B", "idout");
> }}
>
> I set my classpath as below
>
> echo
> $CLASSPATH.:/usr/hdp/2.3.0.0-2557/hadoop/client/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/pig/lib/*:/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar
>
> When i am running my code with java command it is working perfectly but
> when created a jar of my code and trying to run , it is giving me below
> permission error.
>
> *java -cp
> /usr/hdp/2.3.0.0-2557/hadoop/client**/*;*/usr/hdp/2.3.0.0-2557/hado*op-hdfs/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*;/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*;/usr/hdp/2.3.0.0-2557/pig/lib/*;/usr/hdp/2.3.0.0-2557/hadoop/conf;/usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar
> -jar PigTest.jar*
>
> *Error: Could not find or load main class
> .usr.hdp.2.3.0.0-2557.hadoop.client.activation.jar
> -bash: /usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/asm-3.2.jar: Permission denied
> -bash: /usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/aopalliance-1.0.jar:
> Permission denied
> -bash: /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/activation-1.1.jar:
> Permission denied
> -bash: /usr/hdp/2.3.0.0-2557/pig/lib/accumulo-core-1.5.0.jar: Permission
> denied
> -bash: /usr/hdp/2.3.0.0-2557/hadoop/conf: is a directory
> -bash: /usr/hdp/2.3.0.0-2557/pig/pig-0.15.0.2.3.0.0-2557-core-h2.jar:
> Permission denied*
>
>
> Any clue on this?
>
> Thanks
> Shashi
>