You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Curtis Ullerich <cu...@gmail.com> on 2013/06/20 01:53:54 UTC

Compile Just a Subproject

What is the way to compile just a subproject of Hadoop with Maven (without
rebuilding parent modules)? In my case I'm wondering about
DistributedShell. I thought that
this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would
have worked but no changes seem to take effect after building this
way.

Thanks,
Curtis

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
Executing this doesn't seem to work either.
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami

Thank you for pointing out the classpath issue, though. I checked in
hadoop-dist/.../share and found that hadoop had been using the jar there.
The command can't find the Client class unless it's in that directory, so I
suppose a suitable workaround would be to add the distributedshell/target
directory to the classpath myself, or just copy the jar to /share. If
anyone has suggestions for the best way to get around that issue, I would
welcome them. Otherwise, this relieves my confusion--thanks for all the
help!

Curtis



On Fri, Jun 21, 2013 at 5:10 PM, Hitesh Shah <hi...@apache.org> wrote:

> Hello Curtis
>
> Try the following:
>
> hadoop jar ./target/distributed-shell.jar.rebuilt-one
> org.apache.hadoop.yarn.applications.distributedshell.Client -jar ...
>
> If you are running hadoop without the jar command, it will find the first
> instance of Client.class in its classpath which I am guessing is from the
> jars used to start the cluster and not the re-built jar.
>
> -- Hitesh
>
>
> On Jun 21, 2013, at 4:09 PM, Curtis Ullerich wrote:
>
> > I should clarify that the build doesn't actually fail (so that bug
> doesn't seem to be the issue). It reports success. But then the output from
> the just-build jar doesn't change.
> >
> > Here's my output from building:
> >
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn
> clean package -DskipTests
> > [INFO] Scanning for projects...
> > [INFO]
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO] Building hadoop-yarn-applications-distributedshell 3.0.0-SNAPSHOT
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO]
> > [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Deleting
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target
> > [INFO]
> > [INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Executing tasks
> >
> > main:
> >     [mkdir] Created dir:
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-dir
> > [INFO] Executed tasks
> > [INFO]
> > [INFO] --- maven-resources-plugin:2.2:resources (default-resources) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Using default encoding to copy filtered resources.
> > [INFO]
> > [INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Compiling 3 source files to
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\classes
> > [INFO]
> > [INFO] --- maven-resources-plugin:2.2:testResources
> (default-testResources) @ hadoop-yarn-applications-distributedshell ---
> > [INFO] Using default encoding to copy filtered resources.
> > [INFO]
> > [INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile)
> @ hadoop-yarn-applications-distributedshell ---
> > [INFO] Compiling 1 source file to
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-classes
> > [INFO]
> > [INFO] --- maven-jar-plugin:2.3.1:jar (default) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Building jar:
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> > [INFO]
> > [INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Tests are skipped.
> > [INFO]
> > [INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO]
> > [INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO] BUILD SUCCESS
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO] Total time: 3.487s
> > [INFO] Finished at: Fri Jun 21 16:08:02 PDT 2013
> > [INFO] Final Memory: 19M/367M
> > [INFO]
> ------------------------------------------------------------------------
> >
> >
> >
> > On Fri, Jun 21, 2013 at 3:57 PM, Arpit Agarwal <aa...@hortonworks.com>
> wrote:
> > Curtis, if you are running 'mvn clean' in a subproject you may be
> hitting this bug:
> > https://issues.apache.org/jira/browse/HADOOP-9383
> >
> > You might have better luck if you include install along with package in
> the same invocation.
> >
> > -Arpit
> >
> >
> > On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>
> wrote:
> > I've executed the commands as you've said, and the jar that is run is
> unchanged. Here's exactly what I did.
> >
> > C:\l\> start-dfs
> > C:\l\> start-yarn
> > C:\l\> cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>
> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> > ...output...
> > ...modify logging code in
> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>
> mvn clean install -DskipTests
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>
> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> > ...identical output to before...
> >
> > Do you see anything wrong with what I've done?
> >
> > Thanks,
> > Curtis
> >
> >
> >
> >
> > On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>
> wrote:
> > Hi Curtis,
> >
> > where are you picking your jar file from? once you run above command you
> will see the updated jar file in
> >
> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> >
> > I hope you are not using below jar file
> >
> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> >
> >
> > What chris has said is right.
> >
> > once you have taken latest code, you should follow below steps
> > 1) mvn clean install -DskipTests (clean to remove previously generated
> code)
> > 2) now say you are updating distributed shell client code. then go to
> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
> and then run "mvn clean install" and use jar from target sub folder.
> >
> > Thanks,
> > Omkar Joshi
> > Hortonworks Inc.
> >
> >
> > On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <
> curtullerich@gmail.com> wrote:
> > Hi Chris,
> >
> > I really appreciate the response. What you described is what I initially
> tried. The changes never seem to take effect though. Here's what I've done
> (this is Windows):
> >
> > cd %hadoop_install_dir%
> > mvn clean package -DskipTests
> > mvn install -DskipTests
> >
> > --- modify the code in distributed shell's Client.java ---
> >
> > cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> > mvn clean
> > mvn package -DskipTests
> > mvn install -DskipTests
> >
> > Then I run the jar, just as before. I've just been changing log
> statements to see if the changes worked. They don't--the output doesn't
> change after doing this rebuild. I've also tried clearing the files put on
> HDFS in my user directory in case that was the issue. I've taken a more
> thorough look at BUILDING.txt and I seemed to be consistent with the
> procedures described there. Am I missing anything else? I've tried
> restarting yarn and dfs, though I didn't think that would matter.
> >
> > Thanks,
> > Curtis
> >
> >
> >
> >
> > On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
> cnauroth@hortonworks.com> wrote:
> > Hi Curtis,
> >
> > I handle this by running "mvn install -DskipTests" once from the root of
> the whole hadoop project to install the sub-modules in my local Maven
> repository.  Then, you can change your working directory to any sub-module
> (like distributed shell) and run a successful Maven build for just that
> sub-module.  Full details are in the BUILDING.txt file in the root of the
> hadoop project.  Look for the section titled "Building components
> separately".  I hope this helps.
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>
> wrote:
> > What is the way to compile just a subproject of Hadoop with Maven
> (without rebuilding parent modules)? In my case I'm wondering about
> DistributedShell. I thought that this would have worked but no changes seem
> to take effect after building this way.
> >
> > Thanks,
> > Curtis
> >
> >
> >
> >
> >
> >
>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
Executing this doesn't seem to work either.
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami

Thank you for pointing out the classpath issue, though. I checked in
hadoop-dist/.../share and found that hadoop had been using the jar there.
The command can't find the Client class unless it's in that directory, so I
suppose a suitable workaround would be to add the distributedshell/target
directory to the classpath myself, or just copy the jar to /share. If
anyone has suggestions for the best way to get around that issue, I would
welcome them. Otherwise, this relieves my confusion--thanks for all the
help!

Curtis



On Fri, Jun 21, 2013 at 5:10 PM, Hitesh Shah <hi...@apache.org> wrote:

> Hello Curtis
>
> Try the following:
>
> hadoop jar ./target/distributed-shell.jar.rebuilt-one
> org.apache.hadoop.yarn.applications.distributedshell.Client -jar ...
>
> If you are running hadoop without the jar command, it will find the first
> instance of Client.class in its classpath which I am guessing is from the
> jars used to start the cluster and not the re-built jar.
>
> -- Hitesh
>
>
> On Jun 21, 2013, at 4:09 PM, Curtis Ullerich wrote:
>
> > I should clarify that the build doesn't actually fail (so that bug
> doesn't seem to be the issue). It reports success. But then the output from
> the just-build jar doesn't change.
> >
> > Here's my output from building:
> >
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn
> clean package -DskipTests
> > [INFO] Scanning for projects...
> > [INFO]
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO] Building hadoop-yarn-applications-distributedshell 3.0.0-SNAPSHOT
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO]
> > [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Deleting
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target
> > [INFO]
> > [INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Executing tasks
> >
> > main:
> >     [mkdir] Created dir:
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-dir
> > [INFO] Executed tasks
> > [INFO]
> > [INFO] --- maven-resources-plugin:2.2:resources (default-resources) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Using default encoding to copy filtered resources.
> > [INFO]
> > [INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Compiling 3 source files to
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\classes
> > [INFO]
> > [INFO] --- maven-resources-plugin:2.2:testResources
> (default-testResources) @ hadoop-yarn-applications-distributedshell ---
> > [INFO] Using default encoding to copy filtered resources.
> > [INFO]
> > [INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile)
> @ hadoop-yarn-applications-distributedshell ---
> > [INFO] Compiling 1 source file to
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-classes
> > [INFO]
> > [INFO] --- maven-jar-plugin:2.3.1:jar (default) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Building jar:
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> > [INFO]
> > [INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Tests are skipped.
> > [INFO]
> > [INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO]
> > [INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO] BUILD SUCCESS
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO] Total time: 3.487s
> > [INFO] Finished at: Fri Jun 21 16:08:02 PDT 2013
> > [INFO] Final Memory: 19M/367M
> > [INFO]
> ------------------------------------------------------------------------
> >
> >
> >
> > On Fri, Jun 21, 2013 at 3:57 PM, Arpit Agarwal <aa...@hortonworks.com>
> wrote:
> > Curtis, if you are running 'mvn clean' in a subproject you may be
> hitting this bug:
> > https://issues.apache.org/jira/browse/HADOOP-9383
> >
> > You might have better luck if you include install along with package in
> the same invocation.
> >
> > -Arpit
> >
> >
> > On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>
> wrote:
> > I've executed the commands as you've said, and the jar that is run is
> unchanged. Here's exactly what I did.
> >
> > C:\l\> start-dfs
> > C:\l\> start-yarn
> > C:\l\> cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>
> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> > ...output...
> > ...modify logging code in
> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>
> mvn clean install -DskipTests
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>
> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> > ...identical output to before...
> >
> > Do you see anything wrong with what I've done?
> >
> > Thanks,
> > Curtis
> >
> >
> >
> >
> > On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>
> wrote:
> > Hi Curtis,
> >
> > where are you picking your jar file from? once you run above command you
> will see the updated jar file in
> >
> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> >
> > I hope you are not using below jar file
> >
> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> >
> >
> > What chris has said is right.
> >
> > once you have taken latest code, you should follow below steps
> > 1) mvn clean install -DskipTests (clean to remove previously generated
> code)
> > 2) now say you are updating distributed shell client code. then go to
> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
> and then run "mvn clean install" and use jar from target sub folder.
> >
> > Thanks,
> > Omkar Joshi
> > Hortonworks Inc.
> >
> >
> > On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <
> curtullerich@gmail.com> wrote:
> > Hi Chris,
> >
> > I really appreciate the response. What you described is what I initially
> tried. The changes never seem to take effect though. Here's what I've done
> (this is Windows):
> >
> > cd %hadoop_install_dir%
> > mvn clean package -DskipTests
> > mvn install -DskipTests
> >
> > --- modify the code in distributed shell's Client.java ---
> >
> > cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> > mvn clean
> > mvn package -DskipTests
> > mvn install -DskipTests
> >
> > Then I run the jar, just as before. I've just been changing log
> statements to see if the changes worked. They don't--the output doesn't
> change after doing this rebuild. I've also tried clearing the files put on
> HDFS in my user directory in case that was the issue. I've taken a more
> thorough look at BUILDING.txt and I seemed to be consistent with the
> procedures described there. Am I missing anything else? I've tried
> restarting yarn and dfs, though I didn't think that would matter.
> >
> > Thanks,
> > Curtis
> >
> >
> >
> >
> > On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
> cnauroth@hortonworks.com> wrote:
> > Hi Curtis,
> >
> > I handle this by running "mvn install -DskipTests" once from the root of
> the whole hadoop project to install the sub-modules in my local Maven
> repository.  Then, you can change your working directory to any sub-module
> (like distributed shell) and run a successful Maven build for just that
> sub-module.  Full details are in the BUILDING.txt file in the root of the
> hadoop project.  Look for the section titled "Building components
> separately".  I hope this helps.
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>
> wrote:
> > What is the way to compile just a subproject of Hadoop with Maven
> (without rebuilding parent modules)? In my case I'm wondering about
> DistributedShell. I thought that this would have worked but no changes seem
> to take effect after building this way.
> >
> > Thanks,
> > Curtis
> >
> >
> >
> >
> >
> >
>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
Executing this doesn't seem to work either.
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami

Thank you for pointing out the classpath issue, though. I checked in
hadoop-dist/.../share and found that hadoop had been using the jar there.
The command can't find the Client class unless it's in that directory, so I
suppose a suitable workaround would be to add the distributedshell/target
directory to the classpath myself, or just copy the jar to /share. If
anyone has suggestions for the best way to get around that issue, I would
welcome them. Otherwise, this relieves my confusion--thanks for all the
help!

Curtis



On Fri, Jun 21, 2013 at 5:10 PM, Hitesh Shah <hi...@apache.org> wrote:

> Hello Curtis
>
> Try the following:
>
> hadoop jar ./target/distributed-shell.jar.rebuilt-one
> org.apache.hadoop.yarn.applications.distributedshell.Client -jar ...
>
> If you are running hadoop without the jar command, it will find the first
> instance of Client.class in its classpath which I am guessing is from the
> jars used to start the cluster and not the re-built jar.
>
> -- Hitesh
>
>
> On Jun 21, 2013, at 4:09 PM, Curtis Ullerich wrote:
>
> > I should clarify that the build doesn't actually fail (so that bug
> doesn't seem to be the issue). It reports success. But then the output from
> the just-build jar doesn't change.
> >
> > Here's my output from building:
> >
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn
> clean package -DskipTests
> > [INFO] Scanning for projects...
> > [INFO]
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO] Building hadoop-yarn-applications-distributedshell 3.0.0-SNAPSHOT
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO]
> > [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Deleting
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target
> > [INFO]
> > [INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Executing tasks
> >
> > main:
> >     [mkdir] Created dir:
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-dir
> > [INFO] Executed tasks
> > [INFO]
> > [INFO] --- maven-resources-plugin:2.2:resources (default-resources) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Using default encoding to copy filtered resources.
> > [INFO]
> > [INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Compiling 3 source files to
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\classes
> > [INFO]
> > [INFO] --- maven-resources-plugin:2.2:testResources
> (default-testResources) @ hadoop-yarn-applications-distributedshell ---
> > [INFO] Using default encoding to copy filtered resources.
> > [INFO]
> > [INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile)
> @ hadoop-yarn-applications-distributedshell ---
> > [INFO] Compiling 1 source file to
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-classes
> > [INFO]
> > [INFO] --- maven-jar-plugin:2.3.1:jar (default) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Building jar:
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> > [INFO]
> > [INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Tests are skipped.
> > [INFO]
> > [INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO]
> > [INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO] BUILD SUCCESS
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO] Total time: 3.487s
> > [INFO] Finished at: Fri Jun 21 16:08:02 PDT 2013
> > [INFO] Final Memory: 19M/367M
> > [INFO]
> ------------------------------------------------------------------------
> >
> >
> >
> > On Fri, Jun 21, 2013 at 3:57 PM, Arpit Agarwal <aa...@hortonworks.com>
> wrote:
> > Curtis, if you are running 'mvn clean' in a subproject you may be
> hitting this bug:
> > https://issues.apache.org/jira/browse/HADOOP-9383
> >
> > You might have better luck if you include install along with package in
> the same invocation.
> >
> > -Arpit
> >
> >
> > On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>
> wrote:
> > I've executed the commands as you've said, and the jar that is run is
> unchanged. Here's exactly what I did.
> >
> > C:\l\> start-dfs
> > C:\l\> start-yarn
> > C:\l\> cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>
> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> > ...output...
> > ...modify logging code in
> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>
> mvn clean install -DskipTests
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>
> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> > ...identical output to before...
> >
> > Do you see anything wrong with what I've done?
> >
> > Thanks,
> > Curtis
> >
> >
> >
> >
> > On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>
> wrote:
> > Hi Curtis,
> >
> > where are you picking your jar file from? once you run above command you
> will see the updated jar file in
> >
> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> >
> > I hope you are not using below jar file
> >
> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> >
> >
> > What chris has said is right.
> >
> > once you have taken latest code, you should follow below steps
> > 1) mvn clean install -DskipTests (clean to remove previously generated
> code)
> > 2) now say you are updating distributed shell client code. then go to
> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
> and then run "mvn clean install" and use jar from target sub folder.
> >
> > Thanks,
> > Omkar Joshi
> > Hortonworks Inc.
> >
> >
> > On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <
> curtullerich@gmail.com> wrote:
> > Hi Chris,
> >
> > I really appreciate the response. What you described is what I initially
> tried. The changes never seem to take effect though. Here's what I've done
> (this is Windows):
> >
> > cd %hadoop_install_dir%
> > mvn clean package -DskipTests
> > mvn install -DskipTests
> >
> > --- modify the code in distributed shell's Client.java ---
> >
> > cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> > mvn clean
> > mvn package -DskipTests
> > mvn install -DskipTests
> >
> > Then I run the jar, just as before. I've just been changing log
> statements to see if the changes worked. They don't--the output doesn't
> change after doing this rebuild. I've also tried clearing the files put on
> HDFS in my user directory in case that was the issue. I've taken a more
> thorough look at BUILDING.txt and I seemed to be consistent with the
> procedures described there. Am I missing anything else? I've tried
> restarting yarn and dfs, though I didn't think that would matter.
> >
> > Thanks,
> > Curtis
> >
> >
> >
> >
> > On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
> cnauroth@hortonworks.com> wrote:
> > Hi Curtis,
> >
> > I handle this by running "mvn install -DskipTests" once from the root of
> the whole hadoop project to install the sub-modules in my local Maven
> repository.  Then, you can change your working directory to any sub-module
> (like distributed shell) and run a successful Maven build for just that
> sub-module.  Full details are in the BUILDING.txt file in the root of the
> hadoop project.  Look for the section titled "Building components
> separately".  I hope this helps.
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>
> wrote:
> > What is the way to compile just a subproject of Hadoop with Maven
> (without rebuilding parent modules)? In my case I'm wondering about
> DistributedShell. I thought that this would have worked but no changes seem
> to take effect after building this way.
> >
> > Thanks,
> > Curtis
> >
> >
> >
> >
> >
> >
>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
Executing this doesn't seem to work either.
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami

Thank you for pointing out the classpath issue, though. I checked in
hadoop-dist/.../share and found that hadoop had been using the jar there.
The command can't find the Client class unless it's in that directory, so I
suppose a suitable workaround would be to add the distributedshell/target
directory to the classpath myself, or just copy the jar to /share. If
anyone has suggestions for the best way to get around that issue, I would
welcome them. Otherwise, this relieves my confusion--thanks for all the
help!

Curtis



On Fri, Jun 21, 2013 at 5:10 PM, Hitesh Shah <hi...@apache.org> wrote:

> Hello Curtis
>
> Try the following:
>
> hadoop jar ./target/distributed-shell.jar.rebuilt-one
> org.apache.hadoop.yarn.applications.distributedshell.Client -jar ...
>
> If you are running hadoop without the jar command, it will find the first
> instance of Client.class in its classpath which I am guessing is from the
> jars used to start the cluster and not the re-built jar.
>
> -- Hitesh
>
>
> On Jun 21, 2013, at 4:09 PM, Curtis Ullerich wrote:
>
> > I should clarify that the build doesn't actually fail (so that bug
> doesn't seem to be the issue). It reports success. But then the output from
> the just-build jar doesn't change.
> >
> > Here's my output from building:
> >
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn
> clean package -DskipTests
> > [INFO] Scanning for projects...
> > [INFO]
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO] Building hadoop-yarn-applications-distributedshell 3.0.0-SNAPSHOT
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO]
> > [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Deleting
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target
> > [INFO]
> > [INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Executing tasks
> >
> > main:
> >     [mkdir] Created dir:
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-dir
> > [INFO] Executed tasks
> > [INFO]
> > [INFO] --- maven-resources-plugin:2.2:resources (default-resources) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Using default encoding to copy filtered resources.
> > [INFO]
> > [INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Compiling 3 source files to
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\classes
> > [INFO]
> > [INFO] --- maven-resources-plugin:2.2:testResources
> (default-testResources) @ hadoop-yarn-applications-distributedshell ---
> > [INFO] Using default encoding to copy filtered resources.
> > [INFO]
> > [INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile)
> @ hadoop-yarn-applications-distributedshell ---
> > [INFO] Compiling 1 source file to
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-classes
> > [INFO]
> > [INFO] --- maven-jar-plugin:2.3.1:jar (default) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Building jar:
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> > [INFO]
> > [INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO] Tests are skipped.
> > [INFO]
> > [INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO]
> > [INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @
> hadoop-yarn-applications-distributedshell ---
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO] BUILD SUCCESS
> > [INFO]
> ------------------------------------------------------------------------
> > [INFO] Total time: 3.487s
> > [INFO] Finished at: Fri Jun 21 16:08:02 PDT 2013
> > [INFO] Final Memory: 19M/367M
> > [INFO]
> ------------------------------------------------------------------------
> >
> >
> >
> > On Fri, Jun 21, 2013 at 3:57 PM, Arpit Agarwal <aa...@hortonworks.com>
> wrote:
> > Curtis, if you are running 'mvn clean' in a subproject you may be
> hitting this bug:
> > https://issues.apache.org/jira/browse/HADOOP-9383
> >
> > You might have better luck if you include install along with package in
> the same invocation.
> >
> > -Arpit
> >
> >
> > On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>
> wrote:
> > I've executed the commands as you've said, and the jar that is run is
> unchanged. Here's exactly what I did.
> >
> > C:\l\> start-dfs
> > C:\l\> start-yarn
> > C:\l\> cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>
> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> > ...output...
> > ...modify logging code in
> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>
> mvn clean install -DskipTests
> >
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>
> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> > ...identical output to before...
> >
> > Do you see anything wrong with what I've done?
> >
> > Thanks,
> > Curtis
> >
> >
> >
> >
> > On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>
> wrote:
> > Hi Curtis,
> >
> > where are you picking your jar file from? once you run above command you
> will see the updated jar file in
> >
> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> >
> > I hope you are not using below jar file
> >
> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> >
> >
> > What chris has said is right.
> >
> > once you have taken latest code, you should follow below steps
> > 1) mvn clean install -DskipTests (clean to remove previously generated
> code)
> > 2) now say you are updating distributed shell client code. then go to
> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
> and then run "mvn clean install" and use jar from target sub folder.
> >
> > Thanks,
> > Omkar Joshi
> > Hortonworks Inc.
> >
> >
> > On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <
> curtullerich@gmail.com> wrote:
> > Hi Chris,
> >
> > I really appreciate the response. What you described is what I initially
> tried. The changes never seem to take effect though. Here's what I've done
> (this is Windows):
> >
> > cd %hadoop_install_dir%
> > mvn clean package -DskipTests
> > mvn install -DskipTests
> >
> > --- modify the code in distributed shell's Client.java ---
> >
> > cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> > mvn clean
> > mvn package -DskipTests
> > mvn install -DskipTests
> >
> > Then I run the jar, just as before. I've just been changing log
> statements to see if the changes worked. They don't--the output doesn't
> change after doing this rebuild. I've also tried clearing the files put on
> HDFS in my user directory in case that was the issue. I've taken a more
> thorough look at BUILDING.txt and I seemed to be consistent with the
> procedures described there. Am I missing anything else? I've tried
> restarting yarn and dfs, though I didn't think that would matter.
> >
> > Thanks,
> > Curtis
> >
> >
> >
> >
> > On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
> cnauroth@hortonworks.com> wrote:
> > Hi Curtis,
> >
> > I handle this by running "mvn install -DskipTests" once from the root of
> the whole hadoop project to install the sub-modules in my local Maven
> repository.  Then, you can change your working directory to any sub-module
> (like distributed shell) and run a successful Maven build for just that
> sub-module.  Full details are in the BUILDING.txt file in the root of the
> hadoop project.  Look for the section titled "Building components
> separately".  I hope this helps.
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>
> wrote:
> > What is the way to compile just a subproject of Hadoop with Maven
> (without rebuilding parent modules)? In my case I'm wondering about
> DistributedShell. I thought that this would have worked but no changes seem
> to take effect after building this way.
> >
> > Thanks,
> > Curtis
> >
> >
> >
> >
> >
> >
>
>

Re: Compile Just a Subproject

Posted by Hitesh Shah <hi...@apache.org>.
Hello Curtis 

Try the following:

hadoop jar ./target/distributed-shell.jar.rebuilt-one org.apache.hadoop.yarn.applications.distributedshell.Client -jar ...

If you are running hadoop without the jar command, it will find the first instance of Client.class in its classpath which I am guessing is from the jars used to start the cluster and not the re-built jar. 

-- Hitesh


On Jun 21, 2013, at 4:09 PM, Curtis Ullerich wrote:

> I should clarify that the build doesn't actually fail (so that bug doesn't seem to be the issue). It reports success. But then the output from the just-build jar doesn't change.
> 
> Here's my output from building:
> 
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean package -DskipTests
> [INFO] Scanning for projects...
> [INFO]
> [INFO] ------------------------------------------------------------------------
> [INFO] Building hadoop-yarn-applications-distributedshell 3.0.0-SNAPSHOT
> [INFO] ------------------------------------------------------------------------
> [INFO]
> [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Deleting C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target
> [INFO]
> [INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Executing tasks
> 
> main:
>     [mkdir] Created dir: C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-dir
> [INFO] Executed tasks
> [INFO]
> [INFO] --- maven-resources-plugin:2.2:resources (default-resources) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Using default encoding to copy filtered resources.
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Compiling 3 source files to C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\classes
> [INFO]
> [INFO] --- maven-resources-plugin:2.2:testResources (default-testResources) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Using default encoding to copy filtered resources.
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Compiling 1 source file to C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-classes
> [INFO]
> [INFO] --- maven-jar-plugin:2.3.1:jar (default) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Building jar: C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> [INFO]
> [INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Tests are skipped.
> [INFO]
> [INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ hadoop-yarn-applications-distributedshell ---
> [INFO]
> [INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @ hadoop-yarn-applications-distributedshell ---
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 3.487s
> [INFO] Finished at: Fri Jun 21 16:08:02 PDT 2013
> [INFO] Final Memory: 19M/367M
> [INFO] ------------------------------------------------------------------------
> 
> 
> 
> On Fri, Jun 21, 2013 at 3:57 PM, Arpit Agarwal <aa...@hortonworks.com> wrote:
> Curtis, if you are running 'mvn clean' in a subproject you may be hitting this bug:
> https://issues.apache.org/jira/browse/HADOOP-9383
> 
> You might have better luck if you include install along with package in the same invocation.
> 
> -Arpit
> 
> 
> On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com> wrote:
> I've executed the commands as you've said, and the jar that is run is unchanged. Here's exactly what I did.
> 
> C:\l\> start-dfs
> C:\l\> start-yarn
> C:\l\> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command whoami
> ...output...
> ...modify logging code in C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> mvn clean install -DskipTests
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command whoami
> ...identical output to before...
> 
> Do you see anything wrong with what I've done?
> 
> Thanks,
> Curtis
> 
> 
> 
> 
> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com> wrote:
> Hi Curtis,
> 
> where are you picking your jar file from? once you run above command you will see the updated jar file in
> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> 
> I hope you are not using below jar file
> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> 
> 
> What chris has said is right.
> 
> once you have taken latest code, you should follow below steps
> 1) mvn clean install -DskipTests (clean to remove previously generated code)
> 2) now say you are updating distributed shell client code. then go to ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/" and then run "mvn clean install" and use jar from target sub folder.
> 
> Thanks,
> Omkar Joshi
> Hortonworks Inc.
> 
> 
> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <cu...@gmail.com> wrote:
> Hi Chris,
> 
> I really appreciate the response. What you described is what I initially tried. The changes never seem to take effect though. Here's what I've done (this is Windows):
> 
> cd %hadoop_install_dir%
> mvn clean package -DskipTests
> mvn install -DskipTests
> 
> --- modify the code in distributed shell's Client.java ---
> 
> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> mvn clean
> mvn package -DskipTests
> mvn install -DskipTests
> 
> Then I run the jar, just as before. I've just been changing log statements to see if the changes worked. They don't--the output doesn't change after doing this rebuild. I've also tried clearing the files put on HDFS in my user directory in case that was the issue. I've taken a more thorough look at BUILDING.txt and I seemed to be consistent with the procedures described there. Am I missing anything else? I've tried restarting yarn and dfs, though I didn't think that would matter.
> 
> Thanks,
> Curtis
> 
> 
> 
> 
> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cn...@hortonworks.com> wrote:
> Hi Curtis,
> 
> I handle this by running "mvn install -DskipTests" once from the root of the whole hadoop project to install the sub-modules in my local Maven repository.  Then, you can change your working directory to any sub-module (like distributed shell) and run a successful Maven build for just that sub-module.  Full details are in the BUILDING.txt file in the root of the hadoop project.  Look for the section titled "Building components separately".  I hope this helps.
> 
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
> 
> 
> 
> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com> wrote:
> What is the way to compile just a subproject of Hadoop with Maven (without rebuilding parent modules)? In my case I'm wondering about DistributedShell. I thought that this would have worked but no changes seem to take effect after building this way.
> 
> Thanks,
> Curtis
> 
> 
> 
> 
> 
>

Re: Compile Just a Subproject

Posted by Hitesh Shah <hi...@apache.org>.
Hello Curtis 

Try the following:

hadoop jar ./target/distributed-shell.jar.rebuilt-one org.apache.hadoop.yarn.applications.distributedshell.Client -jar ...

If you are running hadoop without the jar command, it will find the first instance of Client.class in its classpath which I am guessing is from the jars used to start the cluster and not the re-built jar. 

-- Hitesh


On Jun 21, 2013, at 4:09 PM, Curtis Ullerich wrote:

> I should clarify that the build doesn't actually fail (so that bug doesn't seem to be the issue). It reports success. But then the output from the just-build jar doesn't change.
> 
> Here's my output from building:
> 
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean package -DskipTests
> [INFO] Scanning for projects...
> [INFO]
> [INFO] ------------------------------------------------------------------------
> [INFO] Building hadoop-yarn-applications-distributedshell 3.0.0-SNAPSHOT
> [INFO] ------------------------------------------------------------------------
> [INFO]
> [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Deleting C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target
> [INFO]
> [INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Executing tasks
> 
> main:
>     [mkdir] Created dir: C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-dir
> [INFO] Executed tasks
> [INFO]
> [INFO] --- maven-resources-plugin:2.2:resources (default-resources) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Using default encoding to copy filtered resources.
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Compiling 3 source files to C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\classes
> [INFO]
> [INFO] --- maven-resources-plugin:2.2:testResources (default-testResources) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Using default encoding to copy filtered resources.
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Compiling 1 source file to C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-classes
> [INFO]
> [INFO] --- maven-jar-plugin:2.3.1:jar (default) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Building jar: C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> [INFO]
> [INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Tests are skipped.
> [INFO]
> [INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ hadoop-yarn-applications-distributedshell ---
> [INFO]
> [INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @ hadoop-yarn-applications-distributedshell ---
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 3.487s
> [INFO] Finished at: Fri Jun 21 16:08:02 PDT 2013
> [INFO] Final Memory: 19M/367M
> [INFO] ------------------------------------------------------------------------
> 
> 
> 
> On Fri, Jun 21, 2013 at 3:57 PM, Arpit Agarwal <aa...@hortonworks.com> wrote:
> Curtis, if you are running 'mvn clean' in a subproject you may be hitting this bug:
> https://issues.apache.org/jira/browse/HADOOP-9383
> 
> You might have better luck if you include install along with package in the same invocation.
> 
> -Arpit
> 
> 
> On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com> wrote:
> I've executed the commands as you've said, and the jar that is run is unchanged. Here's exactly what I did.
> 
> C:\l\> start-dfs
> C:\l\> start-yarn
> C:\l\> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command whoami
> ...output...
> ...modify logging code in C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> mvn clean install -DskipTests
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command whoami
> ...identical output to before...
> 
> Do you see anything wrong with what I've done?
> 
> Thanks,
> Curtis
> 
> 
> 
> 
> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com> wrote:
> Hi Curtis,
> 
> where are you picking your jar file from? once you run above command you will see the updated jar file in
> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> 
> I hope you are not using below jar file
> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> 
> 
> What chris has said is right.
> 
> once you have taken latest code, you should follow below steps
> 1) mvn clean install -DskipTests (clean to remove previously generated code)
> 2) now say you are updating distributed shell client code. then go to ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/" and then run "mvn clean install" and use jar from target sub folder.
> 
> Thanks,
> Omkar Joshi
> Hortonworks Inc.
> 
> 
> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <cu...@gmail.com> wrote:
> Hi Chris,
> 
> I really appreciate the response. What you described is what I initially tried. The changes never seem to take effect though. Here's what I've done (this is Windows):
> 
> cd %hadoop_install_dir%
> mvn clean package -DskipTests
> mvn install -DskipTests
> 
> --- modify the code in distributed shell's Client.java ---
> 
> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> mvn clean
> mvn package -DskipTests
> mvn install -DskipTests
> 
> Then I run the jar, just as before. I've just been changing log statements to see if the changes worked. They don't--the output doesn't change after doing this rebuild. I've also tried clearing the files put on HDFS in my user directory in case that was the issue. I've taken a more thorough look at BUILDING.txt and I seemed to be consistent with the procedures described there. Am I missing anything else? I've tried restarting yarn and dfs, though I didn't think that would matter.
> 
> Thanks,
> Curtis
> 
> 
> 
> 
> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cn...@hortonworks.com> wrote:
> Hi Curtis,
> 
> I handle this by running "mvn install -DskipTests" once from the root of the whole hadoop project to install the sub-modules in my local Maven repository.  Then, you can change your working directory to any sub-module (like distributed shell) and run a successful Maven build for just that sub-module.  Full details are in the BUILDING.txt file in the root of the hadoop project.  Look for the section titled "Building components separately".  I hope this helps.
> 
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
> 
> 
> 
> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com> wrote:
> What is the way to compile just a subproject of Hadoop with Maven (without rebuilding parent modules)? In my case I'm wondering about DistributedShell. I thought that this would have worked but no changes seem to take effect after building this way.
> 
> Thanks,
> Curtis
> 
> 
> 
> 
> 
>

Re: Compile Just a Subproject

Posted by Hitesh Shah <hi...@apache.org>.
Hello Curtis 

Try the following:

hadoop jar ./target/distributed-shell.jar.rebuilt-one org.apache.hadoop.yarn.applications.distributedshell.Client -jar ...

If you are running hadoop without the jar command, it will find the first instance of Client.class in its classpath which I am guessing is from the jars used to start the cluster and not the re-built jar. 

-- Hitesh


On Jun 21, 2013, at 4:09 PM, Curtis Ullerich wrote:

> I should clarify that the build doesn't actually fail (so that bug doesn't seem to be the issue). It reports success. But then the output from the just-build jar doesn't change.
> 
> Here's my output from building:
> 
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean package -DskipTests
> [INFO] Scanning for projects...
> [INFO]
> [INFO] ------------------------------------------------------------------------
> [INFO] Building hadoop-yarn-applications-distributedshell 3.0.0-SNAPSHOT
> [INFO] ------------------------------------------------------------------------
> [INFO]
> [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Deleting C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target
> [INFO]
> [INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Executing tasks
> 
> main:
>     [mkdir] Created dir: C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-dir
> [INFO] Executed tasks
> [INFO]
> [INFO] --- maven-resources-plugin:2.2:resources (default-resources) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Using default encoding to copy filtered resources.
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Compiling 3 source files to C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\classes
> [INFO]
> [INFO] --- maven-resources-plugin:2.2:testResources (default-testResources) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Using default encoding to copy filtered resources.
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Compiling 1 source file to C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-classes
> [INFO]
> [INFO] --- maven-jar-plugin:2.3.1:jar (default) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Building jar: C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> [INFO]
> [INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Tests are skipped.
> [INFO]
> [INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ hadoop-yarn-applications-distributedshell ---
> [INFO]
> [INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @ hadoop-yarn-applications-distributedshell ---
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 3.487s
> [INFO] Finished at: Fri Jun 21 16:08:02 PDT 2013
> [INFO] Final Memory: 19M/367M
> [INFO] ------------------------------------------------------------------------
> 
> 
> 
> On Fri, Jun 21, 2013 at 3:57 PM, Arpit Agarwal <aa...@hortonworks.com> wrote:
> Curtis, if you are running 'mvn clean' in a subproject you may be hitting this bug:
> https://issues.apache.org/jira/browse/HADOOP-9383
> 
> You might have better luck if you include install along with package in the same invocation.
> 
> -Arpit
> 
> 
> On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com> wrote:
> I've executed the commands as you've said, and the jar that is run is unchanged. Here's exactly what I did.
> 
> C:\l\> start-dfs
> C:\l\> start-yarn
> C:\l\> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command whoami
> ...output...
> ...modify logging code in C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> mvn clean install -DskipTests
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command whoami
> ...identical output to before...
> 
> Do you see anything wrong with what I've done?
> 
> Thanks,
> Curtis
> 
> 
> 
> 
> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com> wrote:
> Hi Curtis,
> 
> where are you picking your jar file from? once you run above command you will see the updated jar file in
> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> 
> I hope you are not using below jar file
> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> 
> 
> What chris has said is right.
> 
> once you have taken latest code, you should follow below steps
> 1) mvn clean install -DskipTests (clean to remove previously generated code)
> 2) now say you are updating distributed shell client code. then go to ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/" and then run "mvn clean install" and use jar from target sub folder.
> 
> Thanks,
> Omkar Joshi
> Hortonworks Inc.
> 
> 
> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <cu...@gmail.com> wrote:
> Hi Chris,
> 
> I really appreciate the response. What you described is what I initially tried. The changes never seem to take effect though. Here's what I've done (this is Windows):
> 
> cd %hadoop_install_dir%
> mvn clean package -DskipTests
> mvn install -DskipTests
> 
> --- modify the code in distributed shell's Client.java ---
> 
> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> mvn clean
> mvn package -DskipTests
> mvn install -DskipTests
> 
> Then I run the jar, just as before. I've just been changing log statements to see if the changes worked. They don't--the output doesn't change after doing this rebuild. I've also tried clearing the files put on HDFS in my user directory in case that was the issue. I've taken a more thorough look at BUILDING.txt and I seemed to be consistent with the procedures described there. Am I missing anything else? I've tried restarting yarn and dfs, though I didn't think that would matter.
> 
> Thanks,
> Curtis
> 
> 
> 
> 
> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cn...@hortonworks.com> wrote:
> Hi Curtis,
> 
> I handle this by running "mvn install -DskipTests" once from the root of the whole hadoop project to install the sub-modules in my local Maven repository.  Then, you can change your working directory to any sub-module (like distributed shell) and run a successful Maven build for just that sub-module.  Full details are in the BUILDING.txt file in the root of the hadoop project.  Look for the section titled "Building components separately".  I hope this helps.
> 
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
> 
> 
> 
> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com> wrote:
> What is the way to compile just a subproject of Hadoop with Maven (without rebuilding parent modules)? In my case I'm wondering about DistributedShell. I thought that this would have worked but no changes seem to take effect after building this way.
> 
> Thanks,
> Curtis
> 
> 
> 
> 
> 
>

Re: Compile Just a Subproject

Posted by Hitesh Shah <hi...@apache.org>.
Hello Curtis 

Try the following:

hadoop jar ./target/distributed-shell.jar.rebuilt-one org.apache.hadoop.yarn.applications.distributedshell.Client -jar ...

If you are running hadoop without the jar command, it will find the first instance of Client.class in its classpath which I am guessing is from the jars used to start the cluster and not the re-built jar. 

-- Hitesh


On Jun 21, 2013, at 4:09 PM, Curtis Ullerich wrote:

> I should clarify that the build doesn't actually fail (so that bug doesn't seem to be the issue). It reports success. But then the output from the just-build jar doesn't change.
> 
> Here's my output from building:
> 
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean package -DskipTests
> [INFO] Scanning for projects...
> [INFO]
> [INFO] ------------------------------------------------------------------------
> [INFO] Building hadoop-yarn-applications-distributedshell 3.0.0-SNAPSHOT
> [INFO] ------------------------------------------------------------------------
> [INFO]
> [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Deleting C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target
> [INFO]
> [INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Executing tasks
> 
> main:
>     [mkdir] Created dir: C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-dir
> [INFO] Executed tasks
> [INFO]
> [INFO] --- maven-resources-plugin:2.2:resources (default-resources) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Using default encoding to copy filtered resources.
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Compiling 3 source files to C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\classes
> [INFO]
> [INFO] --- maven-resources-plugin:2.2:testResources (default-testResources) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Using default encoding to copy filtered resources.
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Compiling 1 source file to C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-classes
> [INFO]
> [INFO] --- maven-jar-plugin:2.3.1:jar (default) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Building jar: C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> [INFO]
> [INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @ hadoop-yarn-applications-distributedshell ---
> [INFO] Tests are skipped.
> [INFO]
> [INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ hadoop-yarn-applications-distributedshell ---
> [INFO]
> [INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @ hadoop-yarn-applications-distributedshell ---
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 3.487s
> [INFO] Finished at: Fri Jun 21 16:08:02 PDT 2013
> [INFO] Final Memory: 19M/367M
> [INFO] ------------------------------------------------------------------------
> 
> 
> 
> On Fri, Jun 21, 2013 at 3:57 PM, Arpit Agarwal <aa...@hortonworks.com> wrote:
> Curtis, if you are running 'mvn clean' in a subproject you may be hitting this bug:
> https://issues.apache.org/jira/browse/HADOOP-9383
> 
> You might have better luck if you include install along with package in the same invocation.
> 
> -Arpit
> 
> 
> On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com> wrote:
> I've executed the commands as you've said, and the jar that is run is unchanged. Here's exactly what I did.
> 
> C:\l\> start-dfs
> C:\l\> start-yarn
> C:\l\> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command whoami
> ...output...
> ...modify logging code in C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> mvn clean install -DskipTests
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command whoami
> ...identical output to before...
> 
> Do you see anything wrong with what I've done?
> 
> Thanks,
> Curtis
> 
> 
> 
> 
> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com> wrote:
> Hi Curtis,
> 
> where are you picking your jar file from? once you run above command you will see the updated jar file in
> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> 
> I hope you are not using below jar file
> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
> 
> 
> What chris has said is right.
> 
> once you have taken latest code, you should follow below steps
> 1) mvn clean install -DskipTests (clean to remove previously generated code)
> 2) now say you are updating distributed shell client code. then go to ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/" and then run "mvn clean install" and use jar from target sub folder.
> 
> Thanks,
> Omkar Joshi
> Hortonworks Inc.
> 
> 
> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <cu...@gmail.com> wrote:
> Hi Chris,
> 
> I really appreciate the response. What you described is what I initially tried. The changes never seem to take effect though. Here's what I've done (this is Windows):
> 
> cd %hadoop_install_dir%
> mvn clean package -DskipTests
> mvn install -DskipTests
> 
> --- modify the code in distributed shell's Client.java ---
> 
> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> mvn clean
> mvn package -DskipTests
> mvn install -DskipTests
> 
> Then I run the jar, just as before. I've just been changing log statements to see if the changes worked. They don't--the output doesn't change after doing this rebuild. I've also tried clearing the files put on HDFS in my user directory in case that was the issue. I've taken a more thorough look at BUILDING.txt and I seemed to be consistent with the procedures described there. Am I missing anything else? I've tried restarting yarn and dfs, though I didn't think that would matter.
> 
> Thanks,
> Curtis
> 
> 
> 
> 
> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cn...@hortonworks.com> wrote:
> Hi Curtis,
> 
> I handle this by running "mvn install -DskipTests" once from the root of the whole hadoop project to install the sub-modules in my local Maven repository.  Then, you can change your working directory to any sub-module (like distributed shell) and run a successful Maven build for just that sub-module.  Full details are in the BUILDING.txt file in the root of the hadoop project.  Look for the section titled "Building components separately".  I hope this helps.
> 
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
> 
> 
> 
> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com> wrote:
> What is the way to compile just a subproject of Hadoop with Maven (without rebuilding parent modules)? In my case I'm wondering about DistributedShell. I thought that this would have worked but no changes seem to take effect after building this way.
> 
> Thanks,
> Curtis
> 
> 
> 
> 
> 
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
I should clarify that the build doesn't actually fail (so that bug doesn't
seem to be the issue). It reports success. But then the output from the
just-build jar doesn't change.

Here's my output from building:

C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn
clean package -DskipTests
[INFO] Scanning for projects...
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building hadoop-yarn-applications-distributedshell 3.0.0-SNAPSHOT
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @
hadoop-yarn-applications-distributedshell ---
[INFO] Deleting
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @
hadoop-yarn-applications-distributedshell ---
[INFO] Executing tasks

main:
    [mkdir] Created dir:
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-dir
[INFO] Executed tasks
[INFO]
[INFO] --- maven-resources-plugin:2.2:resources (default-resources) @
hadoop-yarn-applications-distributedshell ---
[INFO] Using default encoding to copy filtered resources.
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @
hadoop-yarn-applications-distributedshell ---
[INFO] Compiling 3 source files to
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\classes
[INFO]
[INFO] --- maven-resources-plugin:2.2:testResources (default-testResources)
@ hadoop-yarn-applications-distributedshell ---
[INFO] Using default encoding to copy filtered resources.
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @
hadoop-yarn-applications-distributedshell ---
[INFO] Compiling 1 source file to
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-classes
[INFO]
[INFO] --- maven-jar-plugin:2.3.1:jar (default) @
hadoop-yarn-applications-distributedshell ---
[INFO] Building jar:
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @
hadoop-yarn-applications-distributedshell ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @
hadoop-yarn-applications-distributedshell ---
[INFO]
[INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @
hadoop-yarn-applications-distributedshell ---
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 3.487s
[INFO] Finished at: Fri Jun 21 16:08:02 PDT 2013
[INFO] Final Memory: 19M/367M
[INFO]
------------------------------------------------------------------------



On Fri, Jun 21, 2013 at 3:57 PM, Arpit Agarwal <aa...@hortonworks.com>wrote:

> Curtis, if you are running 'mvn clean' in a subproject you may be hitting
> this bug:
> https://issues.apache.org/jira/browse/HADOOP-9383
>
> You might have better luck if you include install along with package in
> the same invocation.
>
> -Arpit
>
>
> On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> I've executed the commands as you've said, and the jar that is run is
>> unchanged. Here's exactly what I did.
>>
>> C:\l\> start-dfs
>> C:\l\> start-yarn
>> C:\l\> cd
>> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...output...
>> ...modify logging code in
>> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...identical output to before...
>>
>> Do you see anything wrong with what I've done?
>>
>> Thanks,
>> Curtis
>>
>>
>>
>>
>> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>>
>>> Hi Curtis,
>>>
>>> where are you picking your jar file from? once you run above command you
>>> will see the updated jar file in
>>>
>>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>> I hope you are not using below jar file
>>>
>>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>>
>>> What chris has said is right.
>>>
>>> once you have taken latest code, you should follow below steps
>>> 1) mvn clean install -DskipTests (clean to remove previously generated
>>> code)
>>> 2) now say you are updating distributed shell client code. then go to
>>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>>> and then run "mvn clean install" and use jar from target sub folder.
>>>
>>> Thanks,
>>> Omkar Joshi
>>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <
>>> curtullerich@gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> I really appreciate the response. What you described is what I
>>>> initially tried. The changes never seem to take effect though. Here's what
>>>> I've done (this is Windows):
>>>>
>>>> cd %hadoop_install_dir%
>>>> mvn clean package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> --- modify the code in distributed shell's Client.java ---
>>>>
>>>>
>>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>>> mvn clean
>>>> mvn package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> Then I run the jar, just as before. I've just been changing log
>>>> statements to see if the changes worked. They don't--the output doesn't
>>>> change after doing this rebuild. I've also tried clearing the files put on
>>>> HDFS in my user directory in case that was the issue. I've taken a more
>>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>>> procedures described there. Am I missing anything else? I've tried
>>>> restarting yarn and dfs, though I didn't think that would matter.
>>>>
>>>> Thanks,
>>>> Curtis
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>>> cnauroth@hortonworks.com> wrote:
>>>>
>>>>> Hi Curtis,
>>>>>
>>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>>> repository.  Then, you can change your working directory to any sub-module
>>>>> (like distributed shell) and run a successful Maven build for just that
>>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>>> hadoop project.  Look for the section titled "Building components
>>>>> separately".  I hope this helps.
>>>>>
>>>>> Chris Nauroth
>>>>> Hortonworks
>>>>> http://hortonworks.com/
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>>> curtullerich@gmail.com> wrote:
>>>>>
>>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>>> way.
>>>>>>
>>>>>> Thanks,
>>>>>> Curtis
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
I should clarify that the build doesn't actually fail (so that bug doesn't
seem to be the issue). It reports success. But then the output from the
just-build jar doesn't change.

Here's my output from building:

C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn
clean package -DskipTests
[INFO] Scanning for projects...
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building hadoop-yarn-applications-distributedshell 3.0.0-SNAPSHOT
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @
hadoop-yarn-applications-distributedshell ---
[INFO] Deleting
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @
hadoop-yarn-applications-distributedshell ---
[INFO] Executing tasks

main:
    [mkdir] Created dir:
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-dir
[INFO] Executed tasks
[INFO]
[INFO] --- maven-resources-plugin:2.2:resources (default-resources) @
hadoop-yarn-applications-distributedshell ---
[INFO] Using default encoding to copy filtered resources.
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @
hadoop-yarn-applications-distributedshell ---
[INFO] Compiling 3 source files to
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\classes
[INFO]
[INFO] --- maven-resources-plugin:2.2:testResources (default-testResources)
@ hadoop-yarn-applications-distributedshell ---
[INFO] Using default encoding to copy filtered resources.
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @
hadoop-yarn-applications-distributedshell ---
[INFO] Compiling 1 source file to
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-classes
[INFO]
[INFO] --- maven-jar-plugin:2.3.1:jar (default) @
hadoop-yarn-applications-distributedshell ---
[INFO] Building jar:
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @
hadoop-yarn-applications-distributedshell ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @
hadoop-yarn-applications-distributedshell ---
[INFO]
[INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @
hadoop-yarn-applications-distributedshell ---
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 3.487s
[INFO] Finished at: Fri Jun 21 16:08:02 PDT 2013
[INFO] Final Memory: 19M/367M
[INFO]
------------------------------------------------------------------------



On Fri, Jun 21, 2013 at 3:57 PM, Arpit Agarwal <aa...@hortonworks.com>wrote:

> Curtis, if you are running 'mvn clean' in a subproject you may be hitting
> this bug:
> https://issues.apache.org/jira/browse/HADOOP-9383
>
> You might have better luck if you include install along with package in
> the same invocation.
>
> -Arpit
>
>
> On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> I've executed the commands as you've said, and the jar that is run is
>> unchanged. Here's exactly what I did.
>>
>> C:\l\> start-dfs
>> C:\l\> start-yarn
>> C:\l\> cd
>> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...output...
>> ...modify logging code in
>> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...identical output to before...
>>
>> Do you see anything wrong with what I've done?
>>
>> Thanks,
>> Curtis
>>
>>
>>
>>
>> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>>
>>> Hi Curtis,
>>>
>>> where are you picking your jar file from? once you run above command you
>>> will see the updated jar file in
>>>
>>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>> I hope you are not using below jar file
>>>
>>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>>
>>> What chris has said is right.
>>>
>>> once you have taken latest code, you should follow below steps
>>> 1) mvn clean install -DskipTests (clean to remove previously generated
>>> code)
>>> 2) now say you are updating distributed shell client code. then go to
>>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>>> and then run "mvn clean install" and use jar from target sub folder.
>>>
>>> Thanks,
>>> Omkar Joshi
>>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <
>>> curtullerich@gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> I really appreciate the response. What you described is what I
>>>> initially tried. The changes never seem to take effect though. Here's what
>>>> I've done (this is Windows):
>>>>
>>>> cd %hadoop_install_dir%
>>>> mvn clean package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> --- modify the code in distributed shell's Client.java ---
>>>>
>>>>
>>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>>> mvn clean
>>>> mvn package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> Then I run the jar, just as before. I've just been changing log
>>>> statements to see if the changes worked. They don't--the output doesn't
>>>> change after doing this rebuild. I've also tried clearing the files put on
>>>> HDFS in my user directory in case that was the issue. I've taken a more
>>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>>> procedures described there. Am I missing anything else? I've tried
>>>> restarting yarn and dfs, though I didn't think that would matter.
>>>>
>>>> Thanks,
>>>> Curtis
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>>> cnauroth@hortonworks.com> wrote:
>>>>
>>>>> Hi Curtis,
>>>>>
>>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>>> repository.  Then, you can change your working directory to any sub-module
>>>>> (like distributed shell) and run a successful Maven build for just that
>>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>>> hadoop project.  Look for the section titled "Building components
>>>>> separately".  I hope this helps.
>>>>>
>>>>> Chris Nauroth
>>>>> Hortonworks
>>>>> http://hortonworks.com/
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>>> curtullerich@gmail.com> wrote:
>>>>>
>>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>>> way.
>>>>>>
>>>>>> Thanks,
>>>>>> Curtis
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
I should clarify that the build doesn't actually fail (so that bug doesn't
seem to be the issue). It reports success. But then the output from the
just-build jar doesn't change.

Here's my output from building:

C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn
clean package -DskipTests
[INFO] Scanning for projects...
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building hadoop-yarn-applications-distributedshell 3.0.0-SNAPSHOT
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @
hadoop-yarn-applications-distributedshell ---
[INFO] Deleting
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @
hadoop-yarn-applications-distributedshell ---
[INFO] Executing tasks

main:
    [mkdir] Created dir:
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-dir
[INFO] Executed tasks
[INFO]
[INFO] --- maven-resources-plugin:2.2:resources (default-resources) @
hadoop-yarn-applications-distributedshell ---
[INFO] Using default encoding to copy filtered resources.
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @
hadoop-yarn-applications-distributedshell ---
[INFO] Compiling 3 source files to
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\classes
[INFO]
[INFO] --- maven-resources-plugin:2.2:testResources (default-testResources)
@ hadoop-yarn-applications-distributedshell ---
[INFO] Using default encoding to copy filtered resources.
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @
hadoop-yarn-applications-distributedshell ---
[INFO] Compiling 1 source file to
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-classes
[INFO]
[INFO] --- maven-jar-plugin:2.3.1:jar (default) @
hadoop-yarn-applications-distributedshell ---
[INFO] Building jar:
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @
hadoop-yarn-applications-distributedshell ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @
hadoop-yarn-applications-distributedshell ---
[INFO]
[INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @
hadoop-yarn-applications-distributedshell ---
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 3.487s
[INFO] Finished at: Fri Jun 21 16:08:02 PDT 2013
[INFO] Final Memory: 19M/367M
[INFO]
------------------------------------------------------------------------



On Fri, Jun 21, 2013 at 3:57 PM, Arpit Agarwal <aa...@hortonworks.com>wrote:

> Curtis, if you are running 'mvn clean' in a subproject you may be hitting
> this bug:
> https://issues.apache.org/jira/browse/HADOOP-9383
>
> You might have better luck if you include install along with package in
> the same invocation.
>
> -Arpit
>
>
> On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> I've executed the commands as you've said, and the jar that is run is
>> unchanged. Here's exactly what I did.
>>
>> C:\l\> start-dfs
>> C:\l\> start-yarn
>> C:\l\> cd
>> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...output...
>> ...modify logging code in
>> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...identical output to before...
>>
>> Do you see anything wrong with what I've done?
>>
>> Thanks,
>> Curtis
>>
>>
>>
>>
>> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>>
>>> Hi Curtis,
>>>
>>> where are you picking your jar file from? once you run above command you
>>> will see the updated jar file in
>>>
>>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>> I hope you are not using below jar file
>>>
>>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>>
>>> What chris has said is right.
>>>
>>> once you have taken latest code, you should follow below steps
>>> 1) mvn clean install -DskipTests (clean to remove previously generated
>>> code)
>>> 2) now say you are updating distributed shell client code. then go to
>>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>>> and then run "mvn clean install" and use jar from target sub folder.
>>>
>>> Thanks,
>>> Omkar Joshi
>>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <
>>> curtullerich@gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> I really appreciate the response. What you described is what I
>>>> initially tried. The changes never seem to take effect though. Here's what
>>>> I've done (this is Windows):
>>>>
>>>> cd %hadoop_install_dir%
>>>> mvn clean package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> --- modify the code in distributed shell's Client.java ---
>>>>
>>>>
>>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>>> mvn clean
>>>> mvn package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> Then I run the jar, just as before. I've just been changing log
>>>> statements to see if the changes worked. They don't--the output doesn't
>>>> change after doing this rebuild. I've also tried clearing the files put on
>>>> HDFS in my user directory in case that was the issue. I've taken a more
>>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>>> procedures described there. Am I missing anything else? I've tried
>>>> restarting yarn and dfs, though I didn't think that would matter.
>>>>
>>>> Thanks,
>>>> Curtis
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>>> cnauroth@hortonworks.com> wrote:
>>>>
>>>>> Hi Curtis,
>>>>>
>>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>>> repository.  Then, you can change your working directory to any sub-module
>>>>> (like distributed shell) and run a successful Maven build for just that
>>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>>> hadoop project.  Look for the section titled "Building components
>>>>> separately".  I hope this helps.
>>>>>
>>>>> Chris Nauroth
>>>>> Hortonworks
>>>>> http://hortonworks.com/
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>>> curtullerich@gmail.com> wrote:
>>>>>
>>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>>> way.
>>>>>>
>>>>>> Thanks,
>>>>>> Curtis
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
I should clarify that the build doesn't actually fail (so that bug doesn't
seem to be the issue). It reports success. But then the output from the
just-build jar doesn't change.

Here's my output from building:

C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn
clean package -DskipTests
[INFO] Scanning for projects...
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building hadoop-yarn-applications-distributedshell 3.0.0-SNAPSHOT
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @
hadoop-yarn-applications-distributedshell ---
[INFO] Deleting
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @
hadoop-yarn-applications-distributedshell ---
[INFO] Executing tasks

main:
    [mkdir] Created dir:
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-dir
[INFO] Executed tasks
[INFO]
[INFO] --- maven-resources-plugin:2.2:resources (default-resources) @
hadoop-yarn-applications-distributedshell ---
[INFO] Using default encoding to copy filtered resources.
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @
hadoop-yarn-applications-distributedshell ---
[INFO] Compiling 3 source files to
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\classes
[INFO]
[INFO] --- maven-resources-plugin:2.2:testResources (default-testResources)
@ hadoop-yarn-applications-distributedshell ---
[INFO] Using default encoding to copy filtered resources.
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @
hadoop-yarn-applications-distributedshell ---
[INFO] Compiling 1 source file to
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-classes
[INFO]
[INFO] --- maven-jar-plugin:2.3.1:jar (default) @
hadoop-yarn-applications-distributedshell ---
[INFO] Building jar:
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @
hadoop-yarn-applications-distributedshell ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @
hadoop-yarn-applications-distributedshell ---
[INFO]
[INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @
hadoop-yarn-applications-distributedshell ---
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 3.487s
[INFO] Finished at: Fri Jun 21 16:08:02 PDT 2013
[INFO] Final Memory: 19M/367M
[INFO]
------------------------------------------------------------------------



On Fri, Jun 21, 2013 at 3:57 PM, Arpit Agarwal <aa...@hortonworks.com>wrote:

> Curtis, if you are running 'mvn clean' in a subproject you may be hitting
> this bug:
> https://issues.apache.org/jira/browse/HADOOP-9383
>
> You might have better luck if you include install along with package in
> the same invocation.
>
> -Arpit
>
>
> On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> I've executed the commands as you've said, and the jar that is run is
>> unchanged. Here's exactly what I did.
>>
>> C:\l\> start-dfs
>> C:\l\> start-yarn
>> C:\l\> cd
>> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...output...
>> ...modify logging code in
>> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...identical output to before...
>>
>> Do you see anything wrong with what I've done?
>>
>> Thanks,
>> Curtis
>>
>>
>>
>>
>> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>>
>>> Hi Curtis,
>>>
>>> where are you picking your jar file from? once you run above command you
>>> will see the updated jar file in
>>>
>>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>> I hope you are not using below jar file
>>>
>>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>>
>>> What chris has said is right.
>>>
>>> once you have taken latest code, you should follow below steps
>>> 1) mvn clean install -DskipTests (clean to remove previously generated
>>> code)
>>> 2) now say you are updating distributed shell client code. then go to
>>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>>> and then run "mvn clean install" and use jar from target sub folder.
>>>
>>> Thanks,
>>> Omkar Joshi
>>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <
>>> curtullerich@gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> I really appreciate the response. What you described is what I
>>>> initially tried. The changes never seem to take effect though. Here's what
>>>> I've done (this is Windows):
>>>>
>>>> cd %hadoop_install_dir%
>>>> mvn clean package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> --- modify the code in distributed shell's Client.java ---
>>>>
>>>>
>>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>>> mvn clean
>>>> mvn package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> Then I run the jar, just as before. I've just been changing log
>>>> statements to see if the changes worked. They don't--the output doesn't
>>>> change after doing this rebuild. I've also tried clearing the files put on
>>>> HDFS in my user directory in case that was the issue. I've taken a more
>>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>>> procedures described there. Am I missing anything else? I've tried
>>>> restarting yarn and dfs, though I didn't think that would matter.
>>>>
>>>> Thanks,
>>>> Curtis
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>>> cnauroth@hortonworks.com> wrote:
>>>>
>>>>> Hi Curtis,
>>>>>
>>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>>> repository.  Then, you can change your working directory to any sub-module
>>>>> (like distributed shell) and run a successful Maven build for just that
>>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>>> hadoop project.  Look for the section titled "Building components
>>>>> separately".  I hope this helps.
>>>>>
>>>>> Chris Nauroth
>>>>> Hortonworks
>>>>> http://hortonworks.com/
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>>> curtullerich@gmail.com> wrote:
>>>>>
>>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>>> way.
>>>>>>
>>>>>> Thanks,
>>>>>> Curtis
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Arpit Agarwal <aa...@hortonworks.com>.
Curtis, if you are running 'mvn clean' in a subproject you may be hitting
this bug:
https://issues.apache.org/jira/browse/HADOOP-9383

You might have better luck if you include install along with package in the
same invocation.

-Arpit

On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:

> I've executed the commands as you've said, and the jar that is run is
> unchanged. Here's exactly what I did.
>
> C:\l\> start-dfs
> C:\l\> start-yarn
> C:\l\> cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...output...
> ...modify logging code in
> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...identical output to before...
>
> Do you see anything wrong with what I've done?
>
> Thanks,
> Curtis
>
>
>
>
> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>
>> Hi Curtis,
>>
>> where are you picking your jar file from? once you run above command you
>> will see the updated jar file in
>>
>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>> I hope you are not using below jar file
>>
>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>>
>> What chris has said is right.
>>
>> once you have taken latest code, you should follow below steps
>> 1) mvn clean install -DskipTests (clean to remove previously generated
>> code)
>> 2) now say you are updating distributed shell client code. then go to
>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>> and then run "mvn clean install" and use jar from target sub folder.
>>
>> Thanks,
>> Omkar Joshi
>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>
>>
>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <curtullerich@gmail.com
>> > wrote:
>>
>>> Hi Chris,
>>>
>>> I really appreciate the response. What you described is what I initially
>>> tried. The changes never seem to take effect though. Here's what I've done
>>> (this is Windows):
>>>
>>> cd %hadoop_install_dir%
>>> mvn clean package -DskipTests
>>> mvn install -DskipTests
>>>
>>> --- modify the code in distributed shell's Client.java ---
>>>
>>>
>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>> mvn clean
>>> mvn package -DskipTests
>>> mvn install -DskipTests
>>>
>>> Then I run the jar, just as before. I've just been changing log
>>> statements to see if the changes worked. They don't--the output doesn't
>>> change after doing this rebuild. I've also tried clearing the files put on
>>> HDFS in my user directory in case that was the issue. I've taken a more
>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>> procedures described there. Am I missing anything else? I've tried
>>> restarting yarn and dfs, though I didn't think that would matter.
>>>
>>> Thanks,
>>> Curtis
>>>
>>>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>> cnauroth@hortonworks.com> wrote:
>>>
>>>> Hi Curtis,
>>>>
>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>> repository.  Then, you can change your working directory to any sub-module
>>>> (like distributed shell) and run a successful Maven build for just that
>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>> hadoop project.  Look for the section titled "Building components
>>>> separately".  I hope this helps.
>>>>
>>>> Chris Nauroth
>>>> Hortonworks
>>>> http://hortonworks.com/
>>>>
>>>>
>>>>
>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>> curtullerich@gmail.com> wrote:
>>>>
>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>> way.
>>>>>
>>>>> Thanks,
>>>>> Curtis
>>>>>
>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
Hi Omkar,

That was a typo, good catch. The path to Client.java should be

C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-
applications-distributedshell
/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java

Running as you suggest does build the jar in target with an updated
timestamp. I just need to copy it into
C:\l\hadoop-dist\target\hadoop-3.0.0-SNAPSHOT\share\hadoop\yarn and
override the version packaged there, and then things run as expected.

Thanks for your help!

Curtis



On Fri, Jun 21, 2013 at 7:49 PM, Omkar Joshi <oj...@hortonworks.com> wrote:

>
> Hi Curtis...I see a mismatch in paths which you have mentioned..are they
> same and was a typo or a mistake?
>
> ...modify logging code in *C:/l/hadoop-yarn-applications-**
> distributedshell/src/main/java/org/apache/hadoop/yarn/
> applications/distributedshell/Client.java*
> *
> *
> and
>
> *
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> *>mvn clean install -DskipTests
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
> org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> *
> *
>
> looks like you also have another directory named "*
> hadoop-yarn-applications-**distributedshell" *copied inside c:\1\...
>
>
> to make it more simple...go to "*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\"
> directory and clean everything and then run the "mvn clean install" command
> from *"*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell"
> directory.*
> *
> *
> *
> *
> *Let me know if you can now see the jar file (with updated time stamp) or
> not in target directory.*
>
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
>
>
> On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> I've executed the commands as you've said, and the jar that is run is
>> unchanged. Here's exactly what I did.
>>
>> C:\l\> start-dfs
>> C:\l\> start-yarn
>> C:\l\> cd
>> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...output...
>> ...modify logging code in
>> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...identical output to before...
>>
>> Do you see anything wrong with what I've done?
>>
>> Thanks,
>> Curtis
>>
>>
>>
>>
>> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>>
>>> Hi Curtis,
>>>
>>> where are you picking your jar file from? once you run above command you
>>> will see the updated jar file in
>>>
>>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>> I hope you are not using below jar file
>>>
>>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>>
>>> What chris has said is right.
>>>
>>> once you have taken latest code, you should follow below steps
>>> 1) mvn clean install -DskipTests (clean to remove previously generated
>>> code)
>>> 2) now say you are updating distributed shell client code. then go to
>>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>>> and then run "mvn clean install" and use jar from target sub folder.
>>>
>>> Thanks,
>>> Omkar Joshi
>>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <
>>> curtullerich@gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> I really appreciate the response. What you described is what I
>>>> initially tried. The changes never seem to take effect though. Here's what
>>>> I've done (this is Windows):
>>>>
>>>> cd %hadoop_install_dir%
>>>> mvn clean package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> --- modify the code in distributed shell's Client.java ---
>>>>
>>>>
>>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>>> mvn clean
>>>> mvn package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> Then I run the jar, just as before. I've just been changing log
>>>> statements to see if the changes worked. They don't--the output doesn't
>>>> change after doing this rebuild. I've also tried clearing the files put on
>>>> HDFS in my user directory in case that was the issue. I've taken a more
>>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>>> procedures described there. Am I missing anything else? I've tried
>>>> restarting yarn and dfs, though I didn't think that would matter.
>>>>
>>>> Thanks,
>>>> Curtis
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>>> cnauroth@hortonworks.com> wrote:
>>>>
>>>>> Hi Curtis,
>>>>>
>>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>>> repository.  Then, you can change your working directory to any sub-module
>>>>> (like distributed shell) and run a successful Maven build for just that
>>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>>> hadoop project.  Look for the section titled "Building components
>>>>> separately".  I hope this helps.
>>>>>
>>>>> Chris Nauroth
>>>>> Hortonworks
>>>>> http://hortonworks.com/
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>>> curtullerich@gmail.com> wrote:
>>>>>
>>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>>> way.
>>>>>>
>>>>>> Thanks,
>>>>>> Curtis
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
Hi Omkar,

That was a typo, good catch. The path to Client.java should be

C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-
applications-distributedshell
/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java

Running as you suggest does build the jar in target with an updated
timestamp. I just need to copy it into
C:\l\hadoop-dist\target\hadoop-3.0.0-SNAPSHOT\share\hadoop\yarn and
override the version packaged there, and then things run as expected.

Thanks for your help!

Curtis



On Fri, Jun 21, 2013 at 7:49 PM, Omkar Joshi <oj...@hortonworks.com> wrote:

>
> Hi Curtis...I see a mismatch in paths which you have mentioned..are they
> same and was a typo or a mistake?
>
> ...modify logging code in *C:/l/hadoop-yarn-applications-**
> distributedshell/src/main/java/org/apache/hadoop/yarn/
> applications/distributedshell/Client.java*
> *
> *
> and
>
> *
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> *>mvn clean install -DskipTests
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
> org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> *
> *
>
> looks like you also have another directory named "*
> hadoop-yarn-applications-**distributedshell" *copied inside c:\1\...
>
>
> to make it more simple...go to "*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\"
> directory and clean everything and then run the "mvn clean install" command
> from *"*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell"
> directory.*
> *
> *
> *
> *
> *Let me know if you can now see the jar file (with updated time stamp) or
> not in target directory.*
>
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
>
>
> On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> I've executed the commands as you've said, and the jar that is run is
>> unchanged. Here's exactly what I did.
>>
>> C:\l\> start-dfs
>> C:\l\> start-yarn
>> C:\l\> cd
>> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...output...
>> ...modify logging code in
>> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...identical output to before...
>>
>> Do you see anything wrong with what I've done?
>>
>> Thanks,
>> Curtis
>>
>>
>>
>>
>> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>>
>>> Hi Curtis,
>>>
>>> where are you picking your jar file from? once you run above command you
>>> will see the updated jar file in
>>>
>>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>> I hope you are not using below jar file
>>>
>>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>>
>>> What chris has said is right.
>>>
>>> once you have taken latest code, you should follow below steps
>>> 1) mvn clean install -DskipTests (clean to remove previously generated
>>> code)
>>> 2) now say you are updating distributed shell client code. then go to
>>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>>> and then run "mvn clean install" and use jar from target sub folder.
>>>
>>> Thanks,
>>> Omkar Joshi
>>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <
>>> curtullerich@gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> I really appreciate the response. What you described is what I
>>>> initially tried. The changes never seem to take effect though. Here's what
>>>> I've done (this is Windows):
>>>>
>>>> cd %hadoop_install_dir%
>>>> mvn clean package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> --- modify the code in distributed shell's Client.java ---
>>>>
>>>>
>>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>>> mvn clean
>>>> mvn package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> Then I run the jar, just as before. I've just been changing log
>>>> statements to see if the changes worked. They don't--the output doesn't
>>>> change after doing this rebuild. I've also tried clearing the files put on
>>>> HDFS in my user directory in case that was the issue. I've taken a more
>>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>>> procedures described there. Am I missing anything else? I've tried
>>>> restarting yarn and dfs, though I didn't think that would matter.
>>>>
>>>> Thanks,
>>>> Curtis
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>>> cnauroth@hortonworks.com> wrote:
>>>>
>>>>> Hi Curtis,
>>>>>
>>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>>> repository.  Then, you can change your working directory to any sub-module
>>>>> (like distributed shell) and run a successful Maven build for just that
>>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>>> hadoop project.  Look for the section titled "Building components
>>>>> separately".  I hope this helps.
>>>>>
>>>>> Chris Nauroth
>>>>> Hortonworks
>>>>> http://hortonworks.com/
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>>> curtullerich@gmail.com> wrote:
>>>>>
>>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>>> way.
>>>>>>
>>>>>> Thanks,
>>>>>> Curtis
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
Hi Omkar,

That was a typo, good catch. The path to Client.java should be

C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-
applications-distributedshell
/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java

Running as you suggest does build the jar in target with an updated
timestamp. I just need to copy it into
C:\l\hadoop-dist\target\hadoop-3.0.0-SNAPSHOT\share\hadoop\yarn and
override the version packaged there, and then things run as expected.

Thanks for your help!

Curtis



On Fri, Jun 21, 2013 at 7:49 PM, Omkar Joshi <oj...@hortonworks.com> wrote:

>
> Hi Curtis...I see a mismatch in paths which you have mentioned..are they
> same and was a typo or a mistake?
>
> ...modify logging code in *C:/l/hadoop-yarn-applications-**
> distributedshell/src/main/java/org/apache/hadoop/yarn/
> applications/distributedshell/Client.java*
> *
> *
> and
>
> *
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> *>mvn clean install -DskipTests
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
> org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> *
> *
>
> looks like you also have another directory named "*
> hadoop-yarn-applications-**distributedshell" *copied inside c:\1\...
>
>
> to make it more simple...go to "*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\"
> directory and clean everything and then run the "mvn clean install" command
> from *"*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell"
> directory.*
> *
> *
> *
> *
> *Let me know if you can now see the jar file (with updated time stamp) or
> not in target directory.*
>
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
>
>
> On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> I've executed the commands as you've said, and the jar that is run is
>> unchanged. Here's exactly what I did.
>>
>> C:\l\> start-dfs
>> C:\l\> start-yarn
>> C:\l\> cd
>> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...output...
>> ...modify logging code in
>> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...identical output to before...
>>
>> Do you see anything wrong with what I've done?
>>
>> Thanks,
>> Curtis
>>
>>
>>
>>
>> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>>
>>> Hi Curtis,
>>>
>>> where are you picking your jar file from? once you run above command you
>>> will see the updated jar file in
>>>
>>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>> I hope you are not using below jar file
>>>
>>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>>
>>> What chris has said is right.
>>>
>>> once you have taken latest code, you should follow below steps
>>> 1) mvn clean install -DskipTests (clean to remove previously generated
>>> code)
>>> 2) now say you are updating distributed shell client code. then go to
>>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>>> and then run "mvn clean install" and use jar from target sub folder.
>>>
>>> Thanks,
>>> Omkar Joshi
>>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <
>>> curtullerich@gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> I really appreciate the response. What you described is what I
>>>> initially tried. The changes never seem to take effect though. Here's what
>>>> I've done (this is Windows):
>>>>
>>>> cd %hadoop_install_dir%
>>>> mvn clean package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> --- modify the code in distributed shell's Client.java ---
>>>>
>>>>
>>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>>> mvn clean
>>>> mvn package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> Then I run the jar, just as before. I've just been changing log
>>>> statements to see if the changes worked. They don't--the output doesn't
>>>> change after doing this rebuild. I've also tried clearing the files put on
>>>> HDFS in my user directory in case that was the issue. I've taken a more
>>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>>> procedures described there. Am I missing anything else? I've tried
>>>> restarting yarn and dfs, though I didn't think that would matter.
>>>>
>>>> Thanks,
>>>> Curtis
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>>> cnauroth@hortonworks.com> wrote:
>>>>
>>>>> Hi Curtis,
>>>>>
>>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>>> repository.  Then, you can change your working directory to any sub-module
>>>>> (like distributed shell) and run a successful Maven build for just that
>>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>>> hadoop project.  Look for the section titled "Building components
>>>>> separately".  I hope this helps.
>>>>>
>>>>> Chris Nauroth
>>>>> Hortonworks
>>>>> http://hortonworks.com/
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>>> curtullerich@gmail.com> wrote:
>>>>>
>>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>>> way.
>>>>>>
>>>>>> Thanks,
>>>>>> Curtis
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
Hi Omkar,

That was a typo, good catch. The path to Client.java should be

C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-
applications-distributedshell
/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java

Running as you suggest does build the jar in target with an updated
timestamp. I just need to copy it into
C:\l\hadoop-dist\target\hadoop-3.0.0-SNAPSHOT\share\hadoop\yarn and
override the version packaged there, and then things run as expected.

Thanks for your help!

Curtis



On Fri, Jun 21, 2013 at 7:49 PM, Omkar Joshi <oj...@hortonworks.com> wrote:

>
> Hi Curtis...I see a mismatch in paths which you have mentioned..are they
> same and was a typo or a mistake?
>
> ...modify logging code in *C:/l/hadoop-yarn-applications-**
> distributedshell/src/main/java/org/apache/hadoop/yarn/
> applications/distributedshell/Client.java*
> *
> *
> and
>
> *
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> *>mvn clean install -DskipTests
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
> org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> *
> *
>
> looks like you also have another directory named "*
> hadoop-yarn-applications-**distributedshell" *copied inside c:\1\...
>
>
> to make it more simple...go to "*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\"
> directory and clean everything and then run the "mvn clean install" command
> from *"*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell"
> directory.*
> *
> *
> *
> *
> *Let me know if you can now see the jar file (with updated time stamp) or
> not in target directory.*
>
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
>
>
> On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> I've executed the commands as you've said, and the jar that is run is
>> unchanged. Here's exactly what I did.
>>
>> C:\l\> start-dfs
>> C:\l\> start-yarn
>> C:\l\> cd
>> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...output...
>> ...modify logging code in
>> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...identical output to before...
>>
>> Do you see anything wrong with what I've done?
>>
>> Thanks,
>> Curtis
>>
>>
>>
>>
>> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>>
>>> Hi Curtis,
>>>
>>> where are you picking your jar file from? once you run above command you
>>> will see the updated jar file in
>>>
>>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>> I hope you are not using below jar file
>>>
>>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>>
>>> What chris has said is right.
>>>
>>> once you have taken latest code, you should follow below steps
>>> 1) mvn clean install -DskipTests (clean to remove previously generated
>>> code)
>>> 2) now say you are updating distributed shell client code. then go to
>>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>>> and then run "mvn clean install" and use jar from target sub folder.
>>>
>>> Thanks,
>>> Omkar Joshi
>>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <
>>> curtullerich@gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> I really appreciate the response. What you described is what I
>>>> initially tried. The changes never seem to take effect though. Here's what
>>>> I've done (this is Windows):
>>>>
>>>> cd %hadoop_install_dir%
>>>> mvn clean package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> --- modify the code in distributed shell's Client.java ---
>>>>
>>>>
>>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>>> mvn clean
>>>> mvn package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> Then I run the jar, just as before. I've just been changing log
>>>> statements to see if the changes worked. They don't--the output doesn't
>>>> change after doing this rebuild. I've also tried clearing the files put on
>>>> HDFS in my user directory in case that was the issue. I've taken a more
>>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>>> procedures described there. Am I missing anything else? I've tried
>>>> restarting yarn and dfs, though I didn't think that would matter.
>>>>
>>>> Thanks,
>>>> Curtis
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>>> cnauroth@hortonworks.com> wrote:
>>>>
>>>>> Hi Curtis,
>>>>>
>>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>>> repository.  Then, you can change your working directory to any sub-module
>>>>> (like distributed shell) and run a successful Maven build for just that
>>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>>> hadoop project.  Look for the section titled "Building components
>>>>> separately".  I hope this helps.
>>>>>
>>>>> Chris Nauroth
>>>>> Hortonworks
>>>>> http://hortonworks.com/
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>>> curtullerich@gmail.com> wrote:
>>>>>
>>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>>> way.
>>>>>>
>>>>>> Thanks,
>>>>>> Curtis
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Omkar Joshi <oj...@hortonworks.com>.
Hi Curtis...I see a mismatch in paths which you have mentioned..are they
same and was a typo or a mistake?

...modify logging code in *C:/l/hadoop-yarn-applications-**
distributedshell/src/main/java/org/apache/hadoop/yarn/
applications/distributedshell/Client.java*
*
*
and

*
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
*>mvn clean install -DskipTests
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami
*
*

looks like you also have another directory named "*hadoop-yarn-applications-
**distributedshell" *copied inside c:\1\...


to make it more simple...go to
"*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\"
directory and clean everything and then run the "mvn clean install" command
from *"*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell"
directory.*
*
*
*
*
*Let me know if you can now see the jar file (with updated time stamp) or
not in target directory.*

Thanks,
Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>


On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:

> I've executed the commands as you've said, and the jar that is run is
> unchanged. Here's exactly what I did.
>
> C:\l\> start-dfs
> C:\l\> start-yarn
> C:\l\> cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...output...
> ...modify logging code in
> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...identical output to before...
>
> Do you see anything wrong with what I've done?
>
> Thanks,
> Curtis
>
>
>
>
> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>
>> Hi Curtis,
>>
>> where are you picking your jar file from? once you run above command you
>> will see the updated jar file in
>>
>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>> I hope you are not using below jar file
>>
>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>>
>> What chris has said is right.
>>
>> once you have taken latest code, you should follow below steps
>> 1) mvn clean install -DskipTests (clean to remove previously generated
>> code)
>> 2) now say you are updating distributed shell client code. then go to
>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>> and then run "mvn clean install" and use jar from target sub folder.
>>
>> Thanks,
>> Omkar Joshi
>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>
>>
>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <curtullerich@gmail.com
>> > wrote:
>>
>>> Hi Chris,
>>>
>>> I really appreciate the response. What you described is what I initially
>>> tried. The changes never seem to take effect though. Here's what I've done
>>> (this is Windows):
>>>
>>> cd %hadoop_install_dir%
>>> mvn clean package -DskipTests
>>> mvn install -DskipTests
>>>
>>> --- modify the code in distributed shell's Client.java ---
>>>
>>>
>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>> mvn clean
>>> mvn package -DskipTests
>>> mvn install -DskipTests
>>>
>>> Then I run the jar, just as before. I've just been changing log
>>> statements to see if the changes worked. They don't--the output doesn't
>>> change after doing this rebuild. I've also tried clearing the files put on
>>> HDFS in my user directory in case that was the issue. I've taken a more
>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>> procedures described there. Am I missing anything else? I've tried
>>> restarting yarn and dfs, though I didn't think that would matter.
>>>
>>> Thanks,
>>> Curtis
>>>
>>>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>> cnauroth@hortonworks.com> wrote:
>>>
>>>> Hi Curtis,
>>>>
>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>> repository.  Then, you can change your working directory to any sub-module
>>>> (like distributed shell) and run a successful Maven build for just that
>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>> hadoop project.  Look for the section titled "Building components
>>>> separately".  I hope this helps.
>>>>
>>>> Chris Nauroth
>>>> Hortonworks
>>>> http://hortonworks.com/
>>>>
>>>>
>>>>
>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>> curtullerich@gmail.com> wrote:
>>>>
>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>> way.
>>>>>
>>>>> Thanks,
>>>>> Curtis
>>>>>
>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Arpit Agarwal <aa...@hortonworks.com>.
Curtis, if you are running 'mvn clean' in a subproject you may be hitting
this bug:
https://issues.apache.org/jira/browse/HADOOP-9383

You might have better luck if you include install along with package in the
same invocation.

-Arpit

On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:

> I've executed the commands as you've said, and the jar that is run is
> unchanged. Here's exactly what I did.
>
> C:\l\> start-dfs
> C:\l\> start-yarn
> C:\l\> cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...output...
> ...modify logging code in
> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...identical output to before...
>
> Do you see anything wrong with what I've done?
>
> Thanks,
> Curtis
>
>
>
>
> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>
>> Hi Curtis,
>>
>> where are you picking your jar file from? once you run above command you
>> will see the updated jar file in
>>
>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>> I hope you are not using below jar file
>>
>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>>
>> What chris has said is right.
>>
>> once you have taken latest code, you should follow below steps
>> 1) mvn clean install -DskipTests (clean to remove previously generated
>> code)
>> 2) now say you are updating distributed shell client code. then go to
>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>> and then run "mvn clean install" and use jar from target sub folder.
>>
>> Thanks,
>> Omkar Joshi
>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>
>>
>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <curtullerich@gmail.com
>> > wrote:
>>
>>> Hi Chris,
>>>
>>> I really appreciate the response. What you described is what I initially
>>> tried. The changes never seem to take effect though. Here's what I've done
>>> (this is Windows):
>>>
>>> cd %hadoop_install_dir%
>>> mvn clean package -DskipTests
>>> mvn install -DskipTests
>>>
>>> --- modify the code in distributed shell's Client.java ---
>>>
>>>
>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>> mvn clean
>>> mvn package -DskipTests
>>> mvn install -DskipTests
>>>
>>> Then I run the jar, just as before. I've just been changing log
>>> statements to see if the changes worked. They don't--the output doesn't
>>> change after doing this rebuild. I've also tried clearing the files put on
>>> HDFS in my user directory in case that was the issue. I've taken a more
>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>> procedures described there. Am I missing anything else? I've tried
>>> restarting yarn and dfs, though I didn't think that would matter.
>>>
>>> Thanks,
>>> Curtis
>>>
>>>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>> cnauroth@hortonworks.com> wrote:
>>>
>>>> Hi Curtis,
>>>>
>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>> repository.  Then, you can change your working directory to any sub-module
>>>> (like distributed shell) and run a successful Maven build for just that
>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>> hadoop project.  Look for the section titled "Building components
>>>> separately".  I hope this helps.
>>>>
>>>> Chris Nauroth
>>>> Hortonworks
>>>> http://hortonworks.com/
>>>>
>>>>
>>>>
>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>> curtullerich@gmail.com> wrote:
>>>>
>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>> way.
>>>>>
>>>>> Thanks,
>>>>> Curtis
>>>>>
>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Omkar Joshi <oj...@hortonworks.com>.
Hi Curtis...I see a mismatch in paths which you have mentioned..are they
same and was a typo or a mistake?

...modify logging code in *C:/l/hadoop-yarn-applications-**
distributedshell/src/main/java/org/apache/hadoop/yarn/
applications/distributedshell/Client.java*
*
*
and

*
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
*>mvn clean install -DskipTests
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami
*
*

looks like you also have another directory named "*hadoop-yarn-applications-
**distributedshell" *copied inside c:\1\...


to make it more simple...go to
"*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\"
directory and clean everything and then run the "mvn clean install" command
from *"*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell"
directory.*
*
*
*
*
*Let me know if you can now see the jar file (with updated time stamp) or
not in target directory.*

Thanks,
Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>


On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:

> I've executed the commands as you've said, and the jar that is run is
> unchanged. Here's exactly what I did.
>
> C:\l\> start-dfs
> C:\l\> start-yarn
> C:\l\> cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...output...
> ...modify logging code in
> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...identical output to before...
>
> Do you see anything wrong with what I've done?
>
> Thanks,
> Curtis
>
>
>
>
> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>
>> Hi Curtis,
>>
>> where are you picking your jar file from? once you run above command you
>> will see the updated jar file in
>>
>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>> I hope you are not using below jar file
>>
>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>>
>> What chris has said is right.
>>
>> once you have taken latest code, you should follow below steps
>> 1) mvn clean install -DskipTests (clean to remove previously generated
>> code)
>> 2) now say you are updating distributed shell client code. then go to
>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>> and then run "mvn clean install" and use jar from target sub folder.
>>
>> Thanks,
>> Omkar Joshi
>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>
>>
>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <curtullerich@gmail.com
>> > wrote:
>>
>>> Hi Chris,
>>>
>>> I really appreciate the response. What you described is what I initially
>>> tried. The changes never seem to take effect though. Here's what I've done
>>> (this is Windows):
>>>
>>> cd %hadoop_install_dir%
>>> mvn clean package -DskipTests
>>> mvn install -DskipTests
>>>
>>> --- modify the code in distributed shell's Client.java ---
>>>
>>>
>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>> mvn clean
>>> mvn package -DskipTests
>>> mvn install -DskipTests
>>>
>>> Then I run the jar, just as before. I've just been changing log
>>> statements to see if the changes worked. They don't--the output doesn't
>>> change after doing this rebuild. I've also tried clearing the files put on
>>> HDFS in my user directory in case that was the issue. I've taken a more
>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>> procedures described there. Am I missing anything else? I've tried
>>> restarting yarn and dfs, though I didn't think that would matter.
>>>
>>> Thanks,
>>> Curtis
>>>
>>>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>> cnauroth@hortonworks.com> wrote:
>>>
>>>> Hi Curtis,
>>>>
>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>> repository.  Then, you can change your working directory to any sub-module
>>>> (like distributed shell) and run a successful Maven build for just that
>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>> hadoop project.  Look for the section titled "Building components
>>>> separately".  I hope this helps.
>>>>
>>>> Chris Nauroth
>>>> Hortonworks
>>>> http://hortonworks.com/
>>>>
>>>>
>>>>
>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>> curtullerich@gmail.com> wrote:
>>>>
>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>> way.
>>>>>
>>>>> Thanks,
>>>>> Curtis
>>>>>
>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Omkar Joshi <oj...@hortonworks.com>.
Hi Curtis...I see a mismatch in paths which you have mentioned..are they
same and was a typo or a mistake?

...modify logging code in *C:/l/hadoop-yarn-applications-**
distributedshell/src/main/java/org/apache/hadoop/yarn/
applications/distributedshell/Client.java*
*
*
and

*
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
*>mvn clean install -DskipTests
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami
*
*

looks like you also have another directory named "*hadoop-yarn-applications-
**distributedshell" *copied inside c:\1\...


to make it more simple...go to
"*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\"
directory and clean everything and then run the "mvn clean install" command
from *"*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell"
directory.*
*
*
*
*
*Let me know if you can now see the jar file (with updated time stamp) or
not in target directory.*

Thanks,
Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>


On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:

> I've executed the commands as you've said, and the jar that is run is
> unchanged. Here's exactly what I did.
>
> C:\l\> start-dfs
> C:\l\> start-yarn
> C:\l\> cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...output...
> ...modify logging code in
> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...identical output to before...
>
> Do you see anything wrong with what I've done?
>
> Thanks,
> Curtis
>
>
>
>
> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>
>> Hi Curtis,
>>
>> where are you picking your jar file from? once you run above command you
>> will see the updated jar file in
>>
>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>> I hope you are not using below jar file
>>
>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>>
>> What chris has said is right.
>>
>> once you have taken latest code, you should follow below steps
>> 1) mvn clean install -DskipTests (clean to remove previously generated
>> code)
>> 2) now say you are updating distributed shell client code. then go to
>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>> and then run "mvn clean install" and use jar from target sub folder.
>>
>> Thanks,
>> Omkar Joshi
>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>
>>
>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <curtullerich@gmail.com
>> > wrote:
>>
>>> Hi Chris,
>>>
>>> I really appreciate the response. What you described is what I initially
>>> tried. The changes never seem to take effect though. Here's what I've done
>>> (this is Windows):
>>>
>>> cd %hadoop_install_dir%
>>> mvn clean package -DskipTests
>>> mvn install -DskipTests
>>>
>>> --- modify the code in distributed shell's Client.java ---
>>>
>>>
>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>> mvn clean
>>> mvn package -DskipTests
>>> mvn install -DskipTests
>>>
>>> Then I run the jar, just as before. I've just been changing log
>>> statements to see if the changes worked. They don't--the output doesn't
>>> change after doing this rebuild. I've also tried clearing the files put on
>>> HDFS in my user directory in case that was the issue. I've taken a more
>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>> procedures described there. Am I missing anything else? I've tried
>>> restarting yarn and dfs, though I didn't think that would matter.
>>>
>>> Thanks,
>>> Curtis
>>>
>>>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>> cnauroth@hortonworks.com> wrote:
>>>
>>>> Hi Curtis,
>>>>
>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>> repository.  Then, you can change your working directory to any sub-module
>>>> (like distributed shell) and run a successful Maven build for just that
>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>> hadoop project.  Look for the section titled "Building components
>>>> separately".  I hope this helps.
>>>>
>>>> Chris Nauroth
>>>> Hortonworks
>>>> http://hortonworks.com/
>>>>
>>>>
>>>>
>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>> curtullerich@gmail.com> wrote:
>>>>
>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>> way.
>>>>>
>>>>> Thanks,
>>>>> Curtis
>>>>>
>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Omkar Joshi <oj...@hortonworks.com>.
Hi Curtis...I see a mismatch in paths which you have mentioned..are they
same and was a typo or a mistake?

...modify logging code in *C:/l/hadoop-yarn-applications-**
distributedshell/src/main/java/org/apache/hadoop/yarn/
applications/distributedshell/Client.java*
*
*
and

*
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
*>mvn clean install -DskipTests
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami
*
*

looks like you also have another directory named "*hadoop-yarn-applications-
**distributedshell" *copied inside c:\1\...


to make it more simple...go to
"*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\"
directory and clean everything and then run the "mvn clean install" command
from *"*C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell"
directory.*
*
*
*
*
*Let me know if you can now see the jar file (with updated time stamp) or
not in target directory.*

Thanks,
Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>


On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:

> I've executed the commands as you've said, and the jar that is run is
> unchanged. Here's exactly what I did.
>
> C:\l\> start-dfs
> C:\l\> start-yarn
> C:\l\> cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...output...
> ...modify logging code in
> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...identical output to before...
>
> Do you see anything wrong with what I've done?
>
> Thanks,
> Curtis
>
>
>
>
> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>
>> Hi Curtis,
>>
>> where are you picking your jar file from? once you run above command you
>> will see the updated jar file in
>>
>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>> I hope you are not using below jar file
>>
>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>>
>> What chris has said is right.
>>
>> once you have taken latest code, you should follow below steps
>> 1) mvn clean install -DskipTests (clean to remove previously generated
>> code)
>> 2) now say you are updating distributed shell client code. then go to
>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>> and then run "mvn clean install" and use jar from target sub folder.
>>
>> Thanks,
>> Omkar Joshi
>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>
>>
>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <curtullerich@gmail.com
>> > wrote:
>>
>>> Hi Chris,
>>>
>>> I really appreciate the response. What you described is what I initially
>>> tried. The changes never seem to take effect though. Here's what I've done
>>> (this is Windows):
>>>
>>> cd %hadoop_install_dir%
>>> mvn clean package -DskipTests
>>> mvn install -DskipTests
>>>
>>> --- modify the code in distributed shell's Client.java ---
>>>
>>>
>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>> mvn clean
>>> mvn package -DskipTests
>>> mvn install -DskipTests
>>>
>>> Then I run the jar, just as before. I've just been changing log
>>> statements to see if the changes worked. They don't--the output doesn't
>>> change after doing this rebuild. I've also tried clearing the files put on
>>> HDFS in my user directory in case that was the issue. I've taken a more
>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>> procedures described there. Am I missing anything else? I've tried
>>> restarting yarn and dfs, though I didn't think that would matter.
>>>
>>> Thanks,
>>> Curtis
>>>
>>>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>> cnauroth@hortonworks.com> wrote:
>>>
>>>> Hi Curtis,
>>>>
>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>> repository.  Then, you can change your working directory to any sub-module
>>>> (like distributed shell) and run a successful Maven build for just that
>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>> hadoop project.  Look for the section titled "Building components
>>>> separately".  I hope this helps.
>>>>
>>>> Chris Nauroth
>>>> Hortonworks
>>>> http://hortonworks.com/
>>>>
>>>>
>>>>
>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>> curtullerich@gmail.com> wrote:
>>>>
>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>> way.
>>>>>
>>>>> Thanks,
>>>>> Curtis
>>>>>
>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Arpit Agarwal <aa...@hortonworks.com>.
Curtis, if you are running 'mvn clean' in a subproject you may be hitting
this bug:
https://issues.apache.org/jira/browse/HADOOP-9383

You might have better luck if you include install along with package in the
same invocation.

-Arpit

On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:

> I've executed the commands as you've said, and the jar that is run is
> unchanged. Here's exactly what I did.
>
> C:\l\> start-dfs
> C:\l\> start-yarn
> C:\l\> cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...output...
> ...modify logging code in
> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...identical output to before...
>
> Do you see anything wrong with what I've done?
>
> Thanks,
> Curtis
>
>
>
>
> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>
>> Hi Curtis,
>>
>> where are you picking your jar file from? once you run above command you
>> will see the updated jar file in
>>
>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>> I hope you are not using below jar file
>>
>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>>
>> What chris has said is right.
>>
>> once you have taken latest code, you should follow below steps
>> 1) mvn clean install -DskipTests (clean to remove previously generated
>> code)
>> 2) now say you are updating distributed shell client code. then go to
>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>> and then run "mvn clean install" and use jar from target sub folder.
>>
>> Thanks,
>> Omkar Joshi
>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>
>>
>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <curtullerich@gmail.com
>> > wrote:
>>
>>> Hi Chris,
>>>
>>> I really appreciate the response. What you described is what I initially
>>> tried. The changes never seem to take effect though. Here's what I've done
>>> (this is Windows):
>>>
>>> cd %hadoop_install_dir%
>>> mvn clean package -DskipTests
>>> mvn install -DskipTests
>>>
>>> --- modify the code in distributed shell's Client.java ---
>>>
>>>
>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>> mvn clean
>>> mvn package -DskipTests
>>> mvn install -DskipTests
>>>
>>> Then I run the jar, just as before. I've just been changing log
>>> statements to see if the changes worked. They don't--the output doesn't
>>> change after doing this rebuild. I've also tried clearing the files put on
>>> HDFS in my user directory in case that was the issue. I've taken a more
>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>> procedures described there. Am I missing anything else? I've tried
>>> restarting yarn and dfs, though I didn't think that would matter.
>>>
>>> Thanks,
>>> Curtis
>>>
>>>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>> cnauroth@hortonworks.com> wrote:
>>>
>>>> Hi Curtis,
>>>>
>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>> repository.  Then, you can change your working directory to any sub-module
>>>> (like distributed shell) and run a successful Maven build for just that
>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>> hadoop project.  Look for the section titled "Building components
>>>> separately".  I hope this helps.
>>>>
>>>> Chris Nauroth
>>>> Hortonworks
>>>> http://hortonworks.com/
>>>>
>>>>
>>>>
>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>> curtullerich@gmail.com> wrote:
>>>>
>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>> way.
>>>>>
>>>>> Thanks,
>>>>> Curtis
>>>>>
>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Arpit Agarwal <aa...@hortonworks.com>.
Curtis, if you are running 'mvn clean' in a subproject you may be hitting
this bug:
https://issues.apache.org/jira/browse/HADOOP-9383

You might have better luck if you include install along with package in the
same invocation.

-Arpit

On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <cu...@gmail.com>wrote:

> I've executed the commands as you've said, and the jar that is run is
> unchanged. Here's exactly what I did.
>
> C:\l\> start-dfs
> C:\l\> start-yarn
> C:\l\> cd
> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...output...
> ...modify logging code in
> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn clean install -DskipTests
>
> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command whoami
> ...identical output to before...
>
> Do you see anything wrong with what I've done?
>
> Thanks,
> Curtis
>
>
>
>
> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com>wrote:
>
>> Hi Curtis,
>>
>> where are you picking your jar file from? once you run above command you
>> will see the updated jar file in
>>
>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>> I hope you are not using below jar file
>>
>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>
>>
>> What chris has said is right.
>>
>> once you have taken latest code, you should follow below steps
>> 1) mvn clean install -DskipTests (clean to remove previously generated
>> code)
>> 2) now say you are updating distributed shell client code. then go to
>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>> and then run "mvn clean install" and use jar from target sub folder.
>>
>> Thanks,
>> Omkar Joshi
>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>
>>
>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <curtullerich@gmail.com
>> > wrote:
>>
>>> Hi Chris,
>>>
>>> I really appreciate the response. What you described is what I initially
>>> tried. The changes never seem to take effect though. Here's what I've done
>>> (this is Windows):
>>>
>>> cd %hadoop_install_dir%
>>> mvn clean package -DskipTests
>>> mvn install -DskipTests
>>>
>>> --- modify the code in distributed shell's Client.java ---
>>>
>>>
>>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>> mvn clean
>>> mvn package -DskipTests
>>> mvn install -DskipTests
>>>
>>> Then I run the jar, just as before. I've just been changing log
>>> statements to see if the changes worked. They don't--the output doesn't
>>> change after doing this rebuild. I've also tried clearing the files put on
>>> HDFS in my user directory in case that was the issue. I've taken a more
>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>> procedures described there. Am I missing anything else? I've tried
>>> restarting yarn and dfs, though I didn't think that would matter.
>>>
>>> Thanks,
>>> Curtis
>>>
>>>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>> cnauroth@hortonworks.com> wrote:
>>>
>>>> Hi Curtis,
>>>>
>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>> repository.  Then, you can change your working directory to any sub-module
>>>> (like distributed shell) and run a successful Maven build for just that
>>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>>> hadoop project.  Look for the section titled "Building components
>>>> separately".  I hope this helps.
>>>>
>>>> Chris Nauroth
>>>> Hortonworks
>>>> http://hortonworks.com/
>>>>
>>>>
>>>>
>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>> curtullerich@gmail.com> wrote:
>>>>
>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>>> way.
>>>>>
>>>>> Thanks,
>>>>> Curtis
>>>>>
>>>>
>>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
I've executed the commands as you've said, and the jar that is run is
unchanged. Here's exactly what I did.

C:\l\> start-dfs
C:\l\> start-yarn
C:\l\> cd
hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami
...output...
...modify logging code in
C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn
clean install -DskipTests
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami
...identical output to before...

Do you see anything wrong with what I've done?

Thanks,
Curtis




On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com> wrote:

> Hi Curtis,
>
> where are you picking your jar file from? once you run above command you
> will see the updated jar file in
>
> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>
> I hope you are not using below jar file
>
> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>
>
> What chris has said is right.
>
> once you have taken latest code, you should follow below steps
> 1) mvn clean install -DskipTests (clean to remove previously generated
> code)
> 2) now say you are updating distributed shell client code. then go to
> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
> and then run "mvn clean install" and use jar from target sub folder.
>
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
>
>
> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> Hi Chris,
>>
>> I really appreciate the response. What you described is what I initially
>> tried. The changes never seem to take effect though. Here's what I've done
>> (this is Windows):
>>
>> cd %hadoop_install_dir%
>> mvn clean package -DskipTests
>> mvn install -DskipTests
>>
>> --- modify the code in distributed shell's Client.java ---
>>
>>
>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>> mvn clean
>> mvn package -DskipTests
>> mvn install -DskipTests
>>
>> Then I run the jar, just as before. I've just been changing log
>> statements to see if the changes worked. They don't--the output doesn't
>> change after doing this rebuild. I've also tried clearing the files put on
>> HDFS in my user directory in case that was the issue. I've taken a more
>> thorough look at BUILDING.txt and I seemed to be consistent with the
>> procedures described there. Am I missing anything else? I've tried
>> restarting yarn and dfs, though I didn't think that would matter.
>>
>> Thanks,
>> Curtis
>>
>>
>>
>>
>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cnauroth@hortonworks.com
>> > wrote:
>>
>>> Hi Curtis,
>>>
>>> I handle this by running "mvn install -DskipTests" once from the root of
>>> the whole hadoop project to install the sub-modules in my local Maven
>>> repository.  Then, you can change your working directory to any sub-module
>>> (like distributed shell) and run a successful Maven build for just that
>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>> hadoop project.  Look for the section titled "Building components
>>> separately".  I hope this helps.
>>>
>>> Chris Nauroth
>>> Hortonworks
>>> http://hortonworks.com/
>>>
>>>
>>>
>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <curtullerich@gmail.com
>>> > wrote:
>>>
>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>> way.
>>>>
>>>> Thanks,
>>>> Curtis
>>>>
>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
I've executed the commands as you've said, and the jar that is run is
unchanged. Here's exactly what I did.

C:\l\> start-dfs
C:\l\> start-yarn
C:\l\> cd
hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami
...output...
...modify logging code in
C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn
clean install -DskipTests
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami
...identical output to before...

Do you see anything wrong with what I've done?

Thanks,
Curtis




On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com> wrote:

> Hi Curtis,
>
> where are you picking your jar file from? once you run above command you
> will see the updated jar file in
>
> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>
> I hope you are not using below jar file
>
> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>
>
> What chris has said is right.
>
> once you have taken latest code, you should follow below steps
> 1) mvn clean install -DskipTests (clean to remove previously generated
> code)
> 2) now say you are updating distributed shell client code. then go to
> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
> and then run "mvn clean install" and use jar from target sub folder.
>
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
>
>
> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> Hi Chris,
>>
>> I really appreciate the response. What you described is what I initially
>> tried. The changes never seem to take effect though. Here's what I've done
>> (this is Windows):
>>
>> cd %hadoop_install_dir%
>> mvn clean package -DskipTests
>> mvn install -DskipTests
>>
>> --- modify the code in distributed shell's Client.java ---
>>
>>
>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>> mvn clean
>> mvn package -DskipTests
>> mvn install -DskipTests
>>
>> Then I run the jar, just as before. I've just been changing log
>> statements to see if the changes worked. They don't--the output doesn't
>> change after doing this rebuild. I've also tried clearing the files put on
>> HDFS in my user directory in case that was the issue. I've taken a more
>> thorough look at BUILDING.txt and I seemed to be consistent with the
>> procedures described there. Am I missing anything else? I've tried
>> restarting yarn and dfs, though I didn't think that would matter.
>>
>> Thanks,
>> Curtis
>>
>>
>>
>>
>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cnauroth@hortonworks.com
>> > wrote:
>>
>>> Hi Curtis,
>>>
>>> I handle this by running "mvn install -DskipTests" once from the root of
>>> the whole hadoop project to install the sub-modules in my local Maven
>>> repository.  Then, you can change your working directory to any sub-module
>>> (like distributed shell) and run a successful Maven build for just that
>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>> hadoop project.  Look for the section titled "Building components
>>> separately".  I hope this helps.
>>>
>>> Chris Nauroth
>>> Hortonworks
>>> http://hortonworks.com/
>>>
>>>
>>>
>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <curtullerich@gmail.com
>>> > wrote:
>>>
>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>> way.
>>>>
>>>> Thanks,
>>>> Curtis
>>>>
>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
I've executed the commands as you've said, and the jar that is run is
unchanged. Here's exactly what I did.

C:\l\> start-dfs
C:\l\> start-yarn
C:\l\> cd
hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami
...output...
...modify logging code in
C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn
clean install -DskipTests
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami
...identical output to before...

Do you see anything wrong with what I've done?

Thanks,
Curtis




On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com> wrote:

> Hi Curtis,
>
> where are you picking your jar file from? once you run above command you
> will see the updated jar file in
>
> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>
> I hope you are not using below jar file
>
> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>
>
> What chris has said is right.
>
> once you have taken latest code, you should follow below steps
> 1) mvn clean install -DskipTests (clean to remove previously generated
> code)
> 2) now say you are updating distributed shell client code. then go to
> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
> and then run "mvn clean install" and use jar from target sub folder.
>
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
>
>
> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> Hi Chris,
>>
>> I really appreciate the response. What you described is what I initially
>> tried. The changes never seem to take effect though. Here's what I've done
>> (this is Windows):
>>
>> cd %hadoop_install_dir%
>> mvn clean package -DskipTests
>> mvn install -DskipTests
>>
>> --- modify the code in distributed shell's Client.java ---
>>
>>
>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>> mvn clean
>> mvn package -DskipTests
>> mvn install -DskipTests
>>
>> Then I run the jar, just as before. I've just been changing log
>> statements to see if the changes worked. They don't--the output doesn't
>> change after doing this rebuild. I've also tried clearing the files put on
>> HDFS in my user directory in case that was the issue. I've taken a more
>> thorough look at BUILDING.txt and I seemed to be consistent with the
>> procedures described there. Am I missing anything else? I've tried
>> restarting yarn and dfs, though I didn't think that would matter.
>>
>> Thanks,
>> Curtis
>>
>>
>>
>>
>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cnauroth@hortonworks.com
>> > wrote:
>>
>>> Hi Curtis,
>>>
>>> I handle this by running "mvn install -DskipTests" once from the root of
>>> the whole hadoop project to install the sub-modules in my local Maven
>>> repository.  Then, you can change your working directory to any sub-module
>>> (like distributed shell) and run a successful Maven build for just that
>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>> hadoop project.  Look for the section titled "Building components
>>> separately".  I hope this helps.
>>>
>>> Chris Nauroth
>>> Hortonworks
>>> http://hortonworks.com/
>>>
>>>
>>>
>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <curtullerich@gmail.com
>>> > wrote:
>>>
>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>> way.
>>>>
>>>> Thanks,
>>>> Curtis
>>>>
>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
I've executed the commands as you've said, and the jar that is run is
unchanged. Here's exactly what I did.

C:\l\> start-dfs
C:\l\> start-yarn
C:\l\> cd
hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami
...output...
...modify logging code in
C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn
clean install -DskipTests
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
org.apache.hadoop.yarn.applications.distributedshell.Client -jar
target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command whoami
...identical output to before...

Do you see anything wrong with what I've done?

Thanks,
Curtis




On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <oj...@hortonworks.com> wrote:

> Hi Curtis,
>
> where are you picking your jar file from? once you run above command you
> will see the updated jar file in
>
> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>
> I hope you are not using below jar file
>
> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>
>
> What chris has said is right.
>
> once you have taken latest code, you should follow below steps
> 1) mvn clean install -DskipTests (clean to remove previously generated
> code)
> 2) now say you are updating distributed shell client code. then go to
> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
> and then run "mvn clean install" and use jar from target sub folder.
>
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
>
>
> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> Hi Chris,
>>
>> I really appreciate the response. What you described is what I initially
>> tried. The changes never seem to take effect though. Here's what I've done
>> (this is Windows):
>>
>> cd %hadoop_install_dir%
>> mvn clean package -DskipTests
>> mvn install -DskipTests
>>
>> --- modify the code in distributed shell's Client.java ---
>>
>>
>> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>> mvn clean
>> mvn package -DskipTests
>> mvn install -DskipTests
>>
>> Then I run the jar, just as before. I've just been changing log
>> statements to see if the changes worked. They don't--the output doesn't
>> change after doing this rebuild. I've also tried clearing the files put on
>> HDFS in my user directory in case that was the issue. I've taken a more
>> thorough look at BUILDING.txt and I seemed to be consistent with the
>> procedures described there. Am I missing anything else? I've tried
>> restarting yarn and dfs, though I didn't think that would matter.
>>
>> Thanks,
>> Curtis
>>
>>
>>
>>
>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cnauroth@hortonworks.com
>> > wrote:
>>
>>> Hi Curtis,
>>>
>>> I handle this by running "mvn install -DskipTests" once from the root of
>>> the whole hadoop project to install the sub-modules in my local Maven
>>> repository.  Then, you can change your working directory to any sub-module
>>> (like distributed shell) and run a successful Maven build for just that
>>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>>> hadoop project.  Look for the section titled "Building components
>>> separately".  I hope this helps.
>>>
>>> Chris Nauroth
>>> Hortonworks
>>> http://hortonworks.com/
>>>
>>>
>>>
>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <curtullerich@gmail.com
>>> > wrote:
>>>
>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>>> way.
>>>>
>>>> Thanks,
>>>> Curtis
>>>>
>>>
>>>
>>
>

Re: Compile Just a Subproject

Posted by Omkar Joshi <oj...@hortonworks.com>.
Hi Curtis,

where are you picking your jar file from? once you run above command you
will see the updated jar file in
"..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"

I hope you are not using below jar file
"..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"


What chris has said is right.

once you have taken latest code, you should follow below steps
1) mvn clean install -DskipTests (clean to remove previously generated code)
2) now say you are updating distributed shell client code. then go to
""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
and then run "mvn clean install" and use jar from target sub folder.

Thanks,
Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>


On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <cu...@gmail.com>wrote:

> Hi Chris,
>
> I really appreciate the response. What you described is what I initially
> tried. The changes never seem to take effect though. Here's what I've done
> (this is Windows):
>
> cd %hadoop_install_dir%
> mvn clean package -DskipTests
> mvn install -DskipTests
>
> --- modify the code in distributed shell's Client.java ---
>
>
> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> mvn clean
> mvn package -DskipTests
> mvn install -DskipTests
>
> Then I run the jar, just as before. I've just been changing log statements
> to see if the changes worked. They don't--the output doesn't change after
> doing this rebuild. I've also tried clearing the files put on HDFS in my
> user directory in case that was the issue. I've taken a more thorough look
> at BUILDING.txt and I seemed to be consistent with the procedures described
> there. Am I missing anything else? I've tried restarting yarn and dfs,
> though I didn't think that would matter.
>
> Thanks,
> Curtis
>
>
>
>
> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cn...@hortonworks.com>wrote:
>
>> Hi Curtis,
>>
>> I handle this by running "mvn install -DskipTests" once from the root of
>> the whole hadoop project to install the sub-modules in my local Maven
>> repository.  Then, you can change your working directory to any sub-module
>> (like distributed shell) and run a successful Maven build for just that
>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>> hadoop project.  Look for the section titled "Building components
>> separately".  I hope this helps.
>>
>> Chris Nauroth
>> Hortonworks
>> http://hortonworks.com/
>>
>>
>>
>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>>
>>> What is the way to compile just a subproject of Hadoop with Maven
>>> (without rebuilding parent modules)? In my case I'm wondering about
>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>> way.
>>>
>>> Thanks,
>>> Curtis
>>>
>>
>>
>

Re: Compile Just a Subproject

Posted by Omkar Joshi <oj...@hortonworks.com>.
Hi Curtis,

where are you picking your jar file from? once you run above command you
will see the updated jar file in
"..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"

I hope you are not using below jar file
"..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"


What chris has said is right.

once you have taken latest code, you should follow below steps
1) mvn clean install -DskipTests (clean to remove previously generated code)
2) now say you are updating distributed shell client code. then go to
""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
and then run "mvn clean install" and use jar from target sub folder.

Thanks,
Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>


On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <cu...@gmail.com>wrote:

> Hi Chris,
>
> I really appreciate the response. What you described is what I initially
> tried. The changes never seem to take effect though. Here's what I've done
> (this is Windows):
>
> cd %hadoop_install_dir%
> mvn clean package -DskipTests
> mvn install -DskipTests
>
> --- modify the code in distributed shell's Client.java ---
>
>
> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> mvn clean
> mvn package -DskipTests
> mvn install -DskipTests
>
> Then I run the jar, just as before. I've just been changing log statements
> to see if the changes worked. They don't--the output doesn't change after
> doing this rebuild. I've also tried clearing the files put on HDFS in my
> user directory in case that was the issue. I've taken a more thorough look
> at BUILDING.txt and I seemed to be consistent with the procedures described
> there. Am I missing anything else? I've tried restarting yarn and dfs,
> though I didn't think that would matter.
>
> Thanks,
> Curtis
>
>
>
>
> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cn...@hortonworks.com>wrote:
>
>> Hi Curtis,
>>
>> I handle this by running "mvn install -DskipTests" once from the root of
>> the whole hadoop project to install the sub-modules in my local Maven
>> repository.  Then, you can change your working directory to any sub-module
>> (like distributed shell) and run a successful Maven build for just that
>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>> hadoop project.  Look for the section titled "Building components
>> separately".  I hope this helps.
>>
>> Chris Nauroth
>> Hortonworks
>> http://hortonworks.com/
>>
>>
>>
>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>>
>>> What is the way to compile just a subproject of Hadoop with Maven
>>> (without rebuilding parent modules)? In my case I'm wondering about
>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>> way.
>>>
>>> Thanks,
>>> Curtis
>>>
>>
>>
>

Re: Compile Just a Subproject

Posted by Omkar Joshi <oj...@hortonworks.com>.
Hi Curtis,

where are you picking your jar file from? once you run above command you
will see the updated jar file in
"..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"

I hope you are not using below jar file
"..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"


What chris has said is right.

once you have taken latest code, you should follow below steps
1) mvn clean install -DskipTests (clean to remove previously generated code)
2) now say you are updating distributed shell client code. then go to
""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
and then run "mvn clean install" and use jar from target sub folder.

Thanks,
Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>


On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <cu...@gmail.com>wrote:

> Hi Chris,
>
> I really appreciate the response. What you described is what I initially
> tried. The changes never seem to take effect though. Here's what I've done
> (this is Windows):
>
> cd %hadoop_install_dir%
> mvn clean package -DskipTests
> mvn install -DskipTests
>
> --- modify the code in distributed shell's Client.java ---
>
>
> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> mvn clean
> mvn package -DskipTests
> mvn install -DskipTests
>
> Then I run the jar, just as before. I've just been changing log statements
> to see if the changes worked. They don't--the output doesn't change after
> doing this rebuild. I've also tried clearing the files put on HDFS in my
> user directory in case that was the issue. I've taken a more thorough look
> at BUILDING.txt and I seemed to be consistent with the procedures described
> there. Am I missing anything else? I've tried restarting yarn and dfs,
> though I didn't think that would matter.
>
> Thanks,
> Curtis
>
>
>
>
> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cn...@hortonworks.com>wrote:
>
>> Hi Curtis,
>>
>> I handle this by running "mvn install -DskipTests" once from the root of
>> the whole hadoop project to install the sub-modules in my local Maven
>> repository.  Then, you can change your working directory to any sub-module
>> (like distributed shell) and run a successful Maven build for just that
>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>> hadoop project.  Look for the section titled "Building components
>> separately".  I hope this helps.
>>
>> Chris Nauroth
>> Hortonworks
>> http://hortonworks.com/
>>
>>
>>
>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>>
>>> What is the way to compile just a subproject of Hadoop with Maven
>>> (without rebuilding parent modules)? In my case I'm wondering about
>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>> way.
>>>
>>> Thanks,
>>> Curtis
>>>
>>
>>
>

Re: Compile Just a Subproject

Posted by Omkar Joshi <oj...@hortonworks.com>.
Hi Curtis,

where are you picking your jar file from? once you run above command you
will see the updated jar file in
"..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"

I hope you are not using below jar file
"..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"


What chris has said is right.

once you have taken latest code, you should follow below steps
1) mvn clean install -DskipTests (clean to remove previously generated code)
2) now say you are updating distributed shell client code. then go to
""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
and then run "mvn clean install" and use jar from target sub folder.

Thanks,
Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>


On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <cu...@gmail.com>wrote:

> Hi Chris,
>
> I really appreciate the response. What you described is what I initially
> tried. The changes never seem to take effect though. Here's what I've done
> (this is Windows):
>
> cd %hadoop_install_dir%
> mvn clean package -DskipTests
> mvn install -DskipTests
>
> --- modify the code in distributed shell's Client.java ---
>
>
> cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
> mvn clean
> mvn package -DskipTests
> mvn install -DskipTests
>
> Then I run the jar, just as before. I've just been changing log statements
> to see if the changes worked. They don't--the output doesn't change after
> doing this rebuild. I've also tried clearing the files put on HDFS in my
> user directory in case that was the issue. I've taken a more thorough look
> at BUILDING.txt and I seemed to be consistent with the procedures described
> there. Am I missing anything else? I've tried restarting yarn and dfs,
> though I didn't think that would matter.
>
> Thanks,
> Curtis
>
>
>
>
> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cn...@hortonworks.com>wrote:
>
>> Hi Curtis,
>>
>> I handle this by running "mvn install -DskipTests" once from the root of
>> the whole hadoop project to install the sub-modules in my local Maven
>> repository.  Then, you can change your working directory to any sub-module
>> (like distributed shell) and run a successful Maven build for just that
>> sub-module.  Full details are in the BUILDING.txt file in the root of the
>> hadoop project.  Look for the section titled "Building components
>> separately".  I hope this helps.
>>
>> Chris Nauroth
>> Hortonworks
>> http://hortonworks.com/
>>
>>
>>
>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>>
>>> What is the way to compile just a subproject of Hadoop with Maven
>>> (without rebuilding parent modules)? In my case I'm wondering about
>>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>>> way.
>>>
>>> Thanks,
>>> Curtis
>>>
>>
>>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
Hi Chris,

I really appreciate the response. What you described is what I initially
tried. The changes never seem to take effect though. Here's what I've done
(this is Windows):

cd %hadoop_install_dir%
mvn clean package -DskipTests
mvn install -DskipTests

--- modify the code in distributed shell's Client.java ---

cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
mvn clean
mvn package -DskipTests
mvn install -DskipTests

Then I run the jar, just as before. I've just been changing log statements
to see if the changes worked. They don't--the output doesn't change after
doing this rebuild. I've also tried clearing the files put on HDFS in my
user directory in case that was the issue. I've taken a more thorough look
at BUILDING.txt and I seemed to be consistent with the procedures described
there. Am I missing anything else? I've tried restarting yarn and dfs,
though I didn't think that would matter.

Thanks,
Curtis




On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cn...@hortonworks.com>wrote:

> Hi Curtis,
>
> I handle this by running "mvn install -DskipTests" once from the root of
> the whole hadoop project to install the sub-modules in my local Maven
> repository.  Then, you can change your working directory to any sub-module
> (like distributed shell) and run a successful Maven build for just that
> sub-module.  Full details are in the BUILDING.txt file in the root of the
> hadoop project.  Look for the section titled "Building components
> separately".  I hope this helps.
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> What is the way to compile just a subproject of Hadoop with Maven
>> (without rebuilding parent modules)? In my case I'm wondering about
>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>> way.
>>
>> Thanks,
>> Curtis
>>
>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
Hi Chris,

I really appreciate the response. What you described is what I initially
tried. The changes never seem to take effect though. Here's what I've done
(this is Windows):

cd %hadoop_install_dir%
mvn clean package -DskipTests
mvn install -DskipTests

--- modify the code in distributed shell's Client.java ---

cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
mvn clean
mvn package -DskipTests
mvn install -DskipTests

Then I run the jar, just as before. I've just been changing log statements
to see if the changes worked. They don't--the output doesn't change after
doing this rebuild. I've also tried clearing the files put on HDFS in my
user directory in case that was the issue. I've taken a more thorough look
at BUILDING.txt and I seemed to be consistent with the procedures described
there. Am I missing anything else? I've tried restarting yarn and dfs,
though I didn't think that would matter.

Thanks,
Curtis




On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cn...@hortonworks.com>wrote:

> Hi Curtis,
>
> I handle this by running "mvn install -DskipTests" once from the root of
> the whole hadoop project to install the sub-modules in my local Maven
> repository.  Then, you can change your working directory to any sub-module
> (like distributed shell) and run a successful Maven build for just that
> sub-module.  Full details are in the BUILDING.txt file in the root of the
> hadoop project.  Look for the section titled "Building components
> separately".  I hope this helps.
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> What is the way to compile just a subproject of Hadoop with Maven
>> (without rebuilding parent modules)? In my case I'm wondering about
>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>> way.
>>
>> Thanks,
>> Curtis
>>
>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
Hi Chris,

I really appreciate the response. What you described is what I initially
tried. The changes never seem to take effect though. Here's what I've done
(this is Windows):

cd %hadoop_install_dir%
mvn clean package -DskipTests
mvn install -DskipTests

--- modify the code in distributed shell's Client.java ---

cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
mvn clean
mvn package -DskipTests
mvn install -DskipTests

Then I run the jar, just as before. I've just been changing log statements
to see if the changes worked. They don't--the output doesn't change after
doing this rebuild. I've also tried clearing the files put on HDFS in my
user directory in case that was the issue. I've taken a more thorough look
at BUILDING.txt and I seemed to be consistent with the procedures described
there. Am I missing anything else? I've tried restarting yarn and dfs,
though I didn't think that would matter.

Thanks,
Curtis




On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cn...@hortonworks.com>wrote:

> Hi Curtis,
>
> I handle this by running "mvn install -DskipTests" once from the root of
> the whole hadoop project to install the sub-modules in my local Maven
> repository.  Then, you can change your working directory to any sub-module
> (like distributed shell) and run a successful Maven build for just that
> sub-module.  Full details are in the BUILDING.txt file in the root of the
> hadoop project.  Look for the section titled "Building components
> separately".  I hope this helps.
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> What is the way to compile just a subproject of Hadoop with Maven
>> (without rebuilding parent modules)? In my case I'm wondering about
>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>> way.
>>
>> Thanks,
>> Curtis
>>
>
>

Re: Compile Just a Subproject

Posted by Curtis Ullerich <cu...@gmail.com>.
Hi Chris,

I really appreciate the response. What you described is what I initially
tried. The changes never seem to take effect though. Here's what I've done
(this is Windows):

cd %hadoop_install_dir%
mvn clean package -DskipTests
mvn install -DskipTests

--- modify the code in distributed shell's Client.java ---

cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
mvn clean
mvn package -DskipTests
mvn install -DskipTests

Then I run the jar, just as before. I've just been changing log statements
to see if the changes worked. They don't--the output doesn't change after
doing this rebuild. I've also tried clearing the files put on HDFS in my
user directory in case that was the issue. I've taken a more thorough look
at BUILDING.txt and I seemed to be consistent with the procedures described
there. Am I missing anything else? I've tried restarting yarn and dfs,
though I didn't think that would matter.

Thanks,
Curtis




On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <cn...@hortonworks.com>wrote:

> Hi Curtis,
>
> I handle this by running "mvn install -DskipTests" once from the root of
> the whole hadoop project to install the sub-modules in my local Maven
> repository.  Then, you can change your working directory to any sub-module
> (like distributed shell) and run a successful Maven build for just that
> sub-module.  Full details are in the BUILDING.txt file in the root of the
> hadoop project.  Look for the section titled "Building components
> separately".  I hope this helps.
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>wrote:
>
>> What is the way to compile just a subproject of Hadoop with Maven
>> (without rebuilding parent modules)? In my case I'm wondering about
>> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
>> way.
>>
>> Thanks,
>> Curtis
>>
>
>

Re: Compile Just a Subproject

Posted by Chris Nauroth <cn...@hortonworks.com>.
Hi Curtis,

I handle this by running "mvn install -DskipTests" once from the root of
the whole hadoop project to install the sub-modules in my local Maven
repository.  Then, you can change your working directory to any sub-module
(like distributed shell) and run a successful Maven build for just that
sub-module.  Full details are in the BUILDING.txt file in the root of the
hadoop project.  Look for the section titled "Building components
separately".  I hope this helps.

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>wrote:

> What is the way to compile just a subproject of Hadoop with Maven (without
> rebuilding parent modules)? In my case I'm wondering about
> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
> way.
>
> Thanks,
> Curtis
>

Re: Compile Just a Subproject

Posted by Chris Nauroth <cn...@hortonworks.com>.
Hi Curtis,

I handle this by running "mvn install -DskipTests" once from the root of
the whole hadoop project to install the sub-modules in my local Maven
repository.  Then, you can change your working directory to any sub-module
(like distributed shell) and run a successful Maven build for just that
sub-module.  Full details are in the BUILDING.txt file in the root of the
hadoop project.  Look for the section titled "Building components
separately".  I hope this helps.

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>wrote:

> What is the way to compile just a subproject of Hadoop with Maven (without
> rebuilding parent modules)? In my case I'm wondering about
> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
> way.
>
> Thanks,
> Curtis
>

Re: Compile Just a Subproject

Posted by Chris Nauroth <cn...@hortonworks.com>.
Hi Curtis,

I handle this by running "mvn install -DskipTests" once from the root of
the whole hadoop project to install the sub-modules in my local Maven
repository.  Then, you can change your working directory to any sub-module
(like distributed shell) and run a successful Maven build for just that
sub-module.  Full details are in the BUILDING.txt file in the root of the
hadoop project.  Look for the section titled "Building components
separately".  I hope this helps.

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>wrote:

> What is the way to compile just a subproject of Hadoop with Maven (without
> rebuilding parent modules)? In my case I'm wondering about
> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
> way.
>
> Thanks,
> Curtis
>

Re: Compile Just a Subproject

Posted by Chris Nauroth <cn...@hortonworks.com>.
Hi Curtis,

I handle this by running "mvn install -DskipTests" once from the root of
the whole hadoop project to install the sub-modules in my local Maven
repository.  Then, you can change your working directory to any sub-module
(like distributed shell) and run a successful Maven build for just that
sub-module.  Full details are in the BUILDING.txt file in the root of the
hadoop project.  Look for the section titled "Building components
separately".  I hope this helps.

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <cu...@gmail.com>wrote:

> What is the way to compile just a subproject of Hadoop with Maven (without
> rebuilding parent modules)? In my case I'm wondering about
> DistributedShell. I thought that this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would have worked but no changes seem to take effect after building this
> way.
>
> Thanks,
> Curtis
>