You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by Doron Cohen <cd...@gmail.com> on 2012/10/22 11:22:19 UTC
what is the recommended way to develop Giraph code with Eclipse or
other IDEs
Hi, I am new to Giraph.
Looking at GIRAPH-42 I used Eclipse (on Windows 7) to modify the code.
The process I went through was quite cumbersome, so I wonder if there is a
more efficient way...
Here is what I did.
Checked out giraph-trunk and ran this command:
mvn -Dhadoop=non_secure clean eclipse:eclipse
This created:
- the giraph/target dir with munged code suitable for unsecured env
- Eclipse definition files .project and .classpath for two sub-projects:
giraph and giraph-formats-contrib.
I concentrated in the giraph sub project which has the sent messages
counters code (and all of giraph core I assume).
It turns out that the Eclipse project has - as source folders - the target
source, result of munging (pre-processing).
(Never used munge before, just read about it now trying to figure out what
was going on):
- target/munged/test
- target/munged/main
The Eclipse project compiles with no errors.
Tests are running ok from command line, like this: mvn -Dhadoop=non_secure
test
(Giraph Formats build fails but ignoring this now)
The output summary is:
[INFO] Apache Giraph Parent .............................. SUCCESS
[0.008s]
[INFO] Apache Giraph ..................................... SUCCESS
[3:17.460s]
[INFO] Apache Giraph Formats ............................. FAILURE
[2.805s]
(many exception stack traces printed, but ignoring this for now).
Running the tests from Eclipse fails: 16 out of 120 tests are failing, most
or all of them with this exception:
java.lang.IllegalArgumentException: Can not create a Path from an empty
string
at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
at org.apache.hadoop.fs.Path.<init>(Path.java:90)
at
org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:657)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.giraph.graph.GiraphJob.run(GiraphJob.java:261)
at
org.apache.giraph.TestMutateGraph.testMutateGraph(TestMutateGraph.java:56)
This makes it hard to fix tests, or run them through a debugger.
Also, each execution of the mvn command line wipes out the the target dir
and the Eclipse project must be refreshed.
It will also undo any code changes that were done within Eclipse, as the
sources under target are rewritten.
Last, the source files under target are not in SVN, so it is not possible
to use svn stat/diff etc. to see what has changed or to create a patch.
To workaround this, I opened the source files (as resources) in Eclipse and
edited, then ran the mvn target and refreshed Eclipse.
This is quite inconvenient and might lead to errors.
This is also mentioned in https://issues.apache.org/jira/browse/GIRAPH-207,
which is also where I learned about the eclipse:eclipse target.
How do others develop Giraph code then?
Using Eclipse or other IDEs perhaps? (how?)
Plain VI or EMACS editor with command line mvn commands?
Thanks,
Doron
Re: what is the recommended way to develop Giraph code with Eclipse
or other IDEs
Posted by Eli Reisman <ap...@gmail.com>.
I don't think any of the IDE's respond well to the munging. I use IntelliJ
on Linux and it works well for development, but I run the maven and repo
commands from the shell. I would advise doing all build stuff from the
shell. If you're still having trouble you might check that your Maven is
Maven3, that the environment vars for Maven are set, and that you have a
local maven repo populated with all the dependencies once you try to build
Giraph for the first time.
Hope that helps,
Eli
On Sat, Oct 27, 2012 at 1:10 PM, Doron Cohen <cd...@gmail.com> wrote:
> Thanks, indeed this works better!
> Also NetBeans allows to activate a profile - I was not aware of this
> feature.
> Seems that NetBeans is more suitable than Eclipse for Maven driven
> projects.
> Still, there are 4 compile errors, around the munge if blocks.
> For example: "SaslNettyServer line 24: cannot find symbol: class:
> StandbyException."
> Can you get NetBeans to compile without compile errors?
> Thanks, Doron
>
> On Tue, Oct 23, 2012 at 11:33 AM, Amani Alonazi
> <am...@kaust.edu.sa>wrote:
>
> > Sorry for late response.
> >
> > Once you checkout the repository, you don't have to change anything. The
> > following is the source dirs:
> > - giraph/src/main/java
> >
> >
> > On Mon, Oct 22, 2012 at 4:22 PM, Doron Cohen <cd...@gmail.com> wrote:
> >
> > > Hi Amani,
> > > Thanks for your help!
> > >
> > > NetBeans seems a nice IDE.
> > > Does it solve the munging issue?
> > > I mean, which of these two setups do you have for its source dirs:
> > >
> > > (1) the ones under src:
> > > - giraph/src/main/java
> > > - giraph/src/test/java
> > >
> > > (2) the ones under target:
> > > - giraph/target/munged/main
> > > - giraph/target/munged/test
> > >
> > > Thanks,
> > > Doron
> > >
> > > On Mon, Oct 22, 2012 at 12:10 PM, Amani Alonazi
> > > <am...@kaust.edu.sa>wrote:
> > >
> > > > Hey,
> > > >
> > > > I used NetBeans IDE. It's easy to write code and compile. I used the
> > > > command line (Terminal) to run the code.
> > > >
> > > > I hope this help.
> > > > In case you'll work on NetBeans, let me know if you need any help.
> > > >
> > > > Best,
> > > >
> > > > On Mon, Oct 22, 2012 at 12:22 PM, Doron Cohen <cd...@gmail.com>
> > wrote:
> > > >
> > > > > Hi, I am new to Giraph.
> > > > >
> > > > > Looking at GIRAPH-42 I used Eclipse (on Windows 7) to modify the
> > code.
> > > > >
> > > > > The process I went through was quite cumbersome, so I wonder if
> there
> > > is
> > > > a
> > > > > more efficient way...
> > > > >
> > > > > Here is what I did.
> > > > > Checked out giraph-trunk and ran this command:
> > > > > mvn -Dhadoop=non_secure clean eclipse:eclipse
> > > > >
> > > > > This created:
> > > > > - the giraph/target dir with munged code suitable for unsecured env
> > > > > - Eclipse definition files .project and .classpath for two
> > > sub-projects:
> > > > > giraph and giraph-formats-contrib.
> > > > >
> > > > > I concentrated in the giraph sub project which has the sent
> messages
> > > > > counters code (and all of giraph core I assume).
> > > > >
> > > > > It turns out that the Eclipse project has - as source folders - the
> > > > target
> > > > > source, result of munging (pre-processing).
> > > > > (Never used munge before, just read about it now trying to figure
> out
> > > > what
> > > > > was going on):
> > > > > - target/munged/test
> > > > > - target/munged/main
> > > > >
> > > > > The Eclipse project compiles with no errors.
> > > > >
> > > > > Tests are running ok from command line, like this: mvn
> > > > -Dhadoop=non_secure
> > > > > test
> > > > > (Giraph Formats build fails but ignoring this now)
> > > > > The output summary is:
> > > > > [INFO] Apache Giraph Parent ..............................
> SUCCESS
> > > > > [0.008s]
> > > > > [INFO] Apache Giraph .....................................
> SUCCESS
> > > > > [3:17.460s]
> > > > > [INFO] Apache Giraph Formats .............................
> FAILURE
> > > > > [2.805s]
> > > > > (many exception stack traces printed, but ignoring this for now).
> > > > >
> > > > > Running the tests from Eclipse fails: 16 out of 120 tests are
> > failing,
> > > > most
> > > > > or all of them with this exception:
> > > > > java.lang.IllegalArgumentException: Can not create a Path from an
> > empty
> > > > > string
> > > > > at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
> > > > > at org.apache.hadoop.fs.Path.<init>(Path.java:90)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:657)
> > > > > at
> > > > >
> > >
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
> > > > > at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
> > > > > at
> > org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
> > > > > at org.apache.giraph.graph.GiraphJob.run(GiraphJob.java:261)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.giraph.TestMutateGraph.testMutateGraph(TestMutateGraph.java:56)
> > > > >
> > > > > This makes it hard to fix tests, or run them through a debugger.
> > > > > Also, each execution of the mvn command line wipes out the the
> target
> > > dir
> > > > > and the Eclipse project must be refreshed.
> > > > > It will also undo any code changes that were done within Eclipse,
> as
> > > the
> > > > > sources under target are rewritten.
> > > > >
> > > > > Last, the source files under target are not in SVN, so it is not
> > > possible
> > > > > to use svn stat/diff etc. to see what has changed or to create a
> > patch.
> > > > > To workaround this, I opened the source files (as resources) in
> > Eclipse
> > > > and
> > > > > edited, then ran the mvn target and refreshed Eclipse.
> > > > > This is quite inconvenient and might lead to errors.
> > > > >
> > > > > This is also mentioned in
> > > > https://issues.apache.org/jira/browse/GIRAPH-207
> > > > > ,
> > > > > which is also where I learned about the eclipse:eclipse target.
> > > > >
> > > > > How do others develop Giraph code then?
> > > > > Using Eclipse or other IDEs perhaps? (how?)
> > > > > Plain VI or EMACS editor with command line mvn commands?
> > > > >
> > > > > Thanks,
> > > > > Doron
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Amani AlOnazi
> > > > MSc Computer Science
> > > > King Abdullah University of Science and Technology
> > > > Kingdom of Saudi Arabia
> > > > amani.alonazi@kaust.edu.sa |
> > > >
> > > > --
> > > >
> > > > ------------------------------
> > > > This message and its contents, including attachments are intended
> > solely
> > > > for the original recipient. If you are not the intended recipient or
> > have
> > > > received this message in error, please notify me immediately and
> delete
> > > > this message from your computer system. Any unauthorized use or
> > > > distribution is prohibited. Please consider the environment before
> > > printing
> > > > this email.
> > > >
> > >
> >
> >
> >
> > --
> > Amani AlOnazi
> > MSc Computer Science
> > King Abdullah University of Science and Technology
> > Kingdom of Saudi Arabia
> > amani.alonazi@kaust.edu.sa | +966 (0) 555 191 795
> >
> > --
> >
> > ------------------------------
> > This message and its contents, including attachments are intended solely
> > for the original recipient. If you are not the intended recipient or have
> > received this message in error, please notify me immediately and delete
> > this message from your computer system. Any unauthorized use or
> > distribution is prohibited. Please consider the environment before
> printing
> > this email.
> >
>
Re: what is the recommended way to develop Giraph code with Eclipse
or other IDEs
Posted by Doron Cohen <cd...@gmail.com>.
Thanks, indeed this works better!
Also NetBeans allows to activate a profile - I was not aware of this
feature.
Seems that NetBeans is more suitable than Eclipse for Maven driven projects.
Still, there are 4 compile errors, around the munge if blocks.
For example: "SaslNettyServer line 24: cannot find symbol: class:
StandbyException."
Can you get NetBeans to compile without compile errors?
Thanks, Doron
On Tue, Oct 23, 2012 at 11:33 AM, Amani Alonazi
<am...@kaust.edu.sa>wrote:
> Sorry for late response.
>
> Once you checkout the repository, you don't have to change anything. The
> following is the source dirs:
> - giraph/src/main/java
>
>
> On Mon, Oct 22, 2012 at 4:22 PM, Doron Cohen <cd...@gmail.com> wrote:
>
> > Hi Amani,
> > Thanks for your help!
> >
> > NetBeans seems a nice IDE.
> > Does it solve the munging issue?
> > I mean, which of these two setups do you have for its source dirs:
> >
> > (1) the ones under src:
> > - giraph/src/main/java
> > - giraph/src/test/java
> >
> > (2) the ones under target:
> > - giraph/target/munged/main
> > - giraph/target/munged/test
> >
> > Thanks,
> > Doron
> >
> > On Mon, Oct 22, 2012 at 12:10 PM, Amani Alonazi
> > <am...@kaust.edu.sa>wrote:
> >
> > > Hey,
> > >
> > > I used NetBeans IDE. It's easy to write code and compile. I used the
> > > command line (Terminal) to run the code.
> > >
> > > I hope this help.
> > > In case you'll work on NetBeans, let me know if you need any help.
> > >
> > > Best,
> > >
> > > On Mon, Oct 22, 2012 at 12:22 PM, Doron Cohen <cd...@gmail.com>
> wrote:
> > >
> > > > Hi, I am new to Giraph.
> > > >
> > > > Looking at GIRAPH-42 I used Eclipse (on Windows 7) to modify the
> code.
> > > >
> > > > The process I went through was quite cumbersome, so I wonder if there
> > is
> > > a
> > > > more efficient way...
> > > >
> > > > Here is what I did.
> > > > Checked out giraph-trunk and ran this command:
> > > > mvn -Dhadoop=non_secure clean eclipse:eclipse
> > > >
> > > > This created:
> > > > - the giraph/target dir with munged code suitable for unsecured env
> > > > - Eclipse definition files .project and .classpath for two
> > sub-projects:
> > > > giraph and giraph-formats-contrib.
> > > >
> > > > I concentrated in the giraph sub project which has the sent messages
> > > > counters code (and all of giraph core I assume).
> > > >
> > > > It turns out that the Eclipse project has - as source folders - the
> > > target
> > > > source, result of munging (pre-processing).
> > > > (Never used munge before, just read about it now trying to figure out
> > > what
> > > > was going on):
> > > > - target/munged/test
> > > > - target/munged/main
> > > >
> > > > The Eclipse project compiles with no errors.
> > > >
> > > > Tests are running ok from command line, like this: mvn
> > > -Dhadoop=non_secure
> > > > test
> > > > (Giraph Formats build fails but ignoring this now)
> > > > The output summary is:
> > > > [INFO] Apache Giraph Parent .............................. SUCCESS
> > > > [0.008s]
> > > > [INFO] Apache Giraph ..................................... SUCCESS
> > > > [3:17.460s]
> > > > [INFO] Apache Giraph Formats ............................. FAILURE
> > > > [2.805s]
> > > > (many exception stack traces printed, but ignoring this for now).
> > > >
> > > > Running the tests from Eclipse fails: 16 out of 120 tests are
> failing,
> > > most
> > > > or all of them with this exception:
> > > > java.lang.IllegalArgumentException: Can not create a Path from an
> empty
> > > > string
> > > > at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
> > > > at org.apache.hadoop.fs.Path.<init>(Path.java:90)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:657)
> > > > at
> > > >
> > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
> > > > at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
> > > > at
> org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
> > > > at org.apache.giraph.graph.GiraphJob.run(GiraphJob.java:261)
> > > > at
> > > >
> > >
> >
> org.apache.giraph.TestMutateGraph.testMutateGraph(TestMutateGraph.java:56)
> > > >
> > > > This makes it hard to fix tests, or run them through a debugger.
> > > > Also, each execution of the mvn command line wipes out the the target
> > dir
> > > > and the Eclipse project must be refreshed.
> > > > It will also undo any code changes that were done within Eclipse, as
> > the
> > > > sources under target are rewritten.
> > > >
> > > > Last, the source files under target are not in SVN, so it is not
> > possible
> > > > to use svn stat/diff etc. to see what has changed or to create a
> patch.
> > > > To workaround this, I opened the source files (as resources) in
> Eclipse
> > > and
> > > > edited, then ran the mvn target and refreshed Eclipse.
> > > > This is quite inconvenient and might lead to errors.
> > > >
> > > > This is also mentioned in
> > > https://issues.apache.org/jira/browse/GIRAPH-207
> > > > ,
> > > > which is also where I learned about the eclipse:eclipse target.
> > > >
> > > > How do others develop Giraph code then?
> > > > Using Eclipse or other IDEs perhaps? (how?)
> > > > Plain VI or EMACS editor with command line mvn commands?
> > > >
> > > > Thanks,
> > > > Doron
> > > >
> > >
> > >
> > >
> > > --
> > > Amani AlOnazi
> > > MSc Computer Science
> > > King Abdullah University of Science and Technology
> > > Kingdom of Saudi Arabia
> > > amani.alonazi@kaust.edu.sa |
> > >
> > > --
> > >
> > > ------------------------------
> > > This message and its contents, including attachments are intended
> solely
> > > for the original recipient. If you are not the intended recipient or
> have
> > > received this message in error, please notify me immediately and delete
> > > this message from your computer system. Any unauthorized use or
> > > distribution is prohibited. Please consider the environment before
> > printing
> > > this email.
> > >
> >
>
>
>
> --
> Amani AlOnazi
> MSc Computer Science
> King Abdullah University of Science and Technology
> Kingdom of Saudi Arabia
> amani.alonazi@kaust.edu.sa | +966 (0) 555 191 795
>
> --
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
Re: what is the recommended way to develop Giraph code with Eclipse
or other IDEs
Posted by Amani Alonazi <am...@kaust.edu.sa>.
Sorry for late response.
Once you checkout the repository, you don't have to change anything. The
following is the source dirs:
- giraph/src/main/java
On Mon, Oct 22, 2012 at 4:22 PM, Doron Cohen <cd...@gmail.com> wrote:
> Hi Amani,
> Thanks for your help!
>
> NetBeans seems a nice IDE.
> Does it solve the munging issue?
> I mean, which of these two setups do you have for its source dirs:
>
> (1) the ones under src:
> - giraph/src/main/java
> - giraph/src/test/java
>
> (2) the ones under target:
> - giraph/target/munged/main
> - giraph/target/munged/test
>
> Thanks,
> Doron
>
> On Mon, Oct 22, 2012 at 12:10 PM, Amani Alonazi
> <am...@kaust.edu.sa>wrote:
>
> > Hey,
> >
> > I used NetBeans IDE. It's easy to write code and compile. I used the
> > command line (Terminal) to run the code.
> >
> > I hope this help.
> > In case you'll work on NetBeans, let me know if you need any help.
> >
> > Best,
> >
> > On Mon, Oct 22, 2012 at 12:22 PM, Doron Cohen <cd...@gmail.com> wrote:
> >
> > > Hi, I am new to Giraph.
> > >
> > > Looking at GIRAPH-42 I used Eclipse (on Windows 7) to modify the code.
> > >
> > > The process I went through was quite cumbersome, so I wonder if there
> is
> > a
> > > more efficient way...
> > >
> > > Here is what I did.
> > > Checked out giraph-trunk and ran this command:
> > > mvn -Dhadoop=non_secure clean eclipse:eclipse
> > >
> > > This created:
> > > - the giraph/target dir with munged code suitable for unsecured env
> > > - Eclipse definition files .project and .classpath for two
> sub-projects:
> > > giraph and giraph-formats-contrib.
> > >
> > > I concentrated in the giraph sub project which has the sent messages
> > > counters code (and all of giraph core I assume).
> > >
> > > It turns out that the Eclipse project has - as source folders - the
> > target
> > > source, result of munging (pre-processing).
> > > (Never used munge before, just read about it now trying to figure out
> > what
> > > was going on):
> > > - target/munged/test
> > > - target/munged/main
> > >
> > > The Eclipse project compiles with no errors.
> > >
> > > Tests are running ok from command line, like this: mvn
> > -Dhadoop=non_secure
> > > test
> > > (Giraph Formats build fails but ignoring this now)
> > > The output summary is:
> > > [INFO] Apache Giraph Parent .............................. SUCCESS
> > > [0.008s]
> > > [INFO] Apache Giraph ..................................... SUCCESS
> > > [3:17.460s]
> > > [INFO] Apache Giraph Formats ............................. FAILURE
> > > [2.805s]
> > > (many exception stack traces printed, but ignoring this for now).
> > >
> > > Running the tests from Eclipse fails: 16 out of 120 tests are failing,
> > most
> > > or all of them with this exception:
> > > java.lang.IllegalArgumentException: Can not create a Path from an empty
> > > string
> > > at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
> > > at org.apache.hadoop.fs.Path.<init>(Path.java:90)
> > > at
> > >
> > >
> >
> org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:657)
> > > at
> > >
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
> > > at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
> > > at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
> > > at org.apache.giraph.graph.GiraphJob.run(GiraphJob.java:261)
> > > at
> > >
> >
> org.apache.giraph.TestMutateGraph.testMutateGraph(TestMutateGraph.java:56)
> > >
> > > This makes it hard to fix tests, or run them through a debugger.
> > > Also, each execution of the mvn command line wipes out the the target
> dir
> > > and the Eclipse project must be refreshed.
> > > It will also undo any code changes that were done within Eclipse, as
> the
> > > sources under target are rewritten.
> > >
> > > Last, the source files under target are not in SVN, so it is not
> possible
> > > to use svn stat/diff etc. to see what has changed or to create a patch.
> > > To workaround this, I opened the source files (as resources) in Eclipse
> > and
> > > edited, then ran the mvn target and refreshed Eclipse.
> > > This is quite inconvenient and might lead to errors.
> > >
> > > This is also mentioned in
> > https://issues.apache.org/jira/browse/GIRAPH-207
> > > ,
> > > which is also where I learned about the eclipse:eclipse target.
> > >
> > > How do others develop Giraph code then?
> > > Using Eclipse or other IDEs perhaps? (how?)
> > > Plain VI or EMACS editor with command line mvn commands?
> > >
> > > Thanks,
> > > Doron
> > >
> >
> >
> >
> > --
> > Amani AlOnazi
> > MSc Computer Science
> > King Abdullah University of Science and Technology
> > Kingdom of Saudi Arabia
> > amani.alonazi@kaust.edu.sa |
> >
> > --
> >
> > ------------------------------
> > This message and its contents, including attachments are intended solely
> > for the original recipient. If you are not the intended recipient or have
> > received this message in error, please notify me immediately and delete
> > this message from your computer system. Any unauthorized use or
> > distribution is prohibited. Please consider the environment before
> printing
> > this email.
> >
>
--
Amani AlOnazi
MSc Computer Science
King Abdullah University of Science and Technology
Kingdom of Saudi Arabia
amani.alonazi@kaust.edu.sa | +966 (0) 555 191 795
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: what is the recommended way to develop Giraph code with Eclipse
or other IDEs
Posted by Doron Cohen <cd...@gmail.com>.
Hi Amani,
Thanks for your help!
NetBeans seems a nice IDE.
Does it solve the munging issue?
I mean, which of these two setups do you have for its source dirs:
(1) the ones under src:
- giraph/src/main/java
- giraph/src/test/java
(2) the ones under target:
- giraph/target/munged/main
- giraph/target/munged/test
Thanks,
Doron
On Mon, Oct 22, 2012 at 12:10 PM, Amani Alonazi
<am...@kaust.edu.sa>wrote:
> Hey,
>
> I used NetBeans IDE. It's easy to write code and compile. I used the
> command line (Terminal) to run the code.
>
> I hope this help.
> In case you'll work on NetBeans, let me know if you need any help.
>
> Best,
>
> On Mon, Oct 22, 2012 at 12:22 PM, Doron Cohen <cd...@gmail.com> wrote:
>
> > Hi, I am new to Giraph.
> >
> > Looking at GIRAPH-42 I used Eclipse (on Windows 7) to modify the code.
> >
> > The process I went through was quite cumbersome, so I wonder if there is
> a
> > more efficient way...
> >
> > Here is what I did.
> > Checked out giraph-trunk and ran this command:
> > mvn -Dhadoop=non_secure clean eclipse:eclipse
> >
> > This created:
> > - the giraph/target dir with munged code suitable for unsecured env
> > - Eclipse definition files .project and .classpath for two sub-projects:
> > giraph and giraph-formats-contrib.
> >
> > I concentrated in the giraph sub project which has the sent messages
> > counters code (and all of giraph core I assume).
> >
> > It turns out that the Eclipse project has - as source folders - the
> target
> > source, result of munging (pre-processing).
> > (Never used munge before, just read about it now trying to figure out
> what
> > was going on):
> > - target/munged/test
> > - target/munged/main
> >
> > The Eclipse project compiles with no errors.
> >
> > Tests are running ok from command line, like this: mvn
> -Dhadoop=non_secure
> > test
> > (Giraph Formats build fails but ignoring this now)
> > The output summary is:
> > [INFO] Apache Giraph Parent .............................. SUCCESS
> > [0.008s]
> > [INFO] Apache Giraph ..................................... SUCCESS
> > [3:17.460s]
> > [INFO] Apache Giraph Formats ............................. FAILURE
> > [2.805s]
> > (many exception stack traces printed, but ignoring this for now).
> >
> > Running the tests from Eclipse fails: 16 out of 120 tests are failing,
> most
> > or all of them with this exception:
> > java.lang.IllegalArgumentException: Can not create a Path from an empty
> > string
> > at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
> > at org.apache.hadoop.fs.Path.<init>(Path.java:90)
> > at
> >
> >
> org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:657)
> > at
> > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
> > at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
> > at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
> > at org.apache.giraph.graph.GiraphJob.run(GiraphJob.java:261)
> > at
> >
> org.apache.giraph.TestMutateGraph.testMutateGraph(TestMutateGraph.java:56)
> >
> > This makes it hard to fix tests, or run them through a debugger.
> > Also, each execution of the mvn command line wipes out the the target dir
> > and the Eclipse project must be refreshed.
> > It will also undo any code changes that were done within Eclipse, as the
> > sources under target are rewritten.
> >
> > Last, the source files under target are not in SVN, so it is not possible
> > to use svn stat/diff etc. to see what has changed or to create a patch.
> > To workaround this, I opened the source files (as resources) in Eclipse
> and
> > edited, then ran the mvn target and refreshed Eclipse.
> > This is quite inconvenient and might lead to errors.
> >
> > This is also mentioned in
> https://issues.apache.org/jira/browse/GIRAPH-207
> > ,
> > which is also where I learned about the eclipse:eclipse target.
> >
> > How do others develop Giraph code then?
> > Using Eclipse or other IDEs perhaps? (how?)
> > Plain VI or EMACS editor with command line mvn commands?
> >
> > Thanks,
> > Doron
> >
>
>
>
> --
> Amani AlOnazi
> MSc Computer Science
> King Abdullah University of Science and Technology
> Kingdom of Saudi Arabia
> amani.alonazi@kaust.edu.sa |
>
> --
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
Re: what is the recommended way to develop Giraph code with Eclipse
or other IDEs
Posted by Amani Alonazi <am...@kaust.edu.sa>.
Hey,
I used NetBeans IDE. It's easy to write code and compile. I used the
command line (Terminal) to run the code.
I hope this help.
In case you'll work on NetBeans, let me know if you need any help.
Best,
On Mon, Oct 22, 2012 at 12:22 PM, Doron Cohen <cd...@gmail.com> wrote:
> Hi, I am new to Giraph.
>
> Looking at GIRAPH-42 I used Eclipse (on Windows 7) to modify the code.
>
> The process I went through was quite cumbersome, so I wonder if there is a
> more efficient way...
>
> Here is what I did.
> Checked out giraph-trunk and ran this command:
> mvn -Dhadoop=non_secure clean eclipse:eclipse
>
> This created:
> - the giraph/target dir with munged code suitable for unsecured env
> - Eclipse definition files .project and .classpath for two sub-projects:
> giraph and giraph-formats-contrib.
>
> I concentrated in the giraph sub project which has the sent messages
> counters code (and all of giraph core I assume).
>
> It turns out that the Eclipse project has - as source folders - the target
> source, result of munging (pre-processing).
> (Never used munge before, just read about it now trying to figure out what
> was going on):
> - target/munged/test
> - target/munged/main
>
> The Eclipse project compiles with no errors.
>
> Tests are running ok from command line, like this: mvn -Dhadoop=non_secure
> test
> (Giraph Formats build fails but ignoring this now)
> The output summary is:
> [INFO] Apache Giraph Parent .............................. SUCCESS
> [0.008s]
> [INFO] Apache Giraph ..................................... SUCCESS
> [3:17.460s]
> [INFO] Apache Giraph Formats ............................. FAILURE
> [2.805s]
> (many exception stack traces printed, but ignoring this for now).
>
> Running the tests from Eclipse fails: 16 out of 120 tests are failing, most
> or all of them with this exception:
> java.lang.IllegalArgumentException: Can not create a Path from an empty
> string
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
> at org.apache.hadoop.fs.Path.<init>(Path.java:90)
> at
>
> org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:657)
> at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
> at org.apache.giraph.graph.GiraphJob.run(GiraphJob.java:261)
> at
> org.apache.giraph.TestMutateGraph.testMutateGraph(TestMutateGraph.java:56)
>
> This makes it hard to fix tests, or run them through a debugger.
> Also, each execution of the mvn command line wipes out the the target dir
> and the Eclipse project must be refreshed.
> It will also undo any code changes that were done within Eclipse, as the
> sources under target are rewritten.
>
> Last, the source files under target are not in SVN, so it is not possible
> to use svn stat/diff etc. to see what has changed or to create a patch.
> To workaround this, I opened the source files (as resources) in Eclipse and
> edited, then ran the mvn target and refreshed Eclipse.
> This is quite inconvenient and might lead to errors.
>
> This is also mentioned in https://issues.apache.org/jira/browse/GIRAPH-207
> ,
> which is also where I learned about the eclipse:eclipse target.
>
> How do others develop Giraph code then?
> Using Eclipse or other IDEs perhaps? (how?)
> Plain VI or EMACS editor with command line mvn commands?
>
> Thanks,
> Doron
>
--
Amani AlOnazi
MSc Computer Science
King Abdullah University of Science and Technology
Kingdom of Saudi Arabia
amani.alonazi@kaust.edu.sa |
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.