Building OODT

[Thomas Bennett <tb...@ska.ac.za>]

Hi,

I hope that I'm posting these messages to the correct forum. If not, please
point me in the right direction.

I've been running into some issues with building OODT. I'm new to maven and
I'm building OODT from scratch.

Here is a summary of my discoveries:
 * mvn install 'freezes' when downloading some dependencies. I've tested and
even wget freezes. A ^C and a restart completes the download, but it seems
like odd behavior. It could very well be an issue on my side, but I thought
I would mention it.
 * There are some dependencies that may need to be fixed (or perhaps I need
to update my maven configuration)

My two problem dependencies so far are:
* Tika-0.8-SNAPSHOT
* hsqldb-1.8.0.7

Here is the build error for Tika:

[INFO]
------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO]
------------------------------------------------------------------------
[INFO] Failed to resolve artifact.

Missing:
----------
1) org.apache.tika:tika-core:jar:0.8-SNAPSHOT

Here is the build error for hsqldb

18 Nov 2010 4:06:10 PM
org.apache.oodt.cas.catalog.system.impl.CatalogServiceLocal
setCatalogRepository
INFO: Loaded Catalogs: '[]'
java.sql.BatchUpdateException: failed batch
at org.hsqldb.jdbc.jdbcStatement.executeBatch(Unknown Source)

My solutions for these have been:
* Build Tika jar file from source and run the following command: mvn
install:install-file -DgroupId=org.apache.tika -DartifactId=tika-core
-Dversion=0.8-SNAPSHOT -Dpackaging=jar -Dfile=test.jar
* I've changed the catalog pom.xml file to use hsqldb version 1.8.0.10, but
the error still occurs. I see that executeBatch is
in: org/hsqldb/jdbc/JDBCStatement.java for later versions of hsqldb.

I'm not sure what the best solution is for hsqldb. Any help welcome. Perhaps
the short term solution is for someone to send me a copy of their hsqldb
directory from .m2/repository :)

Kind regards,
Tom

-- 
Thomas Bennett

SKA South Africa

Office :  +2721 506 7341
Mobile : +2779 523 7105
Email  :  tbennett@ska.ac.za

Re: Building OODT

["Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>]

Thanks Victor, yep the M/L filtered it out. Tom should be able to get it from here though:

http://repo1.maven.org/maven2/hsqldb/hsqldb/1.8.0.7/

Cheers,
Chris

On Nov 18, 2010, at 8:21 AM, Hwang, Victor (388G) wrote:

> Just wanted to chime in - I also have that freezing problem as well, though it shows up very inconsistently.
> 
> As for hsqldb, here's a tar of my version which built for me (1.8.0.7)
> 
> Victor
> 
> <ATT00001..txt>


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: Chris.Mattmann@jpl.nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Re: Building OODT

["Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>]

Yep, Dave, +1...

Cheers,
Chris

On Nov 18, 2010, at 7:14 PM, David Kale wrote:

> I think it's fair to say that this is part of the maven building experience,
> for better or for worse.  Maven is automatically downloading a variety of
> dependencies from a variety of locations, and sometimes the mirrors from
> which it is downloading are less than responsive.  I suppose if I had my
> way, maven would do a better job of gracefully handling unresponsive or slow
> mirrors -- and of communicating what is going on to the user.
> 
> Whenever I get annoyed at it, I remind myself that the alternative would be
> to download the dependencies manually, which would also be pretty annoying.
> 
> On a philosophical note, I do wonder if systems like maven encourage
> excessive dependencies and reliance on external libraries.  For large
> projects like OODT, you watch literally tens or hundreds of dependencies go
> by and unless you're extremely curious and meticulous, you probably never
> bother to see what they are, what they do, etc.
> 
> Dave
> 
> 
> On Thu, Nov 18, 2010 at 8:21 AM, Hwang, Victor (388G) <
> Victor.Hwang@jpl.nasa.gov> wrote:
> 
>> Just wanted to chime in - I also have that freezing problem as well, though
>> it shows up very inconsistently.
>> 
>> As for hsqldb, here's a tar of my version which built for me (1.8.0.7)
>> 
>> Victor
>> 
>> 


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: Chris.Mattmann@jpl.nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Re: Building OODT

[David Kale <da...@cs.stanford.edu>]

I think it's fair to say that this is part of the maven building experience,
for better or for worse.  Maven is automatically downloading a variety of
dependencies from a variety of locations, and sometimes the mirrors from
which it is downloading are less than responsive.  I suppose if I had my
way, maven would do a better job of gracefully handling unresponsive or slow
mirrors -- and of communicating what is going on to the user.

Whenever I get annoyed at it, I remind myself that the alternative would be
to download the dependencies manually, which would also be pretty annoying.

On a philosophical note, I do wonder if systems like maven encourage
excessive dependencies and reliance on external libraries.  For large
projects like OODT, you watch literally tens or hundreds of dependencies go
by and unless you're extremely curious and meticulous, you probably never
bother to see what they are, what they do, etc.

Dave


On Thu, Nov 18, 2010 at 8:21 AM, Hwang, Victor (388G) <
Victor.Hwang@jpl.nasa.gov> wrote:

> Just wanted to chime in - I also have that freezing problem as well, though
> it shows up very inconsistently.
>
> As for hsqldb, here's a tar of my version which built for me (1.8.0.7)
>
> Victor
>
>

Re: Building OODT

[Justin Erenkrantz <ju...@erenkrantz.com>]

On Thu, Nov 18, 2010 at 2:47 PM, Hwang, Victor (388G)
<Vi...@jpl.nasa.gov> wrote:
> Hm, does the mailing list not take attachments? Is there somewhere specific I should upload it?

Yes, by default, the mailing lists will strip tarballs and other large
attachments.

You can download hsqldb 1.8.0.7 from:

http://mirrors.ibiblio.org/pub/mirrors/maven2/hsqldb/hsqldb/1.8.0.7/

HTH.  -- justin

Re: Building OODT

["Hwang, Victor (388G)" <Vi...@jpl.nasa.gov>]

Hm, does the mailing list not take attachments? Is there somewhere specific I should upload it?

Victor

Re: Building OODT

["Hwang, Victor (388G)" <Vi...@jpl.nasa.gov>]

Just wanted to chime in - I also have that freezing problem as well, though it shows up very inconsistently.

As for hsqldb, here's a tar of my version which built for me (1.8.0.7)

Victor

Re: Building OODT

[Cameron Goodale <si...@gmail.com>]

Victor,

>As for hsqldb, here's a tar of my version which built for me (1.8.0.7)

I was only able to see a txt file from your post no tarball.

Tom,

Did you get the tarball Victor provided from his posting?  If not let us
know.



Best Regards,

Cameron

-- 

Sent from a Tin Can attached to a String

Re: Building OODT

[Sean Kelly <ke...@apache.org>]

> * mvn install 'freezes' when downloading some dependencies.

I get that too. And yes, interrupting mvn and re-executing it seems to work around the problem just fine.

Eventually your personal m2 repository will have everything you'll need to build OODT—or other mvn-based projects.

Good luck,
Sean.

Re: Building OODT

["Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>]

Hi Tom,

Thanks for your message! Just as an FYI, your messages are getting through. One way to check is that you can visit any of the public archives for Apache projects, specifically you can find the OODT ones here [1] or here [2]. A lot of times I just Google and I can find them (b/c the archives are publicly indexed). So good job, you've got them through! :)

Now, onto some help below...

> 
> Here is a summary of my discoveries:
> * mvn install 'freezes' when downloading some dependencies. I've tested and
> even wget freezes. A ^C and a restart completes the download, but it seems
> like odd behavior. It could very well be an issue on my side, but I thought
> I would mention it.

Thanks for the heads up, here. Several OODT devs and users seem to be experiencing this too. Basically it has to do with the quality of the external dependency provider. In r1023552 [3] I added an external repository to the oodt/core/pom.xml file so that oodt/pushpull would build (it has an external dependency on Cog JGlobus for one of its FTP providers in the protocol layer). I think that this particular repository may be causing some issues for folks who don't already have a local copy of the jar files downloaded by Maven. 

The nice thing is like some other folks have pointed out, once you get the jars, you don't need to download them again, but one quick workaround might be to femove that repository from your local checkout of oodt/core/pom.xml in the repo definition section. If you do that, and your build speeds up, we can file an issue in JIRA to maybe remove that repo and find a more reliable provider

> * There are some dependencies that may need to be fixed (or perhaps I need
> to update my maven configuration)
> 
> My two problem dependencies so far are:
> * Tika-0.8-SNAPSHOT

Yep. The 0.1-incubating version of OODT relied on the latest version of Tika at the time (0.8-SNAPSHOT) because were kind of in between Tika releases. When I pushed the Tika 0.8 official release out, the Maven snapshots repository at Apache ceased to have that snapshot. 

Dave Kale fixed this in the latest trunk version (0.2-incubating) in OODT-59 [4], so you could grab the latest trunk version from SVN by doing the following:

1. svn co http://svn.apache.org/repos/asf/incubator/oodt/trunk ./oodt
2. cd oodt
3. execute build process as normal

Or, alternatively you could just grab the patch for OODT-59 and apply it locally to your checkout. Either way should work fine.

> * hsqldb-1.8.0.7

I think Justin mentioned that he found this dependency in a different repository, but I actually found it in Maven Central, here [5]. I'm wondering if you ran into some intermittent network connectivity issue with Maven Central. Can you check [5] and see if that works for you Tom?


> My solutions for these have been:
> * Build Tika jar file from source and run the following command: mvn
> install:install-file -DgroupId=org.apache.tika -DartifactId=tika-core
> -Dversion=0.8-SNAPSHOT -Dpackaging=jar -Dfile=test.jar

Wow, for being new to Maven (and Tika likely) you did a great job working around this! :)

> * I've changed the catalog pom.xml file to use hsqldb version 1.8.0.10, but
> the error still occurs. I see that executeBatch is
> in: org/hsqldb/jdbc/JDBCStatement.java for later versions of hsqldb.

Gotcha. Yep try grabbing the dependency from [5] (or see if Maven will download it for you now -- it should automatically check Maven Central).

> 
> I'm not sure what the best solution is for hsqldb. Any help welcome. Perhaps
> the short term solution is for someone to send me a copy of their hsqldb
> directory from .m2/repository :)

HTH!

Cheers,
Chris

[1] http://mail-archives.apache.org/mod_mbox/incubator-oodt-dev/
[2] http://www.mail-archive.com/oodt-dev@incubator.apache.org/
[3] http://svn.apache.org/viewvc/?rev=1023552&view=rev
[4] http://issues.apache.org/jira/browse/OODT-59
[5] http://repo1.maven.org/maven2/hsqldb/hsqldb/1.8.0.7/

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: Chris.Mattmann@jpl.nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++