You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Osma Suominen <os...@helsinki.fi> on 2020/12/04 17:00:43 UTC
Running CLI scripts from dev environment
Hi,
as I was investigating an apparent performance regression in tdbquery
(see my recent posts to the users list) I noticed that it's pretty
difficult to run the Jena command line tools from a development
environment. It's possible that I'm missing something obvious here, but
I think this has been a problem for others as well and at some point in
2015-2016 a fix was in place, but it's no longer working.
The problem is that the scripts expect a single directory with all the
relevant JARs that can be used as the classpath. But when building Jena,
the various JARs will be all around the directory tree. They get
collected into the .zip/.tar.gz packages though. So the easiest way to
run the scripts is to extract one of those archives. But that's a bit
cumbersome, it would be easier if one could run them straight after
building the source.
Apparently Rob noticed this while working on JENA-977 way back in 2015
and added a section to apache-jena/pom.xml that copied all the necessary
JARs to apache-jena/lib (commit 3c59213e273). This was further refined
by JENA-1173 - the problem was that the lib/ directory got included into
the source release zip files, so this added another section into
apache-jena/pom.xml to clean up the lib/ directory.
So far so good - I think at this point the mechanism was working so it
was easy to run the scripts from a dev environment (I didn't test this
though).
But then came commit a9701d2706a (by Andy in 2016) where the main
objective appears to be omitting javadocs, but the relevant sections in
the pom.xml were also commented out (and an obvious typo within that
section was corrected as well).
So we're now back to square one, and it's again hard to run the scripts
from a dev environment as there is no single lib/ directory that would
hold all the JARs.
As I said I may have missed something obvious. What is the recommended
way of running, say, tdbquery or riot, after just having built Jena?
-Osma
--
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Kaikukatu 4)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi
Re: Running CLI scripts from dev environment
Posted by Andy Seaborne <an...@apache.org>.
> The problem is that the scripts expect a single directory
> with all the relevant JARs that can be used as the classpath.
Fuseki jars have a copy of the commands in them.
java -cp DIR/jena-fuseki-server-3.18.0-SNAPSHOT.jar jena.riot --version
On 05/12/2020 07:25, Bruno P. Kinoshita wrote:
> Hi Osma,
> This is probably not the recommended way, but what I normally do when I want to test something from master is to call the Java command directly from Eclipse. Eclipse appears to take care of the dependencies required.
> I use the same approach for running Fuseki in development.
Yes - same here. Timing can be done relative to other runs in the same
environment,buit not to running from the command line.
> Bruno
In apache-jena:
mvn dependency:build-classpath \
-DincludeScope=test \
-Dmdep.outputFile=MyScript
java -cp "$(cat MyScript)" arq.sparql -h
Untested because I use a script that caches that output and prepends the
target/classes and target/test-classes directories.
Then a lot of symbolic links so
$ sparql --query ...
works.
As does:
"jena -ver 3.15.0 sparql" works a
Because it caches "mvn dependency:build-classpath -DincludeScope=test"
which needs to be reset when a release happens or the external
dependencies change in a release cycle.
https://gist.github.com/afs/10810a6916f969b87c220d00dafd76d1
(check the details - has file names built in)
What would be really nice is a script to run GraalVM and generate a
precompiled AOT binary for faster command start up. (AOT = Ahead Of Time).
"A script" because we are entering into a more diverse world of CPU
instruction sets.
>
> On Saturday, 5 December 2020, 6:00:56 am NZDT, Osma Suominen <os...@helsinki.fi> wrote:
>
> Hi,
>
> as I was investigating an apparent performance regression in tdbquery
> (see my recent posts to the users list) I noticed that it's pretty
> difficult to run the Jena command line tools from a development
> environment. It's possible that I'm missing something obvious here, but
> I think this has been a problem for others as well and at some point in
> 2015-2016 a fix was in place, but it's no longer working.
>
>
> The problem is that the scripts expect a single directory with all the
> relevant JARs that can be used as the classpath. But when building Jena,
> the various JARs will be all around the directory tree. They get
> collected into the .zip/.tar.gz packages though. So the easiest way to
> run the scripts is to extract one of those archives. But that's a bit
> cumbersome, it would be easier if one could run them straight after
> building the source.
>
> Apparently Rob noticed this while working on JENA-977 way back in 2015
> and added a section to apache-jena/pom.xml that copied all the necessary
> JARs to apache-jena/lib (commit 3c59213e273).
>
> This was further refined
> by JENA-1173 - the problem was that the lib/ directory got included into
> the source release zip files, so this added another section into
> apache-jena/pom.xml to clean up the lib/ directory.
> So far so good - I think at this point the mechanism was working so it
> was easy to run the scripts from a dev environment (I didn't test this
> though).
>
> But then came commit a9701d2706a (by Andy in 2016) where the main
> objective appears to be omitting javadocs, but the relevant sections in
> the pom.xml were also commented out (and an obvious typo within that
> section was corrected as well).
>
> So we're now back to square one, and it's again hard to run the scripts
> from a dev environment as there is no single lib/ directory that would
> hold all the JARs.
>
>
> As I said I may have missed something obvious. What is the recommended
> way of running, say, tdbquery or riot, after just having built Jena?
>
> -Osma
>
>
Re: Running CLI scripts from dev environment
Posted by "Bruno P. Kinoshita" <ki...@apache.org>.
Hi Osma,
This is probably not the recommended way, but what I normally do when I want to test something from master is to call the Java command directly from Eclipse. Eclipse appears to take care of the dependencies required.
I use the same approach for running Fuseki in development.
Bruno
On Saturday, 5 December 2020, 6:00:56 am NZDT, Osma Suominen <os...@helsinki.fi> wrote:
Hi,
as I was investigating an apparent performance regression in tdbquery
(see my recent posts to the users list) I noticed that it's pretty
difficult to run the Jena command line tools from a development
environment. It's possible that I'm missing something obvious here, but
I think this has been a problem for others as well and at some point in
2015-2016 a fix was in place, but it's no longer working.
The problem is that the scripts expect a single directory with all the
relevant JARs that can be used as the classpath. But when building Jena,
the various JARs will be all around the directory tree. They get
collected into the .zip/.tar.gz packages though. So the easiest way to
run the scripts is to extract one of those archives. But that's a bit
cumbersome, it would be easier if one could run them straight after
building the source.
Apparently Rob noticed this while working on JENA-977 way back in 2015
and added a section to apache-jena/pom.xml that copied all the necessary
JARs to apache-jena/lib (commit 3c59213e273). This was further refined
by JENA-1173 - the problem was that the lib/ directory got included into
the source release zip files, so this added another section into
apache-jena/pom.xml to clean up the lib/ directory.
So far so good - I think at this point the mechanism was working so it
was easy to run the scripts from a dev environment (I didn't test this
though).
But then came commit a9701d2706a (by Andy in 2016) where the main
objective appears to be omitting javadocs, but the relevant sections in
the pom.xml were also commented out (and an obvious typo within that
section was corrected as well).
So we're now back to square one, and it's again hard to run the scripts
from a dev environment as there is no single lib/ directory that would
hold all the JARs.
As I said I may have missed something obvious. What is the recommended
way of running, say, tdbquery or riot, after just having built Jena?
-Osma
--
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Kaikukatu 4)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi