You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@groovy.apache.org by Andriy Rysin <ar...@gmail.com> on 2022/02/12 16:03:01 UTC
Running groovy programs
Hi all
I have a question about running/packaging groovy programs.
I have a small suite of (commandline) NLP tools (
https://github.com/brown-uk/nlp_uk) that is often used by non-developers
(NLP students, researches etc).
When the scripts where simple it was very easy to run them:
1) install groovy
2) git clone/pull
3) then you run scripts from anywhere with simple
..../nlp_uk/src/main/groovy/org/nlp_uk/tools/TagText.groovy -i
input_file.txt
The dependencies are pulled via grape, the command is simple to run and
update.
Now scripts got some common parallelization code and I extracted common
code into TextUtils but to keep things still simple I just use:
Eval.me(new File("$SCRIPT_DIR/TextUtils.groovy") .....
for a simple "include" hack.
Now one of the scripts got more complicated and more groovy classes needed
and going this approach gets a bit too much for Eval hack.
I looked into alternative approaches, and they all look heavy to me:
1) add fatJar target to build.gradle
a) extra step for the users to build after each update
b) run more complicated command with -cp
c) one of the dependencies I have does not work well with fatJar
(duplicated filepath in different submodules)
2) run commands via gradle, e.g. gradle runToolX -Pargs="..."
a) needs task for each tool (or argument to take which tool to run)
b) needs (ugly) -Pargs to wrap and pass arguments to the command
c) tricky to figure out current dir (although there are some hacks to use
"user.dir" system property)
3) provide shell/bat scripts to wrap things (e.g. -cp etc) for the users
(so far seems the easiest combination for both me to maintain and the users
to use)
a) this incurs groovy compile on all classes for every run (unless in the
shell script I combine gradle compile + run using built classes in the
classpath)
4) use jpackage/jlink
a) again - extra step to build for the users or me publishing official
versions pre-packaged (making it harder to just grab latest version)
b) packaging the whole set of dependencies for each update instead of
reusing grape cache
I would appreciate any suggestions,
Thanks,
Andriy
Re: Running groovy programs
Posted by Andriy Rysin <ar...@gmail.com>.
Hi Jochen and all,
so a while ago my groovy script grew (big enough) and I had to split it
into multiple classes. But I wanted my users to still be able to run it
with a simple command (e.g. ./TagText.groovy on Linux and groovy
TagText.groovy on Windows).
So now all my scripts have a wrapper, example of one of them is below - the
main script was called TagText.groovy and now the main functionality moved
to TagTextCore.groovy.
This works, but I had to add such a wrapper to all scripts like that. And I
feel that this type of code does not belong in the apps - this is
system-level functionality.
Regards,
Andriy
@CompileStatic
class TagText {
@groovy.transform.SourceURI
static URI SOURCE_URI
static String SCRIPT_DIR=new File(SOURCE_URI).parent
static void main(String[] args) {
long tm1 = System.currentTimeMillis()
def cl = new GroovyClassLoader()
cl.addClasspath(SCRIPT_DIR + "/../../../../")
def resourceDir = SCRIPT_DIR + "/../../../../../resources"
if( ! new File(resourceDir).isDirectory() ) {
new File(resourceDir).mkdirs()
}
cl.addClasspath(resourceDir)
def basePkg = TagText.class.getPackageName()
def tagTextClass = cl.loadClass("${basePkg}.tag.TagTextCore")
def m = tagTextClass.getMethod("main", String[].class)
def mArgs = [args].toArray() // new Object[]{args} - Eclipse chokes on this
m.invoke(null, mArgs)
}
}
On Thu, Mar 24, 2022 at 3:25 AM Jochen Theodorou <bl...@gmx.org> wrote:
> Hi Andriy,
>
> We can add an option, that is then changing the compiler configuration
> and a transform could react to that to add a path to the classpath
> (package corrected of course) This is certainly possible.
>
> bye Jochen
>
>
> On 23.03.22 16:37, Andriy Rysin wrote:
> > I wonder if the 'groovy' executable could be smart enough to detect: if
> > I am running src/pkg1/main.groovy and there's "package pkg1" in that
> > script that we need to add src/ to classpath.
> > But I guess for that groovy would have to compile it first so it's a
> > chicken and egg problem
> >
> > Andriy
> >
> > On Mon, Mar 21, 2022 at 3:41 PM Jochen Theodorou <blackdrag@gmx.org
> > <ma...@gmx.org>> wrote:
> >
> > On 21.03.22 15:26, Andriy Rysin wrote:
> > > Ah, sorry I didn't get it at first. Technically if we need extra
> > wrapper
> > > to start a program I could probably do a shell/bat scripts. I was
> > hoping
> > > we could use the same command (I have other tools that are
> invoked by
> > > their groovy file, so I was trying to keep the more complex one
> > the same).
> >
> > It surely would not be impossible to make a global transform, which
> > helps the compiler with the setup. But this global transform would
> have
> > to be on the classpath. And that could be done by placing it in
> > ~/.groovy/lib/
> >
> > The transform would then add the source path to the classpath for the
> > compiler used for the compilation during the conversion phase. Still
> > requires a one-time setup step of course.
> >
> > Out of the box I think we have no solution for this really.
> >
> > bye Jochen
> >
>
>
Re: Running groovy programs
Posted by Jochen Theodorou <bl...@gmx.org>.
Hi Andriy,
We can add an option, that is then changing the compiler configuration
and a transform could react to that to add a path to the classpath
(package corrected of course) This is certainly possible.
bye Jochen
On 23.03.22 16:37, Andriy Rysin wrote:
> I wonder if the 'groovy' executable could be smart enough to detect: if
> I am running src/pkg1/main.groovy and there's "package pkg1" in that
> script that we need to add src/ to classpath.
> But I guess for that groovy would have to compile it first so it's a
> chicken and egg problem
>
> Andriy
>
> On Mon, Mar 21, 2022 at 3:41 PM Jochen Theodorou <blackdrag@gmx.org
> <ma...@gmx.org>> wrote:
>
> On 21.03.22 15:26, Andriy Rysin wrote:
> > Ah, sorry I didn't get it at first. Technically if we need extra
> wrapper
> > to start a program I could probably do a shell/bat scripts. I was
> hoping
> > we could use the same command (I have other tools that are invoked by
> > their groovy file, so I was trying to keep the more complex one
> the same).
>
> It surely would not be impossible to make a global transform, which
> helps the compiler with the setup. But this global transform would have
> to be on the classpath. And that could be done by placing it in
> ~/.groovy/lib/
>
> The transform would then add the source path to the classpath for the
> compiler used for the compilation during the conversion phase. Still
> requires a one-time setup step of course.
>
> Out of the box I think we have no solution for this really.
>
> bye Jochen
>
Re: Running groovy programs
Posted by Andriy Rysin <ar...@gmail.com>.
I wonder if the 'groovy' executable could be smart enough to detect: if I
am running src/pkg1/main.groovy and there's "package pkg1" in that script
that we need to add src/ to classpath.
But I guess for that groovy would have to compile it first so it's a
chicken and egg problem
Andriy
On Mon, Mar 21, 2022 at 3:41 PM Jochen Theodorou <bl...@gmx.org> wrote:
> On 21.03.22 15:26, Andriy Rysin wrote:
> > Ah, sorry I didn't get it at first. Technically if we need extra wrapper
> > to start a program I could probably do a shell/bat scripts. I was hoping
> > we could use the same command (I have other tools that are invoked by
> > their groovy file, so I was trying to keep the more complex one the
> same).
>
> It surely would not be impossible to make a global transform, which
> helps the compiler with the setup. But this global transform would have
> to be on the classpath. And that could be done by placing it in
> ~/.groovy/lib/
>
> The transform would then add the source path to the classpath for the
> compiler used for the compilation during the conversion phase. Still
> requires a one-time setup step of course.
>
> Out of the box I think we have no solution for this really.
>
> bye Jochen
>
Re: Running groovy programs
Posted by Jochen Theodorou <bl...@gmx.org>.
On 21.03.22 15:26, Andriy Rysin wrote:
> Ah, sorry I didn't get it at first. Technically if we need extra wrapper
> to start a program I could probably do a shell/bat scripts. I was hoping
> we could use the same command (I have other tools that are invoked by
> their groovy file, so I was trying to keep the more complex one the same).
It surely would not be impossible to make a global transform, which
helps the compiler with the setup. But this global transform would have
to be on the classpath. And that could be done by placing it in
~/.groovy/lib/
The transform would then add the source path to the classpath for the
compiler used for the compilation during the conversion phase. Still
requires a one-time setup step of course.
Out of the box I think we have no solution for this really.
bye Jochen
Re: Running groovy programs
Posted by Andriy Rysin <ar...@gmail.com>.
Ah, sorry I didn't get it at first. Technically if we need extra wrapper to
start a program I could probably do a shell/bat scripts. I was hoping we
could use the same command (I have other tools that are invoked by their
groovy file, so I was trying to keep the more complex one the same).
Thanks,
Andriy
On Fri, Mar 18, 2022 at 12:46 PM Jochen Theodorou <bl...@gmx.org> wrote:
> On 18.03.22 15:32, Andriy Rysin wrote:
> > This approach may work well for simple file include but if I have
> > packages (and import statements, which is inevitable if program grows)
> > it does not compile.
>
> Let me explain my suggestion in more detail.
>
> you want to start a script Loader.groovy with groovy
> /path/to/scripts/Loader.groovy <myfile>
>
> The loader could look something like this:
>
> > import java.nio.file.Path
> >
> > @groovy.transform.SourceURI def sourceURI
> > assert sourceURI instanceof java.net.URI
> > def url = Path.of(sourceURI.path).parent.toUri().toURL()
> > def gcl = new GroovyClassLoader(this.class.classLoader)
> > gcl.addURL(url)
> > def c = gcl.loadClass("TagText")
> > c.main(this.args)
>
> Here I load the class TagText written in Groovy, no precompiled. Using
> sourceURI I found the directory where Loader.groovy resides in as url.
> Then I spawn a new class loader which knows this directory and let it
> load my actual main class TagText. For this to work TagText has to be in
> the same director as Loader. Should TagText be in a package, you have to
> use the package name as well of course. For example foo/TagText.groovy
> should have the package foo and then I would have to do
> loadClass("foo.TagText") to load it.
>
> This way I can load TagText from anywhere using Loader, as TagText is in
> a known relative position to Loader.
>
> TagText can then of course have further dependencies. You can also add
> jars this way (you have to produce an url with jar protocol then I think)
>
> BTW: this.args passes through the arguments. This way <myfile> is passed
> to TagText.
>
> bye Jochen
>
>
Re: Running groovy programs
Posted by Jochen Theodorou <bl...@gmx.org>.
On 18.03.22 15:32, Andriy Rysin wrote:
> This approach may work well for simple file include but if I have
> packages (and import statements, which is inevitable if program grows)
> it does not compile.
Let me explain my suggestion in more detail.
you want to start a script Loader.groovy with groovy
/path/to/scripts/Loader.groovy <myfile>
The loader could look something like this:
> import java.nio.file.Path
>
> @groovy.transform.SourceURI def sourceURI
> assert sourceURI instanceof java.net.URI
> def url = Path.of(sourceURI.path).parent.toUri().toURL()
> def gcl = new GroovyClassLoader(this.class.classLoader)
> gcl.addURL(url)
> def c = gcl.loadClass("TagText")
> c.main(this.args)
Here I load the class TagText written in Groovy, no precompiled. Using
sourceURI I found the directory where Loader.groovy resides in as url.
Then I spawn a new class loader which knows this directory and let it
load my actual main class TagText. For this to work TagText has to be in
the same director as Loader. Should TagText be in a package, you have to
use the package name as well of course. For example foo/TagText.groovy
should have the package foo and then I would have to do
loadClass("foo.TagText") to load it.
This way I can load TagText from anywhere using Loader, as TagText is in
a known relative position to Loader.
TagText can then of course have further dependencies. You can also add
jars this way (you have to produce an url with jar protocol then I think)
BTW: this.args passes through the arguments. This way <myfile> is passed
to TagText.
bye Jochen
Re: Running groovy programs
Posted by Andriy Rysin <ar...@gmail.com>.
This approach may work well for simple file include but if I have packages
(and import statements, which is inevitable if program grows) it does not
compile.
Andriy
On Wed, Mar 9, 2022 at 4:47 AM Jochen Theodorou <bl...@gmx.org> wrote:
> On 08.03.22 19:08, Andriy Rysin wrote:
> > So it looks like groovy picks up groovy scripts from the current
> > directory by default.
> > So the users can't just do "/path/to/scripts/TagText.groovy <myfile>"
> > any more.
> > They would have to do:
> > cd /path/to/scripts/
> > ./TagText.groovy <myfile>
> >
> > or they would have to invoke groovy explicitly:
> > groovy -cp /path/to/scripts /path/to/scripts/TagText.groovy <myfile>
>
> what you could also do is to make something like a small loader. Use
>
> https://docs.groovy-lang.org/latest/html/gapi/groovy/transform/SourceURI.html
> to get the uri of the file, then add the to the classloader to then
> finally load the class that is really doing the work and has
> dependencies on other scripts. This loader should not have any direct
> source dependencies for this to work.
>
> bye Jochen
>
>
Re: Running groovy programs
Posted by Jochen Theodorou <bl...@gmx.org>.
On 08.03.22 19:08, Andriy Rysin wrote:
> So it looks like groovy picks up groovy scripts from the current
> directory by default.
> So the users can't just do "/path/to/scripts/TagText.groovy <myfile>"
> any more.
> They would have to do:
> cd /path/to/scripts/
> ./TagText.groovy <myfile>
>
> or they would have to invoke groovy explicitly:
> groovy -cp /path/to/scripts /path/to/scripts/TagText.groovy <myfile>
what you could also do is to make something like a small loader. Use
https://docs.groovy-lang.org/latest/html/gapi/groovy/transform/SourceURI.html
to get the uri of the file, then add the to the classloader to then
finally load the class that is really doing the work and has
dependencies on other scripts. This loader should not have any direct
source dependencies for this to work.
bye Jochen
Re: Running groovy programs
Posted by Andriy Rysin <ar...@gmail.com>.
So it looks like groovy picks up groovy scripts from the current directory
by default.
So the users can't just do "/path/to/scripts/TagText.groovy <myfile>" any
more.
They would have to do:
cd /path/to/scripts/
./TagText.groovy <myfile>
or they would have to invoke groovy explicitly:
groovy -cp /path/to/scripts /path/to/scripts/TagText.groovy <myfile>
On Fri, Mar 4, 2022 at 7:04 AM Jochen Theodorou <bl...@gmx.org> wrote:
> On 12.02.22 17:03, Andriy Rysin wrote:
> > Hi all
> >
> > I have a question about running/packaging groovy programs.
> > I have a small suite of (commandline) NLP tools
> > (https://github.com/brown-uk/nlp_uk
> > <https://github.com/brown-uk/nlp_uk>) that is often used by
> > non-developers (NLP students, researches etc).
> > When the scripts where simple it was very easy to run them:
> > 1) install groovy
> > 2) git clone/pull
> > 3) then you run scripts from anywhere with simple
> > ..../nlp_uk/src/main/groovy/org/nlp_uk/tools/TagText.groovy -i
> > input_file.txt
> >
> > The dependencies are pulled via grape, the command is simple to run and
> > update.
> >
> > Now scripts got some common parallelization code and I extracted common
> > code into TextUtils but to keep things still simple I just use:
> > Eval.me(new File("$SCRIPT_DIR/TextUtils.groovy") .....
> > for a simple "include" hack.
>
> Let us assume TagText.groovy and TextUtils.groovy are in
> nlp_uk/src/main/groovy/org/nlp_uk/tools. Let us further assume, that
> TextUtils.groovy is a class and has a method extractTags(). Then you can
> do just "new TextUtils().extractTags()" in TagText and Groovy will try
> to resolve the class TextUtils by itself, leading to loading (and
> compiling on the fly) TextUtils.groovy and to execute the extractTags
> method from there.
>
> So all you need to do is to put the TextUtils file in a path Groovy can
> find (on the classpath) as well as make it a class. If you want to work
> with packages, then the file needs to be in a sub directory that fits to
> the package naming scheme.
>
> This will also pick up a change in TextUtils and TagText, since nothing
> is precompiled.
>
> If you do not want to work with instances of TextUtils, you can make the
> methods static and then do something like TextUtils.extractTags(). or
> you do "import static TextUtils.extractTags" to import the method and
> then just call it with "extractTags()"
>
> does this help?
>
> bye Jochen
>
>
Re: Running groovy programs
Posted by Jochen Theodorou <bl...@gmx.org>.
On 12.02.22 17:03, Andriy Rysin wrote:
> Hi all
>
> I have a question about running/packaging groovy programs.
> I have a small suite of (commandline) NLP tools
> (https://github.com/brown-uk/nlp_uk
> <https://github.com/brown-uk/nlp_uk>) that is often used by
> non-developers (NLP students, researches etc).
> When the scripts where simple it was very easy to run them:
> 1) install groovy
> 2) git clone/pull
> 3) then you run scripts from anywhere with simple
> ..../nlp_uk/src/main/groovy/org/nlp_uk/tools/TagText.groovy -i
> input_file.txt
>
> The dependencies are pulled via grape, the command is simple to run and
> update.
>
> Now scripts got some common parallelization code and I extracted common
> code into TextUtils but to keep things still simple I just use:
> Eval.me(new File("$SCRIPT_DIR/TextUtils.groovy") .....
> for a simple "include" hack.
Let us assume TagText.groovy and TextUtils.groovy are in
nlp_uk/src/main/groovy/org/nlp_uk/tools. Let us further assume, that
TextUtils.groovy is a class and has a method extractTags(). Then you can
do just "new TextUtils().extractTags()" in TagText and Groovy will try
to resolve the class TextUtils by itself, leading to loading (and
compiling on the fly) TextUtils.groovy and to execute the extractTags
method from there.
So all you need to do is to put the TextUtils file in a path Groovy can
find (on the classpath) as well as make it a class. If you want to work
with packages, then the file needs to be in a sub directory that fits to
the package naming scheme.
This will also pick up a change in TextUtils and TagText, since nothing
is precompiled.
If you do not want to work with instances of TextUtils, you can make the
methods static and then do something like TextUtils.extractTags(). or
you do "import static TextUtils.extractTags" to import the method and
then just call it with "extractTags()"
does this help?
bye Jochen