You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Michael Sokolov <ms...@gmail.com> on 2019/11/12 14:11:40 UTC
kicking tires of gradle build
Hi I am playing around with the gradle build. Overall looks great!
Thanks to everyone who has been pushing this forward. I have a few
questions; maybe just gradle noob questions, since I haven't used it
much (except as part of Android Studio, where all the details are kind
of taken care of for you).
1) I'm not sure which branch is the "current" one. Ideally I'd like to
be using a branch based off master. I see there are lots of branches:
jira/SOLR-13452_gradle_2, 3, 4, ... I started with _7 since that is
what is referenced in the wiki:
https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build,
but that seems to be a little out of date, so I switched to 8, because
you know, bigger is better. What are all these numbers? Just tracking
snapshots along the way? Which is the one based off 8x?
2) When I run any gradle command I get this warning:
> Configure project :
not user home user.gradle /home/sokolov/user.properties
Its grammar is throwing me: does it mean it expects to find these
files and can't find them? I have no user.properties file in my
homedir: should I?
3) I can run tests in a package using (eg) ./gradlew
lucene:lucene-core:test and see the test report output in an html file
- cool. Is it possible to get test output to stdout though? I am used
to running tests in emacs and have a script set up for parsing stack
traces in the output so emacs can jump there. I know I can use
intellij, and I often do, but I would like to also get the emacs
workflow going - definitely should not be a blocker for switching to
this - I am just looking for some hints as to getting errors logged to
stdout by gradle.
4) If I use the --tests option to specify a single test class to run,
and the class does not exist (I made a booboo, say), the build runs no
tests, and succeeds, but this is misleading: it should fail instead,
as the ant build does. Again, not a blocker, but if anybody knows how
to fix this, it would be great. I'll open an issue anyway.
again thanks, this looks awesome; with the daemon it runs so nicely :)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: kicking tires of gradle build
Posted by Michael Sokolov <ms...@gmail.com>.
ah as soon as I sent, I realized that failed test output goes to the
console, so all is well on that front. There are no dumb questions,
right?
On Tue, Nov 12, 2019 at 9:11 AM Michael Sokolov <ms...@gmail.com> wrote:
>
> Hi I am playing around with the gradle build. Overall looks great!
> Thanks to everyone who has been pushing this forward. I have a few
> questions; maybe just gradle noob questions, since I haven't used it
> much (except as part of Android Studio, where all the details are kind
> of taken care of for you).
>
> 1) I'm not sure which branch is the "current" one. Ideally I'd like to
> be using a branch based off master. I see there are lots of branches:
> jira/SOLR-13452_gradle_2, 3, 4, ... I started with _7 since that is
> what is referenced in the wiki:
> https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build,
> but that seems to be a little out of date, so I switched to 8, because
> you know, bigger is better. What are all these numbers? Just tracking
> snapshots along the way? Which is the one based off 8x?
>
> 2) When I run any gradle command I get this warning:
>
> > Configure project :
> not user home user.gradle /home/sokolov/user.properties
>
> Its grammar is throwing me: does it mean it expects to find these
> files and can't find them? I have no user.properties file in my
> homedir: should I?
>
> 3) I can run tests in a package using (eg) ./gradlew
> lucene:lucene-core:test and see the test report output in an html file
> - cool. Is it possible to get test output to stdout though? I am used
> to running tests in emacs and have a script set up for parsing stack
> traces in the output so emacs can jump there. I know I can use
> intellij, and I often do, but I would like to also get the emacs
> workflow going - definitely should not be a blocker for switching to
> this - I am just looking for some hints as to getting errors logged to
> stdout by gradle.
>
> 4) If I use the --tests option to specify a single test class to run,
> and the class does not exist (I made a booboo, say), the build runs no
> tests, and succeeds, but this is misleading: it should fail instead,
> as the ant build does. Again, not a blocker, but if anybody knows how
> to fix this, it would be great. I'll open an issue anyway.
>
> again thanks, this looks awesome; with the daemon it runs so nicely :)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: kicking tires of gradle build
Posted by Dawid Weiss <da...@gmail.com>.
> 1) I'm not sure which branch is the "current" one. Ideally I'd like to
> be using a branch based off master. I see there are lots of branches:
> 4) If I use the --tests option to specify a single test class to run,
> and the class does not exist (I made a booboo, say), the build runs no
> tests, and succeeds, but this is misleading: it should fail instead,
> as the ant build does. Again, not a blocker, but if anybody knows how
> to fix this, it would be great. I'll open an issue anyway.
I think the reason is that when you run an unqualified task against a
multi-project module all tasks of the given name (from all
subprojects) will be ordered and executed. This means that when you
run:
gradlew test --tests Foo
then some projects would end up failing the build because of zero
tests. This could be fixed by not failing on individual test tasks,
adding a listener and check the number of tests executed (from all
projects) after the build completes.
D.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: kicking tires of gradle build
Posted by Mark Miller <ma...@gmail.com>.
and A cool trick for solr tests.
So gradle does not do well for running long and short tests in a mix in
parallel - our ant stuff does much better after first run.
I run some cool gradle to work around this, but then remember Solr tests
could be fast enough that it doesn't matter. But until you have Solr tests
that fast, there is a trick that can help.
So if you set the max number of jvms for a module to use for tests very
high - like say 64, 128 and you set the max grade works to what you
actually want to allow to run, like for 16 cores, maybe 14-16. It's hard to
say - when you remove more blocking and non buttery slow stuff, you can use
more, in a bad state less.
But the idea, now you almost have like virtual JVMs and when run solr tests
across all its modules, you can get the heavy SolrCloud tests to start much
much sooner - which helps everything a lot. Just experiment.
On Tue, Nov 12, 2019 at 8:28 AM Mark Miller <ma...@gmail.com> wrote:
> You might want to pull out my jdeps stuff, probably that needs more work
> by someone up to speed - every project should be doing, its like java gave
> us this tool, it seems super underused.
>
> You might also pull the Docker test that tests tests the build - its slow
> but I couldn't live without it, but this is the type of stuff you guys
> would have to decide to keep and maintain.
>
> On Tue, Nov 12, 2019 at 8:23 AM Mark Miller <ma...@gmail.com> wrote:
>
>> I'm gonna help you here, cause im not sure anyone else fully knows.
>>
>> On Tue, Nov 12, 2019 at 8:12 AM Michael Sokolov <ms...@gmail.com>
>> wrote:
>>
>>> Hi I am playing around with the gradle build. Overall looks great!
>>> Thanks to everyone who has been pushing this forward. I have a few
>>> questions; maybe just gradle noob questions, since I haven't used it
>>> much (except as part of Android Studio, where all the details are kind
>>> of taken care of for you).
>>>
>>> 1) I'm not sure which branch is the "current" one. Ideally I'd like to
>>> be using a branch based off master. I see there are lots of branches:
>>> jira/SOLR-13452_gradle_2, 3, 4, ... I started with _7 since that is
>>> what is referenced in the wiki:
>>>
>>> https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build
>>> ,
>>> but that seems to be a little out of date, so I switched to 8, because
>>> you know, bigger is better. What are all these numbers? Just tracking
>>> snapshots along the way? Which is the one based off 8x?
>>>
>>>
>> 8 was the latest - why these numbers and branches? I made a new one when
>> I brought things up to trunk, because then I could rebase my work and it
>> kept things more sensible.
>>
>> Highest number, always latest, I didn't update the JIRA, I kind of got in
>> a little fight.
>>
>>
>>
>>> 2) When I run any gradle command I get this warning:
>>>
>>> > Configure project :
>>> not user home user.gradle /home/sokolov/user.properties
>>>
>>> Its grammar is throwing me: does it mean it expects to find these
>>> files and can't find them? I have no user.properties file in my
>>> homedir: should I?
>>>
>>
>> I hadn't quite polished this. Everyone was pissed off I hadn't given them
>> a way to configure things and recommended they use their ~/.gradle config
>> file and people didnt like, so this is an attmpt to let you configure by
>> adding a user.properties to the project folder or the you home dir and if
>> its there it should be sucked up and if not it shoudlnt matter.
>>
>>
>>>
>>> 3) I can run tests in a package using (eg) ./gradlew
>>> lucene:lucene-core:test and see the test report output in an html file
>>> - cool. Is it possible to get test output to stdout though? I am used
>>> to running tests in emacs and have a script set up for parsing stack
>>> traces in the output so emacs can jump there. I know I can use
>>> intellij, and I often do, but I would like to also get the emacs
>>> workflow going - definitely should not be a blocker for switching to
>>> this - I am just looking for some hints as to getting errors logged to
>>> stdout by gradle.
>>>
>>
>> Not sure, been a while, I only know stuff well when I work on it and then
>> I forget.
>>
>> The Gradle philosophy is kind of very very minimal output that does not
>> reallyscroll down unless you specify --debug or --info --stacktrace
>> orwhatever.
>>
>>
>>>
>>> 4) If I use the --tests option to specify a single test class to run,
>>> and the class does not exist (I made a booboo, say), the build runs no
>>> tests, and succeeds, but this is misleading: it should fail instead,
>>> as the ant build does. Again, not a blocker, but if anybody knows how
>>> to fix this, it would be great. I'll open an issue anyway.
>>>
>>
>> Yup, tweak that crap, this is probably to deal with some other issue and
>> we need to find the balance.
>>
>>
>>>
>>> again thanks, this looks awesome; with the daemon it runs so nicely :)
>>>
>>
>>
>> Dude it's so amazing and I don't say that because I built it - it took me
>> friggen forever, but I love it. Use the build cache, configure the right
>> number of works and test jvms and monitor your system load ... oh man, I
>> love that gradle build.
>>
>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
>>
>> --
>> - Mark
>>
>> http://about.me/markrmiller
>>
>
>
> --
> - Mark
>
> http://about.me/markrmiller
>
--
- Mark
http://about.me/markrmiller
Re: kicking tires of gradle build
Posted by Mark Miller <ma...@gmail.com>.
You might want to pull out my jdeps stuff, probably that needs more work by
someone up to speed - every project should be doing, its like java gave us
this tool, it seems super underused.
You might also pull the Docker test that tests tests the build - its slow
but I couldn't live without it, but this is the type of stuff you guys
would have to decide to keep and maintain.
On Tue, Nov 12, 2019 at 8:23 AM Mark Miller <ma...@gmail.com> wrote:
> I'm gonna help you here, cause im not sure anyone else fully knows.
>
> On Tue, Nov 12, 2019 at 8:12 AM Michael Sokolov <ms...@gmail.com>
> wrote:
>
>> Hi I am playing around with the gradle build. Overall looks great!
>> Thanks to everyone who has been pushing this forward. I have a few
>> questions; maybe just gradle noob questions, since I haven't used it
>> much (except as part of Android Studio, where all the details are kind
>> of taken care of for you).
>>
>> 1) I'm not sure which branch is the "current" one. Ideally I'd like to
>> be using a branch based off master. I see there are lots of branches:
>> jira/SOLR-13452_gradle_2, 3, 4, ... I started with _7 since that is
>> what is referenced in the wiki:
>> https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build
>> ,
>> but that seems to be a little out of date, so I switched to 8, because
>> you know, bigger is better. What are all these numbers? Just tracking
>> snapshots along the way? Which is the one based off 8x?
>>
>>
> 8 was the latest - why these numbers and branches? I made a new one when I
> brought things up to trunk, because then I could rebase my work and it kept
> things more sensible.
>
> Highest number, always latest, I didn't update the JIRA, I kind of got in
> a little fight.
>
>
>
>> 2) When I run any gradle command I get this warning:
>>
>> > Configure project :
>> not user home user.gradle /home/sokolov/user.properties
>>
>> Its grammar is throwing me: does it mean it expects to find these
>> files and can't find them? I have no user.properties file in my
>> homedir: should I?
>>
>
> I hadn't quite polished this. Everyone was pissed off I hadn't given them
> a way to configure things and recommended they use their ~/.gradle config
> file and people didnt like, so this is an attmpt to let you configure by
> adding a user.properties to the project folder or the you home dir and if
> its there it should be sucked up and if not it shoudlnt matter.
>
>
>>
>> 3) I can run tests in a package using (eg) ./gradlew
>> lucene:lucene-core:test and see the test report output in an html file
>> - cool. Is it possible to get test output to stdout though? I am used
>> to running tests in emacs and have a script set up for parsing stack
>> traces in the output so emacs can jump there. I know I can use
>> intellij, and I often do, but I would like to also get the emacs
>> workflow going - definitely should not be a blocker for switching to
>> this - I am just looking for some hints as to getting errors logged to
>> stdout by gradle.
>>
>
> Not sure, been a while, I only know stuff well when I work on it and then
> I forget.
>
> The Gradle philosophy is kind of very very minimal output that does not
> reallyscroll down unless you specify --debug or --info --stacktrace
> orwhatever.
>
>
>>
>> 4) If I use the --tests option to specify a single test class to run,
>> and the class does not exist (I made a booboo, say), the build runs no
>> tests, and succeeds, but this is misleading: it should fail instead,
>> as the ant build does. Again, not a blocker, but if anybody knows how
>> to fix this, it would be great. I'll open an issue anyway.
>>
>
> Yup, tweak that crap, this is probably to deal with some other issue and
> we need to find the balance.
>
>
>>
>> again thanks, this looks awesome; with the daemon it runs so nicely :)
>>
>
>
> Dude it's so amazing and I don't say that because I built it - it took me
> friggen forever, but I love it. Use the build cache, configure the right
> number of works and test jvms and monitor your system load ... oh man, I
> love that gradle build.
>
>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
> --
> - Mark
>
> http://about.me/markrmiller
>
--
- Mark
http://about.me/markrmiller
Re: kicking tires of gradle build
Posted by Mark Miller <ma...@gmail.com>.
I'm gonna help you here, cause im not sure anyone else fully knows.
On Tue, Nov 12, 2019 at 8:12 AM Michael Sokolov <ms...@gmail.com> wrote:
> Hi I am playing around with the gradle build. Overall looks great!
> Thanks to everyone who has been pushing this forward. I have a few
> questions; maybe just gradle noob questions, since I haven't used it
> much (except as part of Android Studio, where all the details are kind
> of taken care of for you).
>
> 1) I'm not sure which branch is the "current" one. Ideally I'd like to
> be using a branch based off master. I see there are lots of branches:
> jira/SOLR-13452_gradle_2, 3, 4, ... I started with _7 since that is
> what is referenced in the wiki:
> https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build
> ,
> but that seems to be a little out of date, so I switched to 8, because
> you know, bigger is better. What are all these numbers? Just tracking
> snapshots along the way? Which is the one based off 8x?
>
>
8 was the latest - why these numbers and branches? I made a new one when I
brought things up to trunk, because then I could rebase my work and it kept
things more sensible.
Highest number, always latest, I didn't update the JIRA, I kind of got in a
little fight.
> 2) When I run any gradle command I get this warning:
>
> > Configure project :
> not user home user.gradle /home/sokolov/user.properties
>
> Its grammar is throwing me: does it mean it expects to find these
> files and can't find them? I have no user.properties file in my
> homedir: should I?
>
I hadn't quite polished this. Everyone was pissed off I hadn't given them a
way to configure things and recommended they use their ~/.gradle config
file and people didnt like, so this is an attmpt to let you configure by
adding a user.properties to the project folder or the you home dir and if
its there it should be sucked up and if not it shoudlnt matter.
>
> 3) I can run tests in a package using (eg) ./gradlew
> lucene:lucene-core:test and see the test report output in an html file
> - cool. Is it possible to get test output to stdout though? I am used
> to running tests in emacs and have a script set up for parsing stack
> traces in the output so emacs can jump there. I know I can use
> intellij, and I often do, but I would like to also get the emacs
> workflow going - definitely should not be a blocker for switching to
> this - I am just looking for some hints as to getting errors logged to
> stdout by gradle.
>
Not sure, been a while, I only know stuff well when I work on it and then I
forget.
The Gradle philosophy is kind of very very minimal output that does not
reallyscroll down unless you specify --debug or --info --stacktrace
orwhatever.
>
> 4) If I use the --tests option to specify a single test class to run,
> and the class does not exist (I made a booboo, say), the build runs no
> tests, and succeeds, but this is misleading: it should fail instead,
> as the ant build does. Again, not a blocker, but if anybody knows how
> to fix this, it would be great. I'll open an issue anyway.
>
Yup, tweak that crap, this is probably to deal with some other issue and we
need to find the balance.
>
> again thanks, this looks awesome; with the daemon it runs so nicely :)
>
Dude it's so amazing and I don't say that because I built it - it took me
friggen forever, but I love it. Use the build cache, configure the right
number of works and test jvms and monitor your system load ... oh man, I
love that gradle build.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
--
- Mark
http://about.me/markrmiller
Re: kicking tires of gradle build
Posted by Michael Sokolov <ms...@gmail.com>.
Thanks for the answers. I opened
https://issues.apache.org/jira/browse/SOLR-13923 for the minor issue
about failing when no tests run. I'm happily running tests. Now just
need to tweak my emacs-lisp compilation-error-regexp to handle stack
trace output from tests ...
On Tue, Nov 12, 2019 at 10:14 AM Dawid Weiss <da...@gmail.com> wrote:
>
> Also, I wrote a short help for running tests under Gradle branch -- if
> it's of some help to you, please read it!
>
> https://github.com/apache/lucene-solr/blob/jira/SOLR-13452_gradle_7/buildSrc/common/help-text/testHelp.txt
>
> On Tue, Nov 12, 2019 at 3:12 PM Michael Sokolov <ms...@gmail.com> wrote:
> >
> > Hi I am playing around with the gradle build. Overall looks great!
> > Thanks to everyone who has been pushing this forward. I have a few
> > questions; maybe just gradle noob questions, since I haven't used it
> > much (except as part of Android Studio, where all the details are kind
> > of taken care of for you).
> >
> > 1) I'm not sure which branch is the "current" one. Ideally I'd like to
> > be using a branch based off master. I see there are lots of branches:
> > jira/SOLR-13452_gradle_2, 3, 4, ... I started with _7 since that is
> > what is referenced in the wiki:
> > https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build,
> > but that seems to be a little out of date, so I switched to 8, because
> > you know, bigger is better. What are all these numbers? Just tracking
> > snapshots along the way? Which is the one based off 8x?
> >
> > 2) When I run any gradle command I get this warning:
> >
> > > Configure project :
> > not user home user.gradle /home/sokolov/user.properties
> >
> > Its grammar is throwing me: does it mean it expects to find these
> > files and can't find them? I have no user.properties file in my
> > homedir: should I?
> >
> > 3) I can run tests in a package using (eg) ./gradlew
> > lucene:lucene-core:test and see the test report output in an html file
> > - cool. Is it possible to get test output to stdout though? I am used
> > to running tests in emacs and have a script set up for parsing stack
> > traces in the output so emacs can jump there. I know I can use
> > intellij, and I often do, but I would like to also get the emacs
> > workflow going - definitely should not be a blocker for switching to
> > this - I am just looking for some hints as to getting errors logged to
> > stdout by gradle.
> >
> > 4) If I use the --tests option to specify a single test class to run,
> > and the class does not exist (I made a booboo, say), the build runs no
> > tests, and succeeds, but this is misleading: it should fail instead,
> > as the ant build does. Again, not a blocker, but if anybody knows how
> > to fix this, it would be great. I'll open an issue anyway.
> >
> > again thanks, this looks awesome; with the daemon it runs so nicely :)
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: kicking tires of gradle build
Posted by Dawid Weiss <da...@gmail.com>.
Also, I wrote a short help for running tests under Gradle branch -- if
it's of some help to you, please read it!
https://github.com/apache/lucene-solr/blob/jira/SOLR-13452_gradle_7/buildSrc/common/help-text/testHelp.txt
On Tue, Nov 12, 2019 at 3:12 PM Michael Sokolov <ms...@gmail.com> wrote:
>
> Hi I am playing around with the gradle build. Overall looks great!
> Thanks to everyone who has been pushing this forward. I have a few
> questions; maybe just gradle noob questions, since I haven't used it
> much (except as part of Android Studio, where all the details are kind
> of taken care of for you).
>
> 1) I'm not sure which branch is the "current" one. Ideally I'd like to
> be using a branch based off master. I see there are lots of branches:
> jira/SOLR-13452_gradle_2, 3, 4, ... I started with _7 since that is
> what is referenced in the wiki:
> https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build,
> but that seems to be a little out of date, so I switched to 8, because
> you know, bigger is better. What are all these numbers? Just tracking
> snapshots along the way? Which is the one based off 8x?
>
> 2) When I run any gradle command I get this warning:
>
> > Configure project :
> not user home user.gradle /home/sokolov/user.properties
>
> Its grammar is throwing me: does it mean it expects to find these
> files and can't find them? I have no user.properties file in my
> homedir: should I?
>
> 3) I can run tests in a package using (eg) ./gradlew
> lucene:lucene-core:test and see the test report output in an html file
> - cool. Is it possible to get test output to stdout though? I am used
> to running tests in emacs and have a script set up for parsing stack
> traces in the output so emacs can jump there. I know I can use
> intellij, and I often do, but I would like to also get the emacs
> workflow going - definitely should not be a blocker for switching to
> this - I am just looking for some hints as to getting errors logged to
> stdout by gradle.
>
> 4) If I use the --tests option to specify a single test class to run,
> and the class does not exist (I made a booboo, say), the build runs no
> tests, and succeeds, but this is misleading: it should fail instead,
> as the ant build does. Again, not a blocker, but if anybody knows how
> to fix this, it would be great. I'll open an issue anyway.
>
> again thanks, this looks awesome; with the daemon it runs so nicely :)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org