You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Nathan Hartman <ha...@gmail.com> on 2023/11/03 04:00:32 UTC
Re: New release
On Mon, Oct 16, 2023 at 9:16 AM Daniel Sahlberg
<da...@gmail.com> wrote:
>
> Den sön 15 okt. 2023 kl 06:50 skrev Nathan Hartman <ha...@gmail.com>:
>>
>> On Fri, Oct 13, 2023 at 1:58 PM Johan Corveleyn <jc...@gmail.com> wrote:
>>>
>>> On Fri, Oct 13, 2023 at 5:35 PM Stefan Sperling <st...@apache.org> wrote:
>>> >
>>> > On Fri, Oct 13, 2023 at 08:43:59AM +0200, Daniel Sahlberg wrote:
>>> > > Hi,
>>> > >
>>> > > There are quite a number of improvements waiting to be released. Can we
>>> > > muster the energy to do a new release?
>>> > >
>>> > > In trunk there are a lot of changes that warrant a 1.15, but before doing
>>> > > that I think we should also go back to the discussion of changing the /
>>> > > adding a checksum algorithm in the WC. That deserves a separate thread and
>>> > > I'm half way through summarising the previous discussions so I'd like to
>>> > > hold back 1.15 for the moment.
>>> > >
>>> > > Still in 1.14 there have been a number of bugfixes that might be good to
>>> > > get released. Maybe doing 1.14.3 first could set us up to do 1.15 a little
>>> > > later?
>>> >
>>> > The first problem to solve before the ball starts rolling would be
>>> > finding a release manager. I don't have enough spare time to play RM
>>> > this time around but I would support the RM as far as my time allows for.
>>> >
>>> > Doing 1.14.3 first sounds like a good plan. This would help potential
>>> > new releases managers to get bootstrapped into the process. A major
>>> > release tends to involve a little bit more effort because some new problems
>>> > may only show up on specific operating system platforms during -rc testing.
>>>
>>> +1
>>>
>>> --
>>> Johan
>>
>>
>>
>> Hi all,
>>
>> I agree that a 1.14.3 release makes sense for a first-time RM, to be followed by a 1.15.0 at a later stage. (Also, there are some unresolved issues regarding 1.15.0, such as the above-mentioned question of checksum algorithms, that may require more development, but 1.14.3 should be much more straightforward.)
>>
>> At this moment I cannot promise to volunteer for this cycle but I will try to do so.
>>
>> Before I do, I need to get my system bootstrapped properly. I can build SVN and run most of the testsuite, but one pesky longstanding issue I have had is with the JavaHL bindings. That will require a separate thread and I'll describe the issues there. I'll check my notes to see what else might be a showstopper on my end.
>
>
> I've successfully built and tested the JavaHL bindings at one point and I think I can dig out my notes to help you if needed.
>
>>
>>
>> If I can get set up and solve the problems I've encountered previously in a reasonable time frame (and everyone's help is appreciated of course) I'll volunteer.
>
>
> That would be great! I'd volunteer to do the next release after this.
>
> There were some Docker scripts floating around (Both from CMike and from Mark). Maybe we should figure out if these can be included in /tools as a help to set up the environment and to execute the necessary steps.
>
> Kind regards,
> Daniel
Hi all,
Previously I mentioned I plan to RM for the upcoming 1.14.3 release.
This being my first time, I need to solve some issues first.
One of these, which has been a stumbling block for me since the
beginning, is getting the JavaHL bindings to build and test
successfully.
I have made progress on this, but not a complete breakthrough yet.
The progress was discovering that I need to have $JAVA_HOME set in my
environment; without this, our build system (in build/ac-macros/java.m4)
fails to find javac and assigns 'none' to JAVAC; then, much later,
'make check-javahl' tries to call this 'none', which unsurprisingly
doesn't work. (I was surprised that the other logic in this AC_DEFUN
couldn't find javac without $JAVA_HOME being set, but that's a subject
for another thread.) The clue was the use of $JAVA_HOME there.
I tried various values for $JAVA_HOME but nothing that seemed sensible
worked; based on a StackOverflow answer [1], I ended up setting
$JAVA_HOME as follows:
$ export JAVA_HOME=$(readlink -f /usr/bin/javac | sed "s:/bin/javac::")
On my system, this sets $JAVA_HOME to:
/usr/lib/jvm/java-17-openjdk-amd64
This value seems a little strange IMHO but it finally got 'configure' to
find the JDK successfully and got me past the longstanding 'none'
problem!
But then I promptly ran into the next issue:
[[[
$ make check-javahl
(snip)
/usr/lib/jvm/java-17-openjdk-amd64/bin/java -Xcheck:jni
"-Dtest.rootdir=/home/nathan/ramdrive/svndev/svn-1.14.x/subversion/bindings/javahl/test-work"
"-Dtest.srcdir=/home/nathan/ramdrive/svndev/svn-1.14.x/subversion/bindings/javahl"
"-Dtest.rooturl=" "-Dtest.fstype="
"-Djava.library.path=:/home/nathan/ramdrive/svndev/prefix/svn-1.14.x/lib"
-classpath "subversion/bindings/javahl/classes:/home/nathan/ramdrive/svndev/svn-1.14.x/subversion/bindings/javahl/src:"
"-Dtest.cleanup=" "-Dtest.tests="
org.apache.subversion.javahl.RunTests
Error: Could not find or load main class org.apache.subversion.javahl.RunTests
Caused by: java.lang.ClassNotFoundException:
org.apache.subversion.javahl.RunTests
make: *** [Makefile:530: check-apache-javahl] Error 1
]]]
If I understand correctly, there should be a file called
'RunTests.class' in one of the directories specified with -classpath
above. Well, there are other .class files to be found there, but not
RunTests.class. A search for RunTests with any (or no) extension in the
1.14.x tree gives these two RunTests.java (not .class) files:
$ find . -name RunTests\*
./subversion/bindings/javahl/tests/org/apache/subversion/javahl/RunTests.java
./subversion/bindings/javahl/tests/org/tigris/subversion/javahl/RunTests.java
Forgive me if this is something obvious, but I know almost nothing about
the Java ecosystem. :-)
I'll have to continue this exploration tomorrow, but I'm at least glad
to finally have made progress on a longstanding stumbling block.
Meanwhile, any clues would be appreciated!
[1] https://stackoverflow.com/a/29622512
Cheers,
Nathan
Re: New release
Posted by Daniel Sahlberg <da...@gmail.com>.
Great work! :-)
/Daniel
Den tors 30 nov. 2023 kl 06:21 skrev Nathan Hartman <
hartman.nathan@gmail.com>:
> I have kept myself busy studying the information in HACKING [1], in
> Mark's SVN Release Management repo [2], in tools/dist/release.py, etc.
>
> A quick test run with release.py seems to work (mostly) correctly. It
> successfully installs the needed Autoconf, Libtool, and SWIG. It is
> able to generate tarballs and Windows zips (though I haven't tested
> these at all; they're just throwaways right now).
>
> There were a few warnings while rolling these throwaways:
>
> * Copyright year is stale; I committed r1914220 and r1914221 to address
> that.
>
> * CHANGES has unmerged revisions; that's expected because I haven't
> updated CHANGES yet. That's the next task on my list.
>
> * It complained that COMMITTERS file is not found, therefore
> make-keys.sh failed. I remember seeing something about this in Mark's
> svnrm repo notes. Need to revisit that...
>
> Also a couple of questions:
>
> Do we notify translators before a patch release? (i.e., should I send
> out an email as described in #notify-translators [3]?)
>
> I need to guesstimate the release date. If I can get to the point of
> rolling tarballs within the next few days, how much time will be needed
> by those who plan to test?
>
> Thanks,
> Nathan
>
> [1]
> https://subversion.apache.org/docs/community-guide/community-guide.html#release-creating
>
> [2] https://github.com/markphip/svnrm
>
> [3]
> https://subversion.apache.org/docs/community-guide/community-guide.html#notify-translators
>
Re: New release
Posted by Nathan Hartman <ha...@gmail.com>.
I have kept myself busy studying the information in HACKING [1], in
Mark's SVN Release Management repo [2], in tools/dist/release.py, etc.
A quick test run with release.py seems to work (mostly) correctly. It
successfully installs the needed Autoconf, Libtool, and SWIG. It is
able to generate tarballs and Windows zips (though I haven't tested
these at all; they're just throwaways right now).
There were a few warnings while rolling these throwaways:
* Copyright year is stale; I committed r1914220 and r1914221 to address
that.
* CHANGES has unmerged revisions; that's expected because I haven't
updated CHANGES yet. That's the next task on my list.
* It complained that COMMITTERS file is not found, therefore
make-keys.sh failed. I remember seeing something about this in Mark's
svnrm repo notes. Need to revisit that...
Also a couple of questions:
Do we notify translators before a patch release? (i.e., should I send
out an email as described in #notify-translators [3]?)
I need to guesstimate the release date. If I can get to the point of
rolling tarballs within the next few days, how much time will be needed
by those who plan to test?
Thanks,
Nathan
[1] https://subversion.apache.org/docs/community-guide/community-guide.html#release-creating
[2] https://github.com/markphip/svnrm
[3] https://subversion.apache.org/docs/community-guide/community-guide.html#notify-translators
Re: New release
Posted by Daniel Sahlberg <da...@gmail.com>.
Den sön 12 nov. 2023 kl 07:18 skrev Nathan Hartman <hartman.nathan@gmail.com
>:
> On Fri, Nov 10, 2023 at 9:16 AM Daniel Sahlberg
> <da...@gmail.com> wrote:
> >
> > Den fre 10 nov. 2023 kl 15:09 skrev Nathan Hartman <
> hartman.nathan@gmail.com>:
> >>
> >> On Fri, Nov 10, 2023 at 8:02 AM Johan Corveleyn <jc...@gmail.com>
> wrote:
> >>>
> >>> Hi Nathan,
> >>>
> >>> Sorry I haven't helped so far ... I should be able to say something
> >>> useful here, since I'm a java dev in my dayjob. But I have been
> >>> drowning a bit in that same dayjob lately, so I can barely find the
> >>> time to scan these mails, and then forget about them because my
> >>> attention is needed elsewhere.
> >>>
> >>> Anyway, I'll try to find some more time to take a closer look at the
> >>> errors you get.
> >>>
> >>> Just as a first shot: if there is no RunTests.class in the location
> >>> where it should be (one of the directories in the classpath), then it
> >>> probably has not been compiled (or it has not been put in the right
> >>> location). My first guess would also be, like Daniel mentioned, that
> >>> you'll need the --with-junit option (referring to a junit-X.Y.jar
> >>> somewhere on your system) and then the build scripts should compile
> >>> the test classes.
> >>>
> >>> But I guess you already tried that, after Daniel's reply, and you're
> >>> still running into problems. I'll take a closer look when you send
> >>> your detailed mail then.
> >>>
> >>> --
> >>> Johan
> >>
> >>
> >>
> >> Thanks Johan. Apologies for not sending the detailed mail yet. I wanted
> to test a few specific theories but didn't get that far last night because
> I hit a problem unrelated to SVN.
> >>
> >> One thing I'm struggling with is to understand which part of the build
> system is supposed to compile RunTests, i.e., how/where the Java compiler
> is being invoked on it. I'm not seeing it in the build logs, though I do
> see the other Java files being compiled, so that's certainly interesting.
> >
> >
> > I'm fairly sure that RunTests is compiled only when --with-junit is
> present (don't know if "present" is enough or if it has to point to a valid
> JAR file). Next question is if check-javahl should be available at all if
> --with-junit is missing or if check-javahl should make sure RunTests is
> built. Either way, I think it is a bug in the build system. This question
> should probably be carved out to its own e-mail thread. I'll try to figure
> something out and start a new thread.
> >
> > Kind regards,
> > Daniel
>
> Hi all,
>
> Success! I have finally seen the JavaHL tests build and run!
>
> I'd like to say a great big Thank You to everyone for helping me get
> this far!
>
>
[...]
>
> (3) Worked around a problem with serf-1.3.10 that DSahlberg saw also.
>
>
[...]
> Regarding #3:
>
> When building serf-1.3.10, this warning foreshadows problems to come
> later:
>
> buckets/ssl_buckets.c:1193:9: warning: implicit declaration of
> function 'CRYPTO_malloc_init'; did you mean 'CRYPTO_malloc'?
> [-Wimplicit-function-declaration]
>
> Later, when trying to link SVN, this linker error happens:
>
> /usr/bin/ld: /home/nathan/ramdrive/svndev/prefix/serf/lib/libserf-1.so.1:
> undefined reference to `CRYPTO_malloc_init'
> collect2: error: ld returned 1 exit status
> make[1]: *** [build-outputs.mk:1140: subversion/svn/svn] Error 1
> make[1]: Leaving directory '/home/nathan/ramdrive/svndev/svn-1.14.x'
> make: *** [Makefile:1845:
> /home/nathan/ramdrive/svndev/objdir/svn-1.14.x/.compiled] Error 2
>
> For several years, I have been self-building SVN with serf-1.4.x and
> running the test suite successfully with that. I'm referring to serf's
> branches/1.4.x, last changed rev r1845547.
>
Note that serf 1.4 is build using CMake so things might behave differently
compared to SCons.
> IIRC this was suggested by Brane and it's on my TODO list to locate
> that conversation in the archives to see whether that was for this same
> reason.
>
> Perhaps something can be backported to serf-1.3.x to solve this. It
> manifests on recent Debian (DSahlberg reproduced this also [3]).
>
I did some more digging this morning and Nathan was onto something on IRC
[1] when he disabled the merge of the serf/branches/1.3.x-sslbuild branch.
It seems this branch mess things up when building on Debian. Some parts of
the branch are probably needed (including some updates to SConstruct to
handle building with Python3 when building 1.3.9) but it seems that some
changes conflict with improvements in later versions of Serf. I plan to dig
deeper (including setting up OpenBSD and test with LibreSSL) to see if we
can simplify the -sslbuild branch or even upstream some of the changes.
Kind regards,
Daniel
[1] https://colabti.org/irclogger/irclogger_log/svn-dev?date=2023-11-12
Re: New release
Posted by Nathan Hartman <ha...@gmail.com>.
On Fri, Nov 10, 2023 at 9:16 AM Daniel Sahlberg
<da...@gmail.com> wrote:
>
> Den fre 10 nov. 2023 kl 15:09 skrev Nathan Hartman <ha...@gmail.com>:
>>
>> On Fri, Nov 10, 2023 at 8:02 AM Johan Corveleyn <jc...@gmail.com> wrote:
>>>
>>> Hi Nathan,
>>>
>>> Sorry I haven't helped so far ... I should be able to say something
>>> useful here, since I'm a java dev in my dayjob. But I have been
>>> drowning a bit in that same dayjob lately, so I can barely find the
>>> time to scan these mails, and then forget about them because my
>>> attention is needed elsewhere.
>>>
>>> Anyway, I'll try to find some more time to take a closer look at the
>>> errors you get.
>>>
>>> Just as a first shot: if there is no RunTests.class in the location
>>> where it should be (one of the directories in the classpath), then it
>>> probably has not been compiled (or it has not been put in the right
>>> location). My first guess would also be, like Daniel mentioned, that
>>> you'll need the --with-junit option (referring to a junit-X.Y.jar
>>> somewhere on your system) and then the build scripts should compile
>>> the test classes.
>>>
>>> But I guess you already tried that, after Daniel's reply, and you're
>>> still running into problems. I'll take a closer look when you send
>>> your detailed mail then.
>>>
>>> --
>>> Johan
>>
>>
>>
>> Thanks Johan. Apologies for not sending the detailed mail yet. I wanted to test a few specific theories but didn't get that far last night because I hit a problem unrelated to SVN.
>>
>> One thing I'm struggling with is to understand which part of the build system is supposed to compile RunTests, i.e., how/where the Java compiler is being invoked on it. I'm not seeing it in the build logs, though I do see the other Java files being compiled, so that's certainly interesting.
>
>
> I'm fairly sure that RunTests is compiled only when --with-junit is present (don't know if "present" is enough or if it has to point to a valid JAR file). Next question is if check-javahl should be available at all if --with-junit is missing or if check-javahl should make sure RunTests is built. Either way, I think it is a bug in the build system. This question should probably be carved out to its own e-mail thread. I'll try to figure something out and start a new thread.
>
> Kind regards,
> Daniel
Hi all,
Success! I have finally seen the JavaHL tests build and run!
I'd like to say a great big Thank You to everyone for helping me get
this far!
I was writing on IRC as I went through a number of steps [1].
What changed?
(1) Switched from my customized Makefile.svn to SVN's Makefile.svn, as
of the latest rev on trunk now (r1913724).
(2) Worked around a problem building gettext.
(3) Worked around a problem with serf-1.3.10 that DSahlberg saw also.
Regarding #1:
Note: Makefile.svn refers to tools/dev/unix-build/Makefile.svn.
My customized Makefile.svn was based on SVN's Makefile.svn@1871775. As
to my customizations, I can provide them later, but the tl;dr is:
(a) Build newer versions of some dependencies.
(b) Remove support for things I didn't need (like serf-old).
(c) Tweak some details to workaround build problems.
In all cases, any changes I made were done to workaround various build
problems I had encountered. Where appropriate, I upstreamed certain
changes: r1875794, r1872380. I didn't upstream changes that I
considered to be tweaks that only pertained to my machine.
This made it possible for me to build, develop, and test
[local,svn,serf] x [bdb,fsfs], plus all the bindings, except JavaHL
(until now).
In the meantime (since 2019 when r1871775 happened), SVN's Makefile.svn
has received many updates, including much improved handling of
Python-2/3 issues, so I wanted to adopt this and eliminate my
customized one.
Regarding #2:
Starting a few days ago, I ran into a weird problem with gettext. This
manifests with both versions of Makefile.svn (mine and SVN's).
I am not sure why this started happening, but when gettext's configure
runs, there is a nested gettext-tools's configure, which runs until it
prints "checking for Java virtual machine... yes" and then it gets
stuck there indefinitely. It happens with multiple versions of gettext
and after a few nights of investigation, I still don't know why. I
could find nothing about this online. (This is the "brick wall" I ran
into that I mentioned on IRC [2], for those who saw that and wondered.)
It is clearly some idiosyncracy of my machine.
Tonight I found a workaround:
I ran Makefile.svn's gettext-configure target, let it reach that point,
and then killed the process group.
Then I looked in the logs to see exactly how gettext's configure was
invoked.
I then invoked it "manually" by copying and pasting that incantation
into the terminal.
To my surprise, this ran to completion.
After this, I ran 'touch objdir/gettext/.configured' (this empty
dotfile serves as a flag for Makefile.svn not to run configure again).
Then I ran Makefile.svn's gettext-compile target, and it built
successfully.
Why it behaves differently when called from make vs when I call it
manually is still a mystery to me. Maybe some dumb thing like
insufficient PTYs or who knows what. But it's repeatable like
clockwork, ever since a few days ago.
Having gotten past that, for now, I proceeded to build the rest of the
distro, until...
Regarding #3:
When building serf-1.3.10, this warning foreshadows problems to come
later:
buckets/ssl_buckets.c:1193:9: warning: implicit declaration of
function 'CRYPTO_malloc_init'; did you mean 'CRYPTO_malloc'?
[-Wimplicit-function-declaration]
Later, when trying to link SVN, this linker error happens:
/usr/bin/ld: /home/nathan/ramdrive/svndev/prefix/serf/lib/libserf-1.so.1:
undefined reference to `CRYPTO_malloc_init'
collect2: error: ld returned 1 exit status
make[1]: *** [build-outputs.mk:1140: subversion/svn/svn] Error 1
make[1]: Leaving directory '/home/nathan/ramdrive/svndev/svn-1.14.x'
make: *** [Makefile:1845:
/home/nathan/ramdrive/svndev/objdir/svn-1.14.x/.compiled] Error 2
For several years, I have been self-building SVN with serf-1.4.x and
running the test suite successfully with that. I'm referring to serf's
branches/1.4.x, last changed rev r1845547.
IIRC this was suggested by Brane and it's on my TODO list to locate
that conversation in the archives to see whether that was for this same
reason.
Perhaps something can be backported to serf-1.3.x to solve this. It
manifests on recent Debian (DSahlberg reproduced this also [3]).
My machine:
$ uname -a
Linux box 5.10.0-26-amd64 #1 SMP Debian 5.10.197-1 (2023-09-29) x86_64 GNU/Linux
$ lsb_release -d
Description: Debian GNU/Linux 11 (bullseye)
FWIW (since this was asked) SVN's configure is being called with:
--enable-javahl=yes
--with-jdk
--with-jikes=no
--with-junit=/home/nathan/ramdrive/svndev/distfiles/junit-4.10.jar
(and many other options).
Thanks again to everyone for your help and patience! I feel like I've
finally made progress on a nagging issue that existed for a long time.
[1] https://colabti.org/irclogger/irclogger_log/svn-dev?date=2023-11-12
[2] https://colabti.org/irclogger/irclogger_log/svn-dev?date=2023-11-09
[3] https://colabti.org/irclogger/irclogger_log/svn-dev?date=2023-11-11#l137
Cheers,
Nathan
Re: New release
Posted by Daniel Sahlberg <da...@gmail.com>.
Den fre 10 nov. 2023 kl 15:09 skrev Nathan Hartman <hartman.nathan@gmail.com
>:
> On Fri, Nov 10, 2023 at 8:02 AM Johan Corveleyn <jc...@gmail.com> wrote:
>
>> Hi Nathan,
>>
>> Sorry I haven't helped so far ... I should be able to say something
>> useful here, since I'm a java dev in my dayjob. But I have been
>> drowning a bit in that same dayjob lately, so I can barely find the
>> time to scan these mails, and then forget about them because my
>> attention is needed elsewhere.
>>
>> Anyway, I'll try to find some more time to take a closer look at the
>> errors you get.
>>
>> Just as a first shot: if there is no RunTests.class in the location
>> where it should be (one of the directories in the classpath), then it
>> probably has not been compiled (or it has not been put in the right
>> location). My first guess would also be, like Daniel mentioned, that
>> you'll need the --with-junit option (referring to a junit-X.Y.jar
>> somewhere on your system) and then the build scripts should compile
>> the test classes.
>>
>> But I guess you already tried that, after Daniel's reply, and you're
>> still running into problems. I'll take a closer look when you send
>> your detailed mail then.
>>
>> --
>> Johan
>
>
>
> Thanks Johan. Apologies for not sending the detailed mail yet. I wanted to
> test a few specific theories but didn't get that far last night because I
> hit a problem unrelated to SVN.
>
> One thing I'm struggling with is to understand which part of the build
> system is supposed to compile RunTests, i.e., how/where the Java compiler
> is being invoked on it. I'm not seeing it in the build logs, though I do
> see the other Java files being compiled, so that's certainly interesting.
>
I'm fairly sure that RunTests is compiled only when --with-junit is present
(don't know if "present" is enough or if it has to point to a valid JAR
file). Next question is if check-javahl should be available at all if
--with-junit is missing or if check-javahl should make sure RunTests is
built. Either way, I think it is a bug in the build system. This question
should probably be carved out to its own e-mail thread. I'll try to figure
something out and start a new thread.
Kind regards,
Daniel
Re: New release
Posted by Nathan Hartman <ha...@gmail.com>.
On Fri, Nov 10, 2023 at 8:02 AM Johan Corveleyn <jc...@gmail.com> wrote:
> Hi Nathan,
>
> Sorry I haven't helped so far ... I should be able to say something
> useful here, since I'm a java dev in my dayjob. But I have been
> drowning a bit in that same dayjob lately, so I can barely find the
> time to scan these mails, and then forget about them because my
> attention is needed elsewhere.
>
> Anyway, I'll try to find some more time to take a closer look at the
> errors you get.
>
> Just as a first shot: if there is no RunTests.class in the location
> where it should be (one of the directories in the classpath), then it
> probably has not been compiled (or it has not been put in the right
> location). My first guess would also be, like Daniel mentioned, that
> you'll need the --with-junit option (referring to a junit-X.Y.jar
> somewhere on your system) and then the build scripts should compile
> the test classes.
>
> But I guess you already tried that, after Daniel's reply, and you're
> still running into problems. I'll take a closer look when you send
> your detailed mail then.
>
> --
> Johan
Thanks Johan. Apologies for not sending the detailed mail yet. I wanted to
test a few specific theories but didn't get that far last night because I
hit a problem unrelated to SVN.
One thing I'm struggling with is to understand which part of the build
system is supposed to compile RunTests, i.e., how/where the Java compiler
is being invoked on it. I'm not seeing it in the build logs, though I do
see the other Java files being compiled, so that's certainly interesting.
Regarding --with-junit= pointing to a junit-X.Y.jar, yes, this option is
present, though one of the items on my list of things to check is that it's
actually found at that path; and of course I can't check that atm because I
nuked the whole setup. :-)
I'll reply soon with more useful details.
Cheers,
Nathan
Re: New release
Posted by Johan Corveleyn <jc...@gmail.com>.
On Thu, Nov 9, 2023 at 6:08 PM Nathan Hartman <ha...@gmail.com> wrote:
>
> On Fri, Nov 3, 2023 at 12:00 AM Nathan Hartman <ha...@gmail.com> wrote:
> > Previously I mentioned I plan to RM for the upcoming 1.14.3 release.
> > This being my first time, I need to solve some issues first.
> >
> > One of these, which has been a stumbling block for me since the
> > beginning, is getting the JavaHL bindings to build and test
> > successfully.
> >
> > I have made progress on this, but not a complete breakthrough yet.
>
> First, thanks to Stefan, Daniel, and Mark for your input.
>
> I haven't solved my issue with the JavaHL tests yet. I have (mostly)
> written a detailed reply, where I respond to each question and
> suggestion. I'll send it soon, but first, while I was writing it, I
> thought of several specific additional things I want to check. So,
> rather than send a reply full of loose ends, I'll investigate those
> items (probably tonight) and I'll respond then -- wish me luck, and
> hopefully I'll be able to report success!
>
> Nathan
Hi Nathan,
Sorry I haven't helped so far ... I should be able to say something
useful here, since I'm a java dev in my dayjob. But I have been
drowning a bit in that same dayjob lately, so I can barely find the
time to scan these mails, and then forget about them because my
attention is needed elsewhere.
Anyway, I'll try to find some more time to take a closer look at the
errors you get.
Just as a first shot: if there is no RunTests.class in the location
where it should be (one of the directories in the classpath), then it
probably has not been compiled (or it has not been put in the right
location). My first guess would also be, like Daniel mentioned, that
you'll need the --with-junit option (referring to a junit-X.Y.jar
somewhere on your system) and then the build scripts should compile
the test classes.
But I guess you already tried that, after Daniel's reply, and you're
still running into problems. I'll take a closer look when you send
your detailed mail then.
--
Johan
Re: New release
Posted by Nathan Hartman <ha...@gmail.com>.
On Fri, Nov 3, 2023 at 12:00 AM Nathan Hartman <ha...@gmail.com> wrote:
> Previously I mentioned I plan to RM for the upcoming 1.14.3 release.
> This being my first time, I need to solve some issues first.
>
> One of these, which has been a stumbling block for me since the
> beginning, is getting the JavaHL bindings to build and test
> successfully.
>
> I have made progress on this, but not a complete breakthrough yet.
First, thanks to Stefan, Daniel, and Mark for your input.
I haven't solved my issue with the JavaHL tests yet. I have (mostly)
written a detailed reply, where I respond to each question and
suggestion. I'll send it soon, but first, while I was writing it, I
thought of several specific additional things I want to check. So,
rather than send a reply full of loose ends, I'll investigate those
items (probably tonight) and I'll respond then -- wish me luck, and
hopefully I'll be able to report success!
Nathan
Re: New release
Posted by Mark Phippard <ma...@gmail.com>.
On Fri, Nov 3, 2023 at 6:45 AM Daniel Sahlberg
<da...@gmail.com> wrote:
>
> Den fre 3 nov. 2023 kl 10:00 skrev Stefan Sperling <st...@apache.org>:
>>
>> On Fri, Nov 03, 2023 at 12:00:32AM -0400, Nathan Hartman wrote:
>
> [...]
>>
>> >
>> >
>> > Hi all,
>> >
>> > Previously I mentioned I plan to RM for the upcoming 1.14.3 release.
>> > This being my first time, I need to solve some issues first.
>> >
>> > One of these, which has been a stumbling block for me since the
>> > beginning, is getting the JavaHL bindings to build and test
>> > successfully.
>> >
>> > I have made progress on this, but not a complete breakthrough yet.
>> >
>> > The progress was discovering that I need to have $JAVA_HOME set in my
>> > environment; without this, our build system (in build/ac-macros/java.m4)
>> > fails to find javac and assigns 'none' to JAVAC; then, much later,
>> > 'make check-javahl' tries to call this 'none', which unsurprisingly
>> > doesn't work. (I was surprised that the other logic in this AC_DEFUN
>> > couldn't find javac without $JAVA_HOME being set, but that's a subject
>> > for another thread.) The clue was the use of $JAVA_HOME there.
>> >
>> > I tried various values for $JAVA_HOME but nothing that seemed sensible
>> > worked; based on a StackOverflow answer [1], I ended up setting
>> > $JAVA_HOME as follows:
>> >
>> > $ export JAVA_HOME=$(readlink -f /usr/bin/javac | sed "s:/bin/javac::")
>> >
>> > On my system, this sets $JAVA_HOME to:
>> > /usr/lib/jvm/java-17-openjdk-amd64
>
>
> In my case (Ubuntu 23.04), I need to use the --with-jdk argument to configure:
> [[[
> $ ./configure --enable-javahl --with-jdk=/usr/lib/jvm/java-17-openjdk-amd64/
> ]]]
> otherwise I get the following error message:
> [[[
> configure: error: Cannot compile JavaHL without a suitable JDK.
> Please specify a suitable JDK using the --with-jdk option.
> ]]]
> Don't know if&why the build on MacOS is different.
>
>>
>> >
>> > This value seems a little strange IMHO but it finally got 'configure' to
>> > find the JDK successfully and got me past the longstanding 'none'
>> > problem!
>
>
> Great! There are some MacOS specific instructions in the building instructions for JavaHL [2]. Don't know if this refers to your problem or not,
>
>>
>> >
>> > But then I promptly ran into the next issue:
>> >
>> > [[[
>> >
>> > $ make check-javahl
>> > (snip)
>> > /usr/lib/jvm/java-17-openjdk-amd64/bin/java -Xcheck:jni
>> > "-Dtest.rootdir=/home/nathan/ramdrive/svndev/svn-1.14.x/subversion/bindings/javahl/test-work"
>> > "-Dtest.srcdir=/home/nathan/ramdrive/svndev/svn-1.14.x/subversion/bindings/javahl"
>> > "-Dtest.rooturl=" "-Dtest.fstype="
>> > "-Djava.library.path=:/home/nathan/ramdrive/svndev/prefix/svn-1.14.x/lib"
>> > -classpath "subversion/bindings/javahl/classes:/home/nathan/ramdrive/svndev/svn-1.14.x/subversion/bindings/javahl/src:"
>> > "-Dtest.cleanup=" "-Dtest.tests="
>> > org.apache.subversion.javahl.RunTests
>> > Error: Could not find or load main class org.apache.subversion.javahl.RunTests
>> > Caused by: java.lang.ClassNotFoundException:
>> > org.apache.subversion.javahl.RunTests
>> > make: *** [Makefile:530: check-apache-javahl] Error 1
>> >
>> > ]]]
>> >
>> > If I understand correctly, there should be a file called
>> > 'RunTests.class' in one of the directories specified with -classpath
>> > above. Well, there are other .class files to be found there, but not
>> > RunTests.class. A search for RunTests with any (or no) extension in the
>> > 1.14.x tree gives these two RunTests.java (not .class) files:
>> >
>> > $ find . -name RunTests\*
>> > ./subversion/bindings/javahl/tests/org/apache/subversion/javahl/RunTests.java
>> > ./subversion/bindings/javahl/tests/org/tigris/subversion/javahl/RunTests.java
>> >
>> > Forgive me if this is something obvious, but I know almost nothing about
>> > the Java ecosystem. :-)
>
>
> A .class is basically a .o file but for the JVM. So I guess RunTests.java was never compiled.
>
>>
>> >
>> > I'll have to continue this exploration tomorrow, but I'm at least glad
>> > to finally have made progress on a longstanding stumbling block.
>> >
>> > Meanwhile, any clues would be appreciated!
>> >
>> > [1] https://stackoverflow.com/a/29622512
>> >
>> > Cheers,
>> > Nathan
>> >
>>
>> In my build script (in Subversion's tree at tools/dev/unix-build/) I pass the
>> following flags to Subversion's configure script if JavaHL is enabled:
>>
>> --enable-javahl=yes --with-jdk --with-jikes=no \
>> --with-junit=$(DISTDIR)/$(JUNIT_DIST)
>>
>> Did you also pass --with-junit?
>
>
> I did some tests and it seems that the --with-junit=/path/to/junit-X.Y.jar is causing RunTests.java to be built, I can reproduce Nathan's error if I don't add this option to configure.
>
>>
>>
>> For some reason I don't recall, my build script is fetching a self-hosted
>> junit JAR file from here: https://stsp.name/distfiles/junit-4.10.jar
>> (sha256: 36a747ca1e0b86f6ea88055b8723bb87030d627766da6288bf077afdeeb0f75a)
>
>
> The build instructions for javahl [2] say that you should download JUnit from https://junit.org/ ("4.11 has been tested"). I assume you've optimised away the step of going to the website to download the jar (I'm doing something similar for my own builds)..
>
>
>> Given that you have self-compiled svn libraries in a non-standard prefix
>> you might also want to try setting LD_LIBRARY_PATH when invoking the tests.
>> Loading a wrong set of libsvn_*.so files would certainly cause issues, and the
>> ld.so runtime linker probably won't see JVM settings like -Djava.library.path.
>>
>> env LD_LIBRARY_PATH="/home/nathan/ramdrive/svndev/prefix/svn-1.14.x/lib" \
>> make check-all-javahl
>>
The scripts I used are here:
https://github.com/markphip/svnrm/tree/main/scripts
I built on a clean Debian install and the scripts get all the packages
needed before running configure etc.
My recollection of the main complication going forward was what to do
about Python. I needed to use an older Debian base so we could still
build for Python 2.x
Mark
Re: New release
Posted by Daniel Sahlberg <da...@gmail.com>.
Den fre 3 nov. 2023 kl 10:00 skrev Stefan Sperling <st...@apache.org>:
> On Fri, Nov 03, 2023 at 12:00:32AM -0400, Nathan Hartman wrote:
>
[...]
> >
> >
> > Hi all,
> >
> > Previously I mentioned I plan to RM for the upcoming 1.14.3 release.
> > This being my first time, I need to solve some issues first.
> >
> > One of these, which has been a stumbling block for me since the
> > beginning, is getting the JavaHL bindings to build and test
> > successfully.
> >
> > I have made progress on this, but not a complete breakthrough yet.
> >
> > The progress was discovering that I need to have $JAVA_HOME set in my
> > environment; without this, our build system (in build/ac-macros/java.m4)
> > fails to find javac and assigns 'none' to JAVAC; then, much later,
> > 'make check-javahl' tries to call this 'none', which unsurprisingly
> > doesn't work. (I was surprised that the other logic in this AC_DEFUN
> > couldn't find javac without $JAVA_HOME being set, but that's a subject
> > for another thread.) The clue was the use of $JAVA_HOME there.
> >
> > I tried various values for $JAVA_HOME but nothing that seemed sensible
> > worked; based on a StackOverflow answer [1], I ended up setting
> > $JAVA_HOME as follows:
> >
> > $ export JAVA_HOME=$(readlink -f /usr/bin/javac | sed "s:/bin/javac::")
> >
> > On my system, this sets $JAVA_HOME to:
> > /usr/lib/jvm/java-17-openjdk-amd64
>
In my case (Ubuntu 23.04), I need to use the --with-jdk argument to
configure:
[[[
$ ./configure --enable-javahl --with-jdk=/usr/lib/jvm/java-17-openjdk-amd64/
]]]
otherwise I get the following error message:
[[[
configure: error: Cannot compile JavaHL without a suitable JDK.
Please specify a suitable JDK using the --with-jdk option.
]]]
Don't know if&why the build on MacOS is different.
> >
> > This value seems a little strange IMHO but it finally got 'configure' to
> > find the JDK successfully and got me past the longstanding 'none'
> > problem!
>
Great! There are some MacOS specific instructions in the building
instructions for JavaHL [2]. Don't know if this refers to your problem or
not,
> >
> > But then I promptly ran into the next issue:
> >
> > [[[
> >
> > $ make check-javahl
> > (snip)
> > /usr/lib/jvm/java-17-openjdk-amd64/bin/java -Xcheck:jni
> >
> "-Dtest.rootdir=/home/nathan/ramdrive/svndev/svn-1.14.x/subversion/bindings/javahl/test-work"
> >
> "-Dtest.srcdir=/home/nathan/ramdrive/svndev/svn-1.14.x/subversion/bindings/javahl"
> > "-Dtest.rooturl=" "-Dtest.fstype="
> > "-Djava.library.path=:/home/nathan/ramdrive/svndev/prefix/svn-1.14.x/lib"
> > -classpath
> "subversion/bindings/javahl/classes:/home/nathan/ramdrive/svndev/svn-1.14.x/subversion/bindings/javahl/src:"
> > "-Dtest.cleanup=" "-Dtest.tests="
> > org.apache.subversion.javahl.RunTests
> > Error: Could not find or load main class
> org.apache.subversion.javahl.RunTests
> > Caused by: java.lang.ClassNotFoundException:
> > org.apache.subversion.javahl.RunTests
> > make: *** [Makefile:530: check-apache-javahl] Error 1
> >
> > ]]]
> >
> > If I understand correctly, there should be a file called
> > 'RunTests.class' in one of the directories specified with -classpath
> > above. Well, there are other .class files to be found there, but not
> > RunTests.class. A search for RunTests with any (or no) extension in the
> > 1.14.x tree gives these two RunTests.java (not .class) files:
> >
> > $ find . -name RunTests\*
> >
> ./subversion/bindings/javahl/tests/org/apache/subversion/javahl/RunTests.java
> >
> ./subversion/bindings/javahl/tests/org/tigris/subversion/javahl/RunTests.java
> >
> > Forgive me if this is something obvious, but I know almost nothing about
> > the Java ecosystem. :-)
>
A .class is basically a .o file but for the JVM. So I guess RunTests.java
was never compiled.
> >
> > I'll have to continue this exploration tomorrow, but I'm at least glad
> > to finally have made progress on a longstanding stumbling block.
> >
> > Meanwhile, any clues would be appreciated!
> >
> > [1] https://stackoverflow.com/a/29622512
> >
> > Cheers,
> > Nathan
> >
>
> In my build script (in Subversion's tree at tools/dev/unix-build/) I pass
> the
> following flags to Subversion's configure script if JavaHL is enabled:
>
> --enable-javahl=yes --with-jdk --with-jikes=no \
> --with-junit=$(DISTDIR)/$(JUNIT_DIST)
>
> Did you also pass --with-junit?
>
I did some tests and it seems that the --with-junit=/path/to/junit-X.Y.jar
is causing RunTests.java to be built, I can reproduce Nathan's error if I
don't add this option to configure.
>
> For some reason I don't recall, my build script is fetching a self-hosted
> junit JAR file from here: https://stsp.name/distfiles/junit-4.10.jar
> (sha256 <https://stsp.name/distfiles/junit-4.10.jar(sha256>:
> 36a747ca1e0b86f6ea88055b8723bb87030d627766da6288bf077afdeeb0f75a)
>
The build instructions for javahl [2] say that you should download JUnit
from https://junit.org/ ("4.11 has been tested"). I assume you've optimised
away the step of going to the website to download the jar (I'm doing
something similar for my own builds)..
Given that you have self-compiled svn libraries in a non-standard prefix
> you might also want to try setting LD_LIBRARY_PATH when invoking the tests.
> Loading a wrong set of libsvn_*.so files would certainly cause issues, and
> the
> ld.so runtime linker probably won't see JVM settings like
> -Djava.library.path.
>
> env LD_LIBRARY_PATH="/home/nathan/ramdrive/svndev/prefix/svn-1.14.x/lib"
> \
> make check-all-javahl
>
> Hope this helps,
> Stefan
>
Kind regards,
Daniel
[2]
https://svn.apache.org/repos/asf/subversion/trunk/subversion/bindings/javahl/README
Re: New release
Posted by Stefan Sperling <st...@apache.org>.
On Fri, Nov 03, 2023 at 12:00:32AM -0400, Nathan Hartman wrote:
> On Mon, Oct 16, 2023 at 9:16 AM Daniel Sahlberg
> <da...@gmail.com> wrote:
> >
> > Den sön 15 okt. 2023 kl 06:50 skrev Nathan Hartman <ha...@gmail.com>:
> >>
> >> On Fri, Oct 13, 2023 at 1:58 PM Johan Corveleyn <jc...@gmail.com> wrote:
> >>>
> >>> On Fri, Oct 13, 2023 at 5:35 PM Stefan Sperling <st...@apache.org> wrote:
> >>> >
> >>> > On Fri, Oct 13, 2023 at 08:43:59AM +0200, Daniel Sahlberg wrote:
> >>> > > Hi,
> >>> > >
> >>> > > There are quite a number of improvements waiting to be released. Can we
> >>> > > muster the energy to do a new release?
> >>> > >
> >>> > > In trunk there are a lot of changes that warrant a 1.15, but before doing
> >>> > > that I think we should also go back to the discussion of changing the /
> >>> > > adding a checksum algorithm in the WC. That deserves a separate thread and
> >>> > > I'm half way through summarising the previous discussions so I'd like to
> >>> > > hold back 1.15 for the moment.
> >>> > >
> >>> > > Still in 1.14 there have been a number of bugfixes that might be good to
> >>> > > get released. Maybe doing 1.14.3 first could set us up to do 1.15 a little
> >>> > > later?
> >>> >
> >>> > The first problem to solve before the ball starts rolling would be
> >>> > finding a release manager. I don't have enough spare time to play RM
> >>> > this time around but I would support the RM as far as my time allows for.
> >>> >
> >>> > Doing 1.14.3 first sounds like a good plan. This would help potential
> >>> > new releases managers to get bootstrapped into the process. A major
> >>> > release tends to involve a little bit more effort because some new problems
> >>> > may only show up on specific operating system platforms during -rc testing.
> >>>
> >>> +1
> >>>
> >>> --
> >>> Johan
> >>
> >>
> >>
> >> Hi all,
> >>
> >> I agree that a 1.14.3 release makes sense for a first-time RM, to be followed by a 1.15.0 at a later stage. (Also, there are some unresolved issues regarding 1.15.0, such as the above-mentioned question of checksum algorithms, that may require more development, but 1.14.3 should be much more straightforward.)
> >>
> >> At this moment I cannot promise to volunteer for this cycle but I will try to do so.
> >>
> >> Before I do, I need to get my system bootstrapped properly. I can build SVN and run most of the testsuite, but one pesky longstanding issue I have had is with the JavaHL bindings. That will require a separate thread and I'll describe the issues there. I'll check my notes to see what else might be a showstopper on my end.
> >
> >
> > I've successfully built and tested the JavaHL bindings at one point and I think I can dig out my notes to help you if needed.
> >
> >>
> >>
> >> If I can get set up and solve the problems I've encountered previously in a reasonable time frame (and everyone's help is appreciated of course) I'll volunteer.
> >
> >
> > That would be great! I'd volunteer to do the next release after this.
> >
> > There were some Docker scripts floating around (Both from CMike and from Mark). Maybe we should figure out if these can be included in /tools as a help to set up the environment and to execute the necessary steps.
> >
> > Kind regards,
> > Daniel
>
>
> Hi all,
>
> Previously I mentioned I plan to RM for the upcoming 1.14.3 release.
> This being my first time, I need to solve some issues first.
>
> One of these, which has been a stumbling block for me since the
> beginning, is getting the JavaHL bindings to build and test
> successfully.
>
> I have made progress on this, but not a complete breakthrough yet.
>
> The progress was discovering that I need to have $JAVA_HOME set in my
> environment; without this, our build system (in build/ac-macros/java.m4)
> fails to find javac and assigns 'none' to JAVAC; then, much later,
> 'make check-javahl' tries to call this 'none', which unsurprisingly
> doesn't work. (I was surprised that the other logic in this AC_DEFUN
> couldn't find javac without $JAVA_HOME being set, but that's a subject
> for another thread.) The clue was the use of $JAVA_HOME there.
>
> I tried various values for $JAVA_HOME but nothing that seemed sensible
> worked; based on a StackOverflow answer [1], I ended up setting
> $JAVA_HOME as follows:
>
> $ export JAVA_HOME=$(readlink -f /usr/bin/javac | sed "s:/bin/javac::")
>
> On my system, this sets $JAVA_HOME to:
> /usr/lib/jvm/java-17-openjdk-amd64
>
> This value seems a little strange IMHO but it finally got 'configure' to
> find the JDK successfully and got me past the longstanding 'none'
> problem!
>
> But then I promptly ran into the next issue:
>
> [[[
>
> $ make check-javahl
> (snip)
> /usr/lib/jvm/java-17-openjdk-amd64/bin/java -Xcheck:jni
> "-Dtest.rootdir=/home/nathan/ramdrive/svndev/svn-1.14.x/subversion/bindings/javahl/test-work"
> "-Dtest.srcdir=/home/nathan/ramdrive/svndev/svn-1.14.x/subversion/bindings/javahl"
> "-Dtest.rooturl=" "-Dtest.fstype="
> "-Djava.library.path=:/home/nathan/ramdrive/svndev/prefix/svn-1.14.x/lib"
> -classpath "subversion/bindings/javahl/classes:/home/nathan/ramdrive/svndev/svn-1.14.x/subversion/bindings/javahl/src:"
> "-Dtest.cleanup=" "-Dtest.tests="
> org.apache.subversion.javahl.RunTests
> Error: Could not find or load main class org.apache.subversion.javahl.RunTests
> Caused by: java.lang.ClassNotFoundException:
> org.apache.subversion.javahl.RunTests
> make: *** [Makefile:530: check-apache-javahl] Error 1
>
> ]]]
>
> If I understand correctly, there should be a file called
> 'RunTests.class' in one of the directories specified with -classpath
> above. Well, there are other .class files to be found there, but not
> RunTests.class. A search for RunTests with any (or no) extension in the
> 1.14.x tree gives these two RunTests.java (not .class) files:
>
> $ find . -name RunTests\*
> ./subversion/bindings/javahl/tests/org/apache/subversion/javahl/RunTests.java
> ./subversion/bindings/javahl/tests/org/tigris/subversion/javahl/RunTests.java
>
> Forgive me if this is something obvious, but I know almost nothing about
> the Java ecosystem. :-)
>
> I'll have to continue this exploration tomorrow, but I'm at least glad
> to finally have made progress on a longstanding stumbling block.
>
> Meanwhile, any clues would be appreciated!
>
> [1] https://stackoverflow.com/a/29622512
>
> Cheers,
> Nathan
>
In my build script (in Subversion's tree at tools/dev/unix-build/) I pass the
following flags to Subversion's configure script if JavaHL is enabled:
--enable-javahl=yes --with-jdk --with-jikes=no \
--with-junit=$(DISTDIR)/$(JUNIT_DIST)
Did you also pass --with-junit?
For some reason I don't recall, my build script is fetching a self-hosted
junit JAR file from here: https://stsp.name/distfiles/junit-4.10.jar
(sha256: 36a747ca1e0b86f6ea88055b8723bb87030d627766da6288bf077afdeeb0f75a)
Given that you have self-compiled svn libraries in a non-standard prefix
you might also want to try setting LD_LIBRARY_PATH when invoking the tests.
Loading a wrong set of libsvn_*.so files would certainly cause issues, and the
ld.so runtime linker probably won't see JVM settings like -Djava.library.path.
env LD_LIBRARY_PATH="/home/nathan/ramdrive/svndev/prefix/svn-1.14.x/lib" \
make check-all-javahl
Hope this helps,
Stefan