You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@serf.apache.org by Daniel Sahlberg <da...@gmail.com> on 2025/12/11 18:59:41 UTC
Releasing Serf 1.5.0
Hi,
Prompted by Evgeny's suggestion to move forward with releasing Subversion
1.15.0[1], I'd like to suggest getting a new minor release of Serf out of
the door as well - preferrably first!
There are a few features in Subversion that depend on recent development in
Serf (for example the error reporting) and it would be nice to have it
released together.
Brane has made a summary in Jira, see SERF-208[2], open points copied below:
* Generalized error callbacks, discussed in [3]. From what I can see we
have an error callback mechanism that works for SSL errors. There is also
code in Subversion to support this. We need to decide if [a] We are happy
with the current situation, or [b] Can someone step up to improve it.
Personally, I'm leaning towards [a].
* Issue SERF-195, which is a substantial rewrite of the request queues. The
code is in a separate branch SERF-195. The code looks good to me but I
haven't analyzed in detail. It should be possible to merge to trunk.
* Issue SERF-209, concerning intermitent test suite failures under MacOS. I
would suggest to leave this aside for later.
Can we get these decided/merged and roll a release? I think it would be
good to do this in trunk before branching.
Kind regards,
Daniel
[1] https://lists.apache.org/thread/x0s1c8jolql5hdkq40jm3jfnpm6wjp9s
[2] https://issues.apache.org/jira/browse/SERF-208
[3] https://lists.apache.org/thread/7khn697o2srmg8wvqy4t3xyxq4cr8v8v
Re: Releasing Serf 1.3.11 [was: Releasing Serf 1.5.0]
Posted by Branko Čibej <br...@apache.org>.
On 5. 6. 2026 16:01, Branko Čibej wrote:
> On 28. 5. 2026 22:37, Branko Čibej wrote:
>> On 11. 12. 2025 19:59, Daniel Sahlberg wrote:
>>> Hi,
>>>
>>> Prompted by Evgeny's suggestion to move forward with releasing
>>> Subversion
>>> 1.15.0[1], I'd like to suggest getting a new minor release of Serf
>>> out of
>>> the door as well - preferrably first!
>>>
>>> There are a few features in Subversion that depend on recent
>>> development in
>>> Serf (for example the error reporting) and it would be nice to have it
>>> released together.
>>>
>>> Brane has made a summary in Jira, see SERF-208[2], open points
>>> copied below:
>>> * Generalized error callbacks, discussed in [3]. From what I can see we
>>> have an error callback mechanism that works for SSL errors. There is
>>> also
>>> code in Subversion to support this. We need to decide if [a] We are
>>> happy
>>> with the current situation, or [b] Can someone step up to improve it.
>>> Personally, I'm leaning towards [a].
>>> * Issue SERF-195, which is a substantial rewrite of the request
>>> queues. The
>>> code is in a separate branch SERF-195. The code looks good to me but I
>>> haven't analyzed in detail. It should be possible to merge to trunk.
>>> * Issue SERF-209, concerning intermitent test suite failures under
>>> MacOS. I
>>> would suggest to leave this aside for later.
>>>
>>> Can we get these decided/merged and roll a release? I think it would be
>>> good to do this in trunk before branching.
>>>
>>> Kind regards,
>>> Daniel
>>>
>>>
>>>
>>>
>>> [1]https://lists.apache.org/thread/x0s1c8jolql5hdkq40jm3jfnpm6wjp9s
>>> [2]https://issues.apache.org/jira/browse/SERF-208
>>> [3]https://lists.apache.org/thread/7khn697o2srmg8wvqy4t3xyxq4cr8v8v
>>
>>
>> We dropped the ball again :)
>>
>> What is left now? I updated the API compatibility table on the web
>> site some time ago. With, I must say, a lot of fun trying to build
>> all the Serf releases since 1.0.0, but I somehow succeeded anyway.
>> There's nothing interesting there.
>>
>> I'd really like to get this out of the way, if for no other reason
>> than that we can announce that Subversion 1.15 works with Serf 1.5
>> (it does, whether or not we use the experimental feature flag).
>>
>> My proposal: let's do the final API review and roll the release.
>> Then, for Subversion 1.16, we can integrate the better error
>> callbacks and other new features and make them non-experimental, just
>> dependent on the Serf version.
>>
>> And then – as discussed with Bert earlier in this thread – we drop
>> the V1 bucket compatibility and roll Serf-2.0 sooner rather than later.
>>
>> IIRC there was some discussion of also making one last 1.3.x release?
>> Because of some known bug?
>>
>>
>> Just my two groats.
>
>
> On the topic of 1.3.x, this is the current state in STATUS:
>
> * r1875937: Seems obsolete. The branch builds with OpenSSL 1.1.1w and
> 3.x.
> * r1775239-1781240: 1.3.x doesn't build with LibreSSL and never did.
> Seems like a very invasive chang for a patch release, if we decide
> to release 1.3.11. The upcoming 1.5.0 has much better support for
> LibreSSL.
> * r1933327: Supports newer MSVC, should go into 1.3.11.
> * r1935023: Minimal support for OpenSSL 4.0, should go into 1.3.11.
>
>
> TL;DR: I propose we merge r1933327 and r1935023 for the next 1.3
> release and discard the other two entries.
Has anyone successfully run the 1.3.x unit tests recently? 'test_all
context' and 'test_all auth' both hang for me, on Linux and macOS.
Bucket and SSL tests pass. Subversion's davautocheck with USE_SSL=1
works fine with Serf from the 1.3.x branch.
-- Brane
Releasing Serf 1.3.11 [was: Releasing Serf 1.5.0]
Posted by Branko Čibej <br...@apache.org>.
On 28. 5. 2026 22:37, Branko Čibej wrote:
> On 11. 12. 2025 19:59, Daniel Sahlberg wrote:
>> Hi,
>>
>> Prompted by Evgeny's suggestion to move forward with releasing
>> Subversion
>> 1.15.0[1], I'd like to suggest getting a new minor release of Serf
>> out of
>> the door as well - preferrably first!
>>
>> There are a few features in Subversion that depend on recent
>> development in
>> Serf (for example the error reporting) and it would be nice to have it
>> released together.
>>
>> Brane has made a summary in Jira, see SERF-208[2], open points copied
>> below:
>> * Generalized error callbacks, discussed in [3]. From what I can see we
>> have an error callback mechanism that works for SSL errors. There is
>> also
>> code in Subversion to support this. We need to decide if [a] We are
>> happy
>> with the current situation, or [b] Can someone step up to improve it.
>> Personally, I'm leaning towards [a].
>> * Issue SERF-195, which is a substantial rewrite of the request
>> queues. The
>> code is in a separate branch SERF-195. The code looks good to me but I
>> haven't analyzed in detail. It should be possible to merge to trunk.
>> * Issue SERF-209, concerning intermitent test suite failures under
>> MacOS. I
>> would suggest to leave this aside for later.
>>
>> Can we get these decided/merged and roll a release? I think it would be
>> good to do this in trunk before branching.
>>
>> Kind regards,
>> Daniel
>>
>>
>>
>>
>> [1]https://lists.apache.org/thread/x0s1c8jolql5hdkq40jm3jfnpm6wjp9s
>> [2]https://issues.apache.org/jira/browse/SERF-208
>> [3]https://lists.apache.org/thread/7khn697o2srmg8wvqy4t3xyxq4cr8v8v
>
>
> We dropped the ball again :)
>
> What is left now? I updated the API compatibility table on the web
> site some time ago. With, I must say, a lot of fun trying to build all
> the Serf releases since 1.0.0, but I somehow succeeded anyway. There's
> nothing interesting there.
>
> I'd really like to get this out of the way, if for no other reason
> than that we can announce that Subversion 1.15 works with Serf 1.5 (it
> does, whether or not we use the experimental feature flag).
>
> My proposal: let's do the final API review and roll the release. Then,
> for Subversion 1.16, we can integrate the better error callbacks and
> other new features and make them non-experimental, just dependent on
> the Serf version.
>
> And then – as discussed with Bert earlier in this thread – we drop the
> V1 bucket compatibility and roll Serf-2.0 sooner rather than later.
>
> IIRC there was some discussion of also making one last 1.3.x release?
> Because of some known bug?
>
>
> Just my two groats.
On the topic of 1.3.x, this is the current state in STATUS:
* r1875937: Seems obsolete. The branch builds with OpenSSL 1.1.1w and 3.x.
* r1775239-1781240: 1.3.x doesn't build with LibreSSL and never did.
Seems like a very invasive chang for a patch release, if we decide
to release 1.3.11. The upcoming 1.5.0 has much better support for
LibreSSL.
* r1933327: Supports newer MSVC, should go into 1.3.11.
* r1935023: Minimal support for OpenSSL 4.0, should go into 1.3.11.
TL;DR: I propose we merge r1933327 and r1935023 for the next 1.3 release
and discard the other two entries.
-- Brane
Re: Releasing Serf 1.5.0
Posted by Branko Čibej <br...@apache.org>.
On 28. 5. 2026 22:37, Branko Čibej wrote:
> On 11. 12. 2025 19:59, Daniel Sahlberg wrote:
>> Hi,
>>
>> Prompted by Evgeny's suggestion to move forward with releasing
>> Subversion
>> 1.15.0[1], I'd like to suggest getting a new minor release of Serf
>> out of
>> the door as well - preferrably first!
>>
>> There are a few features in Subversion that depend on recent
>> development in
>> Serf (for example the error reporting) and it would be nice to have it
>> released together.
>>
>> Brane has made a summary in Jira, see SERF-208[2], open points copied
>> below:
>> * Generalized error callbacks, discussed in [3]. From what I can see we
>> have an error callback mechanism that works for SSL errors. There is
>> also
>> code in Subversion to support this. We need to decide if [a] We are
>> happy
>> with the current situation, or [b] Can someone step up to improve it.
>> Personally, I'm leaning towards [a].
>> * Issue SERF-195, which is a substantial rewrite of the request
>> queues. The
>> code is in a separate branch SERF-195. The code looks good to me but I
>> haven't analyzed in detail. It should be possible to merge to trunk.
>> * Issue SERF-209, concerning intermitent test suite failures under
>> MacOS. I
>> would suggest to leave this aside for later.
>>
>> Can we get these decided/merged and roll a release? I think it would be
>> good to do this in trunk before branching.
>>
>> Kind regards,
>> Daniel
>>
>>
>>
>>
>> [1]https://lists.apache.org/thread/x0s1c8jolql5hdkq40jm3jfnpm6wjp9s
>> [2]https://issues.apache.org/jira/browse/SERF-208
>> [3]https://lists.apache.org/thread/7khn697o2srmg8wvqy4t3xyxq4cr8v8v
>
>
> We dropped the ball again :)
>
> What is left now? I updated the API compatibility table on the web
> site some time ago. With, I must say, a lot of fun trying to build all
> the Serf releases since 1.0.0, but I somehow succeeded anyway. There's
> nothing interesting there.
I meant 0.1.0. I had to build OpenSSL 0.9 for that one, and I had a hard
time downloading the sources for that. Somehow the Internet these days
believes I shouldn't even see them, let alone build them, heh.
-- Brane
Re: Releasing Serf 1.5.0
Posted by Branko Čibej <br...@apache.org>.
On 11. 12. 2025 19:59, Daniel Sahlberg wrote:
> Hi,
>
> Prompted by Evgeny's suggestion to move forward with releasing Subversion
> 1.15.0[1], I'd like to suggest getting a new minor release of Serf out of
> the door as well - preferrably first!
>
> There are a few features in Subversion that depend on recent development in
> Serf (for example the error reporting) and it would be nice to have it
> released together.
>
> Brane has made a summary in Jira, see SERF-208[2], open points copied below:
> * Generalized error callbacks, discussed in [3]. From what I can see we
> have an error callback mechanism that works for SSL errors. There is also
> code in Subversion to support this. We need to decide if [a] We are happy
> with the current situation, or [b] Can someone step up to improve it.
> Personally, I'm leaning towards [a].
> * Issue SERF-195, which is a substantial rewrite of the request queues. The
> code is in a separate branch SERF-195. The code looks good to me but I
> haven't analyzed in detail. It should be possible to merge to trunk.
> * Issue SERF-209, concerning intermitent test suite failures under MacOS. I
> would suggest to leave this aside for later.
>
> Can we get these decided/merged and roll a release? I think it would be
> good to do this in trunk before branching.
>
> Kind regards,
> Daniel
>
>
>
>
> [1]https://lists.apache.org/thread/x0s1c8jolql5hdkq40jm3jfnpm6wjp9s
> [2]https://issues.apache.org/jira/browse/SERF-208
> [3]https://lists.apache.org/thread/7khn697o2srmg8wvqy4t3xyxq4cr8v8v
We dropped the ball again :)
What is left now? I updated the API compatibility table on the web site
some time ago. With, I must say, a lot of fun trying to build all the
Serf releases since 1.0.0, but I somehow succeeded anyway. There's
nothing interesting there.
I'd really like to get this out of the way, if for no other reason than
that we can announce that Subversion 1.15 works with Serf 1.5 (it does,
whether or not we use the experimental feature flag).
My proposal: let's do the final API review and roll the release. Then,
for Subversion 1.16, we can integrate the better error callbacks and
other new features and make them non-experimental, just dependent on the
Serf version.
And then – as discussed with Bert earlier in this thread – we drop the
V1 bucket compatibility and roll Serf-2.0 sooner rather than later.
IIRC there was some discussion of also making one last 1.3.x release?
Because of some known bug?
Just my two groats.
-- Brane
Re: Releasing Serf 1.5.0
Posted by Branko Čibej <br...@apache.org>.
On 4. 1. 26 18:02, Branko Čibej wrote:
> On 4. 1. 26 17:40, Daniel Sahlberg wrote:
[...]
>> Comment: N/A for Serf? We should probably still have such discussion
>> after
>> reviewing the new APIs (below).
>
> The "current stable branch" in our case is 1.3.x. Doesn't seem to be
> anything there. Re LibreSSL, I've tested Serf builds on OpenBSD with
> LibreSSL. Some SSL tests of course fail because of different error
> reporting. We have special-caseing there for different OpenSSL
> versions, we could add similar for LibreSSL – not too onerous since it
> advertises as OpenSSL 2.x.
>
> On that note, Fedora/Centos/RHEL have patched OpenSSL 3.x that has
> stricter constraints, causing some of our SSL tests fail, too.
>
> We don't really have to delay 1.5 for this as long as we review the
> failures and decide they're cosmetic only. Those kinds of fixes can be
> backported later.
>
> I can take a look at the state of that again. I should be able to
> build with LibreSSL on my mac, too.
It was a good thing that I tried building with older OpenSSL and with
LibreSSL, I found some nits in the code that way; see r1931107.
Long story short: Our SSL tests pass with no comments with OpenSSL
1.1.1w, the last release from that stream. With LibreSSL 4.2.1, we get
11 failures and all of them are because of different expected output
(errors) emitted by LibreSSL.
IIRC, with OpenSSL 3.2 on Fedora 43, we get 4 such failures.
IMO let's leave the cosmetics for trunk and 1.5.x backports.
However, I would like to gather all the Cert/URI related conditional
code to one place; right now it's scattered all over ssl_buckets.c and
there's no good reason for that. It's one of the for r1931107.
-- Brane
Re: Releasing Serf 1.5.0
Posted by Branko Čibej <br...@apache.org>.
On 4. 1. 26 17:40, Daniel Sahlberg wrote:
> Den sön 4 jan. 2026 kl 10:07 skrev Branko Čibej<br...@apache.org>:
>
>> On 4. 1. 26 09:18, Daniel Sahlberg wrote:
>>> In the interest of readability, I'm replying to an old e-mail here to
>>> summarise the discussion else-thread.
>>>
>>> Den tors 11 dec. 2025 kl 20:43 skrev Branko Čibej<br...@apache.org>:
>>>
>>>> On 11. 12. 25 19:59, Daniel Sahlberg wrote:
>>>>> Hi,
>>>>>
>>>>> Prompted by Evgeny's suggestion to move forward with releasing
>> Subversion
>>>>> 1.15.0[1], I'd like to suggest getting a new minor release of Serf out
>> of
>>>>> the door as well - preferrably first!
>>>>>
>>>>> There are a few features in Subversion that depend on recent
>> development
>>>> in
>>>>> Serf (for example the error reporting) and it would be nice to have it
>>>>> released together.
>>>>>
>>>>> Brane has made a summary in Jira, see SERF-208[2], open points copied
>>>> below:
>>>>> * Generalized error callbacks, discussed in [3]. From what I can see we
>>>>> have an error callback mechanism that works for SSL errors. There is
>> also
>>>>> code in Subversion to support this. We need to decide if [a] We are
>> happy
>>>>> with the current situation, or [b] Can someone step up to improve it.
>>>>> Personally, I'm leaning towards [a].
>>>> I'm not happy with the current custom callback just for SSL errors. I'd
>>>> be -0.9 to release it as a public API. I sort of dropped the ball .
>>>>
>>> Work is ongoing, with some committed to the repository and some still in
>>> progress (including the earlier mail from Brane with a patch).
>>>
>>> It seems we should be able to get this done not too far into the future.
>>>
>>>
>>>>> * Issue SERF-195, which is a substantial rewrite of the request queues.
>>>> The
>>>>> code is in a separate branch SERF-195. The code looks good to me but I
>>>>> haven't analyzed in detail. It should be possible to merge to trunk.
>>>> The patch works, in the sense that it causes no regressions, but I
>>>> haven't been able to reproduce the reported issue. It's also makes the
>>>> code slightly more readable, but I couldn't find the mentioned race
>>>> condition during review, it looks like just a refactoring of the
>>>> original code.
>>>>
>>>> I think we need another pair of eyes to do a careful review of the
>>>> branch changes.
>>>>
>>> I have removed SERF-195 from the checklist as per consensus else-thread.
>>>
>>>
>>>>> * Issue SERF-209, concerning intermitent test suite failures under
>>>> MacOS. I
>>>>> would suggest to leave this aside for later.
>>>> Yes, it's not critical. Might be something in the test code itself,
>>>> i.e., not related to serf proper.
>>>>> Can we get these decided/merged and roll a release? I think it would be
>>>>> good to do this in trunk before branching.
>>> FWIW: I've just rebuilt TortoiseSVN based on Serf trunk and the 1.15.x
>>> branch of Subversion (with SVN__SERF_EXPERIMENTAL and
>> SVN__SERF_TEST_HTTP2
>>> to let Subversion use some new Serf features. Build went fine and some
>>> initial tests (checkout, commit) also worked as expected. I'll use this
>> on
>>> my work laptop for a few days to test out
>>>
>>> Cheers,
>>> Daniel
>>
>> We should also replace all the "@since New in 1.4" with 1.5, to avoid
>> confusion?
>>
>> -- Brane
>>
> Yes, that is probably a good idea. Do you mind if I do it?
Sure. It's SERF-210 now.
> I'm planning to update Graham's Subversion patch, utilising the error
> callbacks, to the new callback structure to see how it would work out in
> practice. That should go on another mailing list though.
>
> If that works out, then I think we should get going with a release.
> Borrowing the checklist Evgeny posted on Subversion 1.15:
>
> 🗹 Review the Roadmap. Should any of the outstanding items be addressed
> before branching?
>
> Comment: Discussed, in SERF-208 and in the mailing list. In short, two
> issues were deferred to trunk/1.6 and all other closed except the error
> callbacks.
>
> ☐ Review the incumbent stable branch's STATUS file for outstanding -0 and
> -1 votes. Was code that is going to be included in the new branch objected
> to?
>
> Comment: N/A for Serf? We should probably still have such discussion after
> reviewing the new APIs (below).
The "current stable branch" in our case is 1.3.x. Doesn't seem to be
anything there. Re LibreSSL, I've tested Serf builds on OpenBSD with
LibreSSL. Some SSL tests of course fail because of different error
reporting. We have special-caseing there for different OpenSSL versions,
we could add similar for LibreSSL – not too onerous since it advertises
as OpenSSL 2.x.
On that note, Fedora/Centos/RHEL have patched OpenSSL 3.x that has
stricter constraints, causing some of our SSL tests fail, too.
We don't really have to delay 1.5 for this as long as we review the
failures and decide they're cosmetic only. Those kinds of fixes can be
backported later.
I can take a look at the state of that again. I should be able to build
with LibreSSL on my mac, too.
> ☐ Do any tree-wide, mechanical changes.
>
> Comment: I have not reviewed if there are any such changes for Serf, but I
> assume there are at least some version #defines after branching.
SERF_MAJOR/MINOR_VERSION in serf.h and the @since 1.4 bits.
Copyright years in the file headers and the web site (which is still
2023; I didn't change that just because of the logo/font update).
> ☐ Review new and changed APIs for design and style (e.g. Doxygen mark-up,
> undocumented parameters, deprecation, public/private status, etc.).
>
> Comment: ... todo ...
Yes, our Doxygen markup isn't the best. As long as we have it for new
functions that appeared between 1.3 and trunk, I think we can live with
that.
> ☐ Run compatibility tests against the incumbent minor release.
>
> Comment: N/A?
That would be something along the lines of running the 1.3 branch tests
with a 1.5 libserf, as far as it's feasible. Internal-test should be
skipped, but the rest should work.
Hm, that's a pain because the tests link with the static lib, oh well,
I'll give it a go.
Also running the Subversion test suite against a https:// mod-dav-svn
with trunk serf, with or without SVN__SERF_EXPERIMENTAL.
-- Brane
Re: Releasing Serf 1.5.0
Posted by Daniel Sahlberg <da...@gmail.com>.
Den sön 4 jan. 2026 kl 10:07 skrev Branko Čibej <br...@apache.org>:
> On 4. 1. 26 09:18, Daniel Sahlberg wrote:
> > In the interest of readability, I'm replying to an old e-mail here to
> > summarise the discussion else-thread.
> >
> > Den tors 11 dec. 2025 kl 20:43 skrev Branko Čibej<br...@apache.org>:
> >
> >> On 11. 12. 25 19:59, Daniel Sahlberg wrote:
> >>> Hi,
> >>>
> >>> Prompted by Evgeny's suggestion to move forward with releasing
> Subversion
> >>> 1.15.0[1], I'd like to suggest getting a new minor release of Serf out
> of
> >>> the door as well - preferrably first!
> >>>
> >>> There are a few features in Subversion that depend on recent
> development
> >> in
> >>> Serf (for example the error reporting) and it would be nice to have it
> >>> released together.
> >>>
> >>> Brane has made a summary in Jira, see SERF-208[2], open points copied
> >> below:
> >>> * Generalized error callbacks, discussed in [3]. From what I can see we
> >>> have an error callback mechanism that works for SSL errors. There is
> also
> >>> code in Subversion to support this. We need to decide if [a] We are
> happy
> >>> with the current situation, or [b] Can someone step up to improve it.
> >>> Personally, I'm leaning towards [a].
> >> I'm not happy with the current custom callback just for SSL errors. I'd
> >> be -0.9 to release it as a public API. I sort of dropped the ball .
> >>
> > Work is ongoing, with some committed to the repository and some still in
> > progress (including the earlier mail from Brane with a patch).
> >
> > It seems we should be able to get this done not too far into the future.
> >
> >
> >>> * Issue SERF-195, which is a substantial rewrite of the request queues.
> >> The
> >>> code is in a separate branch SERF-195. The code looks good to me but I
> >>> haven't analyzed in detail. It should be possible to merge to trunk.
> >> The patch works, in the sense that it causes no regressions, but I
> >> haven't been able to reproduce the reported issue. It's also makes the
> >> code slightly more readable, but I couldn't find the mentioned race
> >> condition during review, it looks like just a refactoring of the
> >> original code.
> >>
> >> I think we need another pair of eyes to do a careful review of the
> >> branch changes.
> >>
> > I have removed SERF-195 from the checklist as per consensus else-thread.
> >
> >
> >>> * Issue SERF-209, concerning intermitent test suite failures under
> >> MacOS. I
> >>> would suggest to leave this aside for later.
> >> Yes, it's not critical. Might be something in the test code itself,
> >> i.e., not related to serf proper.
> >
> >>> Can we get these decided/merged and roll a release? I think it would be
> >>> good to do this in trunk before branching.
> > FWIW: I've just rebuilt TortoiseSVN based on Serf trunk and the 1.15.x
> > branch of Subversion (with SVN__SERF_EXPERIMENTAL and
> SVN__SERF_TEST_HTTP2
> > to let Subversion use some new Serf features. Build went fine and some
> > initial tests (checkout, commit) also worked as expected. I'll use this
> on
> > my work laptop for a few days to test out
> >
> > Cheers,
> > Daniel
>
>
> We should also replace all the "@since New in 1.4" with 1.5, to avoid
> confusion?
>
> -- Brane
>
Yes, that is probably a good idea. Do you mind if I do it?
I'm planning to update Graham's Subversion patch, utilising the error
callbacks, to the new callback structure to see how it would work out in
practice. That should go on another mailing list though.
If that works out, then I think we should get going with a release.
Borrowing the checklist Evgeny posted on Subversion 1.15:
🗹 Review the Roadmap. Should any of the outstanding items be addressed
before branching?
Comment: Discussed, in SERF-208 and in the mailing list. In short, two
issues were deferred to trunk/1.6 and all other closed except the error
callbacks.
☐ Review the incumbent stable branch's STATUS file for outstanding -0 and
-1 votes. Was code that is going to be included in the new branch objected
to?
Comment: N/A for Serf? We should probably still have such discussion after
reviewing the new APIs (below).
☐ Do any tree-wide, mechanical changes.
Comment: I have not reviewed if there are any such changes for Serf, but I
assume there are at least some version #defines after branching.
☐ Review new and changed APIs for design and style (e.g. Doxygen mark-up,
undocumented parameters, deprecation, public/private status, etc.).
Comment: ... todo ...
☐ Run compatibility tests against the incumbent minor release.
Comment: N/A?
Cheers,
Daniel
Re: Releasing Serf 1.5.0
Posted by Branko Čibej <br...@apache.org>.
On 4. 1. 26 09:18, Daniel Sahlberg wrote:
> In the interest of readability, I'm replying to an old e-mail here to
> summarise the discussion else-thread.
>
> Den tors 11 dec. 2025 kl 20:43 skrev Branko Čibej<br...@apache.org>:
>
>> On 11. 12. 25 19:59, Daniel Sahlberg wrote:
>>> Hi,
>>>
>>> Prompted by Evgeny's suggestion to move forward with releasing Subversion
>>> 1.15.0[1], I'd like to suggest getting a new minor release of Serf out of
>>> the door as well - preferrably first!
>>>
>>> There are a few features in Subversion that depend on recent development
>> in
>>> Serf (for example the error reporting) and it would be nice to have it
>>> released together.
>>>
>>> Brane has made a summary in Jira, see SERF-208[2], open points copied
>> below:
>>> * Generalized error callbacks, discussed in [3]. From what I can see we
>>> have an error callback mechanism that works for SSL errors. There is also
>>> code in Subversion to support this. We need to decide if [a] We are happy
>>> with the current situation, or [b] Can someone step up to improve it.
>>> Personally, I'm leaning towards [a].
>> I'm not happy with the current custom callback just for SSL errors. I'd
>> be -0.9 to release it as a public API. I sort of dropped the ball .
>>
> Work is ongoing, with some committed to the repository and some still in
> progress (including the earlier mail from Brane with a patch).
>
> It seems we should be able to get this done not too far into the future.
>
>
>>> * Issue SERF-195, which is a substantial rewrite of the request queues.
>> The
>>> code is in a separate branch SERF-195. The code looks good to me but I
>>> haven't analyzed in detail. It should be possible to merge to trunk.
>> The patch works, in the sense that it causes no regressions, but I
>> haven't been able to reproduce the reported issue. It's also makes the
>> code slightly more readable, but I couldn't find the mentioned race
>> condition during review, it looks like just a refactoring of the
>> original code.
>>
>> I think we need another pair of eyes to do a careful review of the
>> branch changes.
>>
> I have removed SERF-195 from the checklist as per consensus else-thread.
>
>
>>> * Issue SERF-209, concerning intermitent test suite failures under
>> MacOS. I
>>> would suggest to leave this aside for later.
>> Yes, it's not critical. Might be something in the test code itself,
>> i.e., not related to serf proper.
>
>>> Can we get these decided/merged and roll a release? I think it would be
>>> good to do this in trunk before branching.
> FWIW: I've just rebuilt TortoiseSVN based on Serf trunk and the 1.15.x
> branch of Subversion (with SVN__SERF_EXPERIMENTAL and SVN__SERF_TEST_HTTP2
> to let Subversion use some new Serf features. Build went fine and some
> initial tests (checkout, commit) also worked as expected. I'll use this on
> my work laptop for a few days to test out
>
> Cheers,
> Daniel
We should also replace all the "@since New in 1.4" with 1.5, to avoid
confusion?
-- Brane
Re: Releasing Serf 1.5.0
Posted by Daniel Sahlberg <da...@gmail.com>.
In the interest of readability, I'm replying to an old e-mail here to
summarise the discussion else-thread.
Den tors 11 dec. 2025 kl 20:43 skrev Branko Čibej <br...@apache.org>:
> On 11. 12. 25 19:59, Daniel Sahlberg wrote:
> > Hi,
> >
> > Prompted by Evgeny's suggestion to move forward with releasing Subversion
> > 1.15.0[1], I'd like to suggest getting a new minor release of Serf out of
> > the door as well - preferrably first!
> >
> > There are a few features in Subversion that depend on recent development
> in
> > Serf (for example the error reporting) and it would be nice to have it
> > released together.
> >
> > Brane has made a summary in Jira, see SERF-208[2], open points copied
> below:
> > * Generalized error callbacks, discussed in [3]. From what I can see we
> > have an error callback mechanism that works for SSL errors. There is also
> > code in Subversion to support this. We need to decide if [a] We are happy
> > with the current situation, or [b] Can someone step up to improve it.
> > Personally, I'm leaning towards [a].
>
> I'm not happy with the current custom callback just for SSL errors. I'd
> be -0.9 to release it as a public API. I sort of dropped the ball .
>
Work is ongoing, with some committed to the repository and some still in
progress (including the earlier mail from Brane with a patch).
It seems we should be able to get this done not too far into the future.
>
> > * Issue SERF-195, which is a substantial rewrite of the request queues.
> The
> > code is in a separate branch SERF-195. The code looks good to me but I
> > haven't analyzed in detail. It should be possible to merge to trunk.
>
> The patch works, in the sense that it causes no regressions, but I
> haven't been able to reproduce the reported issue. It's also makes the
> code slightly more readable, but I couldn't find the mentioned race
> condition during review, it looks like just a refactoring of the
> original code.
>
> I think we need another pair of eyes to do a careful review of the
> branch changes.
>
I have removed SERF-195 from the checklist as per consensus else-thread.
>
> > * Issue SERF-209, concerning intermitent test suite failures under
> MacOS. I
> > would suggest to leave this aside for later.
>
> Yes, it's not critical. Might be something in the test code itself,
> i.e., not related to serf proper.
> > Can we get these decided/merged and roll a release? I think it would be
> > good to do this in trunk before branching.
>
FWIW: I've just rebuilt TortoiseSVN based on Serf trunk and the 1.15.x
branch of Subversion (with SVN__SERF_EXPERIMENTAL and SVN__SERF_TEST_HTTP2
to let Subversion use some new Serf features. Build went fine and some
initial tests (checkout, commit) also worked as expected. I'll use this on
my work laptop for a few days to test out
Cheers,
Daniel
> >
> > Kind regards,
> > Daniel
> >
> >
> >
> >
> > [1]https://lists.apache.org/thread/x0s1c8jolql5hdkq40jm3jfnpm6wjp9s
> > [2]https://issues.apache.org/jira/browse/SERF-208
> > [3]https://lists.apache.org/thread/7khn697o2srmg8wvqy4t3xyxq4cr8v8v
> >
>
Re: Error callbacks [was: Releasing Serf 1.5.0]
Posted by Daniel Sahlberg <da...@gmail.com>.
Den tors 1 jan. 2026 kl 15:40 skrev Branko Čibej <br...@apache.org>:
> On 12. 12. 25 23:48, Branko Čibej wrote:
> > On 12. 12. 25 20:49, Branko Čibej wrote:
> >>
> >> The problem is that the error callback is global, not connection- or
> >> at least context-specific. That simply doesn't scale.
> >
> >
> > In addition to not scaling, it's also a headache for Serf's threading
> > model. Serf functions are always called within a single thread
> > associated with serf_context_t, but the application can be
> > multi-threaded and have multiple contexts active in parallel. The
> > error callbacks will have no idea which thread it was called from.
> >
> > Serf has very little global mutable data, I think only the recent
> > changes for registering authentication methods introduce global locks.
> > More locking == more opportunities for subtle errors.
>
>
> I'm implementing error callbacks and found an "interesting"
> circumstance. In ssl_buckets.c, where there was a ssl_context-specific
> error callback, some error conditions were reported *only* if the error
> callback was defined. Basically, not defining an error callback would
> mean that some failures weren't noticed and things would just go
> swimmingly on as if nothing happened.
>
> I found this when I moved to a different callback model and an SSL test
> started failing ... it turns out that the test has a bug and previously
> "succeeded" only because actual errors were not being reported. What fun.
> :D
>
> The core bug is of course not properly returning errors if some optional
> callback wasn't defined. I've fixed this with the new callback code and
> I'm now looking at how to fix the test, it's basically a test setup bug.
>
> I want to make this a one-commit thing so that we can review the new
> callback design with actual example code, and it's easy to revert if
> necessary. If everyone's happy with how I decided to go about this,
> there'll be more changes to come to get reporting from the ssl_context
> hammered out and some tests added.
>
> Just a few more minutes hours days now...
>
> -- Brane
That's great news! Looking forward to see some code!
Cheers,
Daniel
Re: Error callbacks
Posted by Branko Čibej <br...@apache.org>.
On 1. 1. 26 15:40, Branko Čibej wrote:
>
> I found this when I moved to a different callback model and an SSL
> test started failing ... it turns out that the test has a bug and
> previously "succeeded" only because actual errors were not being
> reported. What fun. :D
>
> The core bug is of course not properly returning errors if some
> optional callback wasn't defined. I've fixed this with the new
> callback code and I'm now looking at how to fix the test, it's
> basically a test setup bug.
It turned out not to be a test setup bug but an old-fashioned code
off-by-one bug; r1931046.
-- Brane
Error callbacks [was: Releasing Serf 1.5.0]
Posted by Branko Čibej <br...@apache.org>.
On 12. 12. 25 23:48, Branko Čibej wrote:
> On 12. 12. 25 20:49, Branko Čibej wrote:
>>
>> The problem is that the error callback is global, not connection- or
>> at least context-specific. That simply doesn't scale.
>
>
> In addition to not scaling, it's also a headache for Serf's threading
> model. Serf functions are always called within a single thread
> associated with serf_context_t, but the application can be
> multi-threaded and have multiple contexts active in parallel. The
> error callbacks will have no idea which thread it was called from.
>
> Serf has very little global mutable data, I think only the recent
> changes for registering authentication methods introduce global locks.
> More locking == more opportunities for subtle errors.
I'm implementing error callbacks and found an "interesting"
circumstance. In ssl_buckets.c, where there was a ssl_context-specific
error callback, some error conditions were reported *only* if the error
callback was defined. Basically, not defining an error callback would
mean that some failures weren't noticed and things would just go
swimmingly on as if nothing happened.
I found this when I moved to a different callback model and an SSL test
started failing ... it turns out that the test has a bug and previously
"succeeded" only because actual errors were not being reported. What fun. :D
The core bug is of course not properly returning errors if some optional
callback wasn't defined. I've fixed this with the new callback code and
I'm now looking at how to fix the test, it's basically a test setup bug.
I want to make this a one-commit thing so that we can review the new
callback design with actual example code, and it's easy to revert if
necessary. If everyone's happy with how I decided to go about this,
there'll be more changes to come to get reporting from the ssl_context
hammered out and some tests added.
Just a few more minutes hours days now...
-- Brane
Re: Releasing Serf 1.5.0
Posted by Branko Čibej <br...@apache.org>.
On 12. 12. 25 20:49, Branko Čibej wrote:
>
> The problem is that the error callback is global, not connection- or
> at least context-specific. That simply doesn't scale.
In addition to not scaling, it's also a headache for Serf's threading
model. Serf functions are always called within a single thread
associated with serf_context_t, but the application can be
multi-threaded and have multiple contexts active in parallel. The error
callbacks will have no idea which thread it was called from.
Serf has very little global mutable data, I think only the recent
changes for registering authentication methods introduce global locks.
More locking == more opportunities for subtle errors.
-- Brane
Re: Reproducing SERF-195 [was: Releasing Serf 1.5.0]
Posted by Branko Čibej <br...@apache.org>.
On 1. 1. 26 12:59, Daniel Sahlberg wrote:
> Den mån 15 dec. 2025 kl 23:59 skrev Daniel Sahlberg <
> daniel.l.sahlberg@gmail.com>:
>
>> After sending, I read through the issue description and it mentions
>> reproducing with Subversion. So I will make one last try building Serf from
>> trunk with the usleep() call and see if something gives.
>>
> It took a while to find the time to look at this but I've now built Serf
> and Subversion from trunk with the added usleep() (plus some randomness
> just for fun - the random factor should be between 0.5 and 1.5) and
> without/with the SERF-195 patch. I then tried to check out
> https://svn.apache.org/repos/private/foundation (which require basic auth).
>
> With a delay between 37.5 ms and 112.5 ms (=average around the 75'000 us in
> the bug report) the checkout succeed.
>
> With a delay between 375 ms and 1125 ms (=10 times more then in the bug
> report), the checkout fails as follows:
>
> [[[
> A foundation/officers/workbench
> A foundation/board
> X random: 377231.422929 in conn 0x7f23e0b92028
> X random: 967579.765638 in conn 0x7f23e0b92028
> X random: 590046.259093 in conn 0x7f23e0b92028
> X random: 849541.822669 in conn 0x7f23e0b92028
> X random: 889138.332808 in conn 0x7f23e0b92028
> X random: 866866.873573 in conn 0x7f23e0b92028
> X random: 491075.777503 in conn 0x7f23e0b92028
> X random: 616534.923083 in conn 0x7f23e0b92028
> X random: 785294.915508 in conn 0x7f23e0b92028
> X random: 1103729.169573 in conn 0x7f23e0b92028
> X random: 638844.156411 in conn 0x7f23e0b92028
> X random: 383173.109269 in conn 0x7f23e0d12430
> X random: 707753.396282 in conn 0x7f23e0d12430
> subversion/svn/checkout-cmd.c:168,
> subversion/libsvn_client/checkout.c:318,
> subversion/libsvn_client/checkout.c:274,
> subversion/libsvn_client/update.c:722,
> subversion/libsvn_client/update.c:563,
> subversion/libsvn_wc/adm_crawler.c:868,
> subversion/libsvn_ra_serf/update.c:2518,
> subversion/libsvn_ra_serf/update.c:2508,
> subversion/libsvn_ra_serf/update.c:2439,
> subversion/libsvn_ra_serf/util.c:966: (apr_err=APR_EOF)
> svn: E070014: Error running context: End of file found
> ]]]
>
> Checking with Wireshark it seems the server is closing the connection on us.
>
> BUT: It isn't the memory corruption issue mentioned and more importantly it
> fails exactly the same way both with trunk (no patches, except for the
> usleep) and with the SERF-195 branch.
>
> So I'm leaning towards closing SERF-195 as "not reproducible" but I agree
> with Brane that the refactoring of linked-list code to dedicated functions
> make it easier to review and make sure all cases manage the linked lists
> correctly. So we might keep the branch and merge at a later date.
>
> Thoughts?
+1
Let's merge the branch to trunk after the 1.5 release.
-- Brane
Re: Reproducing SERF-195 [was: Releasing Serf 1.5.0]
Posted by Daniel Sahlberg <da...@gmail.com>.
Den mån 15 dec. 2025 kl 23:59 skrev Daniel Sahlberg <
daniel.l.sahlberg@gmail.com>:
>
> After sending, I read through the issue description and it mentions
> reproducing with Subversion. So I will make one last try building Serf from
> trunk with the usleep() call and see if something gives.
>
It took a while to find the time to look at this but I've now built Serf
and Subversion from trunk with the added usleep() (plus some randomness
just for fun - the random factor should be between 0.5 and 1.5) and
without/with the SERF-195 patch. I then tried to check out
https://svn.apache.org/repos/private/foundation (which require basic auth).
With a delay between 37.5 ms and 112.5 ms (=average around the 75'000 us in
the bug report) the checkout succeed.
With a delay between 375 ms and 1125 ms (=10 times more then in the bug
report), the checkout fails as follows:
[[[
A foundation/officers/workbench
A foundation/board
X random: 377231.422929 in conn 0x7f23e0b92028
X random: 967579.765638 in conn 0x7f23e0b92028
X random: 590046.259093 in conn 0x7f23e0b92028
X random: 849541.822669 in conn 0x7f23e0b92028
X random: 889138.332808 in conn 0x7f23e0b92028
X random: 866866.873573 in conn 0x7f23e0b92028
X random: 491075.777503 in conn 0x7f23e0b92028
X random: 616534.923083 in conn 0x7f23e0b92028
X random: 785294.915508 in conn 0x7f23e0b92028
X random: 1103729.169573 in conn 0x7f23e0b92028
X random: 638844.156411 in conn 0x7f23e0b92028
X random: 383173.109269 in conn 0x7f23e0d12430
X random: 707753.396282 in conn 0x7f23e0d12430
subversion/svn/checkout-cmd.c:168,
subversion/libsvn_client/checkout.c:318,
subversion/libsvn_client/checkout.c:274,
subversion/libsvn_client/update.c:722,
subversion/libsvn_client/update.c:563,
subversion/libsvn_wc/adm_crawler.c:868,
subversion/libsvn_ra_serf/update.c:2518,
subversion/libsvn_ra_serf/update.c:2508,
subversion/libsvn_ra_serf/update.c:2439,
subversion/libsvn_ra_serf/util.c:966: (apr_err=APR_EOF)
svn: E070014: Error running context: End of file found
]]]
Checking with Wireshark it seems the server is closing the connection on us.
BUT: It isn't the memory corruption issue mentioned and more importantly it
fails exactly the same way both with trunk (no patches, except for the
usleep) and with the SERF-195 branch.
So I'm leaning towards closing SERF-195 as "not reproducible" but I agree
with Brane that the refactoring of linked-list code to dedicated functions
make it easier to review and make sure all cases manage the linked lists
correctly. So we might keep the branch and merge at a later date.
Thoughts?
Cheers,
Daniel
Re: Reproducing SERF-195 [was: Releasing Serf 1.5.0]
Posted by Daniel Sahlberg <da...@gmail.com>.
mån 15 dec. 2025 kl. 23:45 skrev Branko Čibej <br...@apache.org>:
> On 15. 12. 25 19:30, Daniel Sahlberg wrote:
…
> >
> > I've been thinking about the usleep() and it doesn't make sense that it
> > should introduce any problems for a single threaded application. From
> how I
> > understand the description, it seems like something is removing stuff
> from
> > the linked lists during sleep. But if there is only one thread, that
> thread
> > should be the only one working the lists so nothing should happen.
> >
> > Could it be that the original poster is reusing the same context across
> > multiple threads?
>
>
> I've been thinking along those lines, yes.
>
>
> > If so, this is not a supported use case and we should close this issue?
>
>
> The changes on that branch aren't bad, it's nice to encapsulate
> linked-list management. But if we can't reproduce the original issue, I
> wouldn't release this in 1.5, because it's not tested all that much. I
> would merge to trunk _after_ we cut the 1.5 branch, though.
After sending, I read through the issue description and it mentions
reproducing with Subversion. So I will make one last try building Serf from
trunk with the usleep() call and see if something gives.
>
> If we decide to do it this way, then all that's left are the &*#@^Y*!
> error callbacks.
I think waiting is a good idea unless I can reproduce above. Let’s continue
discussing the error callbacks in a separate thread!
Cheers
Daniel
>
>
> -- Brane
Re: Reproducing SERF-195 [was: Releasing Serf 1.5.0]
Posted by Branko Čibej <br...@apache.org>.
On 15. 12. 25 19:30, Daniel Sahlberg wrote:
> Den lör 13 dec. 2025 kl 20:43 skrev Branko Čibej<br...@apache.org>:
>
>> On 13. 12. 25 20:14, Daniel Sahlberg wrote:
>>> Den lör 13 dec. 2025 kl 16:39 skrev Branko Čibej<br...@apache.org>:
>>>
>>>> On 13. 12. 25 03:35, Branko Čibej wrote:
>>>>> On 12. 12. 25 20:49, Branko Čibej wrote:
>>>>>> On 12. 12. 25 20:02, Daniel Sahlberg wrote:
>>>>>>> Den tors 11 dec. 2025 kl 19:43 skrev Branko Čibej<br...@apache.org>:
>>>>>>>> On 11. 12. 25 19:59, Daniel Sahlberg wrote:
>>>>>>>> * Issue SERF-195, which is a substantial rewrite of the request
>>>>>>>> queues.
>>>>>>>> The
>>>>>>>>> code is in a separate branch SERF-195. The code looks good to me
>>>>>>>>> but I
>>>>>>>>> haven't analyzed in detail. It should be possible to merge to
>> trunk.
>>>>>>>> The patch works, in the sense that it causes no regressions, but I
>>>>>>>> haven't been able to reproduce the reported issue. It's also makes
>> the
>>>>>>>> code slightly more readable, but I couldn't find the mentioned race
>>>>>>>> condition during review, it looks like just a refactoring of the
>>>>>>>> original code.
>>>>>>>>
>>>>>>>> I think we need another pair of eyes to do a careful review of the
>>>>>>>> branch changes.
>>>>>>>>
>>>>>>> I'll try to reproduce the initial issue and see if I can figure out
>>>>>>> what
>>>>>>> goes wrong. Do you have any ideas where I should start? I assume a
>>>>>>> multithreaded client (with multiple connections) would be needed?
>>>>>> Multi-threading on the client has no effect here. Request queues are
>>>>>> per-context and contexts require single-threaded access.
>>>>>>
>>>>>> The original issue submission just adds a sleep() in
>>>>>> serf__setup_request_basic_auth(), so I guess you don't need a
>>>>>> multi-threaded client, you just need a server that always requires
>>>>>> authentication. That's why, e.g., linking Subversion with that and
>>>>>> doing read from the Apache repos won't work. You could try to set up
>>>>>> a local multi-threaded http server that requires basic auth and
>>>>>> serves some data -- easy to do in Python -- then write a Serf-based
>>>>>> client that sends requests in a loop. Serf-get would be a good
>>>>>> starting point.
>>>>>>
>>>>>> I think the important points here are:
>>>>>>
>>>>>> * all requests to the server must require authentication;
>>>>>> * the server must be able to process requests in parallel and out of
>>>>>> order: this is the only way I can think of where processed
>> requests
>>>>>> could be removed from the middle of the queue, not the beginning
>> or
>>>>>> the end;
>>>>>> * the server must be "slow" enough that the client request queue
>> fills
>>>>>> up, so it needs both an upper limit to the number of parallel
>>>>>> requests and a fake latency for the responses.
>>>>>>
>>>>>>
>>>>>> I'll give the server script a go.
>>>>> See r1930478.
>>>>>
>>>>> Try something like this:
>>>>>
>>>>> $ authserver.py 2>/dev/null &
>>>>> $ serf_get -n 400 -c 10 -x 20 -U user -P pass -m
>>>>> HEADhttp://127.0.0.1:8087/
>>>>>
>>>>>
>>>>> However, this still doesn't reproduce the reported issue. One possible
>>>>> reason is that serf_get does *not* pipeline the requests in each
>>>>> connection but only sends the next request when the previous response
>>>>> has arrived. I haven't been able to track this down, don't know if
>>>>> it's a problem with serf_get or authserver.py.
>>>> I was mistaken about the pipelining. After adding some instrumentation
>>>> to serf_get (see below), it turns out that requests are pipelined, and
>>>> yet I can't reproduce the issue.
>>>>
>>>> This is the patch for the basic auth handler, the same as in the
>>>> description of SERF-195:
>>>>
>>>> --- serf-trunk/auth/auth_basic.c (revision 1930477)
>>>> +++ serf-trunk/auth/auth_basic.c (working copy)
>>>> @@ -19,6 +19,7 @@
>>>> */
>>>>
>>>> /*** Basic authentication ***/
>>>> +#include <unistd.h>
>>>>
>>>> #include <serf.h>
>>>> #include <serf_private.h>
>>>> @@ -153,6 +154,7 @@ serf__setup_request_basic_auth(const
>> serf__authn_schem
>>>> basic_info = authn_info->baton;
>>>>
>>>> if (basic_info && basic_info->header && basic_info->value) {
>>>> + usleep(75000); /* 75 milliseconds */
>>>> serf_bucket_headers_setn(hdrs_bkt, basic_info->header,
>>>> basic_info->value);
>>>> return APR_SUCCESS;
>>>>
>>>>
>>>> I also noticed something interesting: Each connection sends up to
>>>> max-pending requests at once, but then waits for all the replies before
>>>> sending the next batch. I haven't checked if this is a bug in serf_get
>>>> or in Serf proper; the connection has more requests ready, so it should
>>>> send new ones as soon any received response makes room in the pending
>>>> requests queue. I saw this with the following invocation:
>>>>
>>>> serf_get -d -n 500 -c 1 -x 50 -U user -P something -m GEThttp://
>>>> 127.0.0.1:8087/ >/dev/null
>>>>
>>>>
>>>>
>>>> The instrumentation patch for serf_get:
>>>>
>>>> --- serf-trunk/test/serf_get.c (revision 1930478)
>>>> +++ serf-trunk/test/serf_get.c (working copy)
>>>> @@ -590,6 +590,7 @@ int main(int argc, const char **argv)
>>>> apr_getopt_t *opt;
>>>> int opt_c;
>>>> const char *opt_arg;
>>>> + unsigned int prev_min = 0, prev_max = UINT_MAX, prev_all = 0;
>>>>
>>>> apr_initialize();
>>>> atexit(apr_terminate);
>>>> @@ -910,6 +911,24 @@ int main(int argc, const char **argv)
>>>> break;
>>>> }
>>>> /* Debugging purposes only! */
>>>> + if (debug) {
>>>> + unsigned int min = UINT_MAX, max = 0, all = 0;
>>>> + for (i = 0; i < conn_count; i++) {
>>>> + serf_connection_t *conn = connections[i];
>>>> + unsigned int pending =
>>>> (serf_connection_pending_requests(conn)
>>>> + -
>>>> serf_connection_queued_requests(conn));
>>>> + all += pending;
>>>> + if (min > pending) min = pending;
>>>> + if (max < pending) max = pending;
>>>> + }
>>>> + if (prev_min != min || prev_max != max || prev_all != all)
>> {
>>>> + fprintf(stderr, ">pending: %u [avg=%0.1f min=%u
>>>> max=%u]\n",
>>>> + all, (double)all/conn_count, min, max);
>>>> + prev_min = min;
>>>> + prev_max = max;
>>>> + prev_all = all;
>>>> + }
>>>> + }
>>>> serf_debug__closed_conn(app_ctx.bkt_alloc);
>>>> }
>>>>
>>>>
>>>> -- Brane
>>>>
>>> With a longer delay in authserver.py (10.0) and a high enough number of
>>> concurrent connections (15 seems enough) and high enough outstanding
>>> requests (>2), I seem to get an error:
>>> [[[
>>> dsg@devi-25-01:~/serf_trunk/test$ ./serf_get -d -n 150 -c 15 -x 5 -U
>> foo -P
>>> barhttp://127.0.0.1:8087
>>> ...
>>> Error running context: (70014) End of file found
>>> ]]]
>>>
>>> With 2 outstanding requests:
>>> [[[
>>> dsg@devi-25-01:~/serf_trunk/test$ ./serf_get -d -n 150 -c 15 -x 2 -U
>> foo -P
>>> barhttp://127.0.0.1:8087
>>> ...
>>> 2025-12-13T19:06:06.161609+00 ERROR [l:127.0.0.1:35170 r:127.0.0.1:8087]
>>> receiving raw: Error 104 while reading.
>>> Error running context: (104) Connection reset by peer
>>> ]]]
>>>
>>> If I add the usleep(), I can't reproduce these errors anymore. It seems
>> to
>>> be exactly opposite of what the original poster says - as far as I
>>> understand they claim that the error is triggered by adding a usleep?
>>
>> SERF-195 is about memory corruption and crashes on the client. This
>> appears to be something else.
>>
>> In any case, it would make sense to retry your test with serf_get from
>> the SERF-195 branch (but note that branch needs to be sync-merged from
>> trunk).
>>
> Tried it and it crashes the same way - which as Brane correctly notes -
> isn't quite the same as described in this issue.
>
> I've been thinking about the usleep() and it doesn't make sense that it
> should introduce any problems for a single threaded application. From how I
> understand the description, it seems like something is removing stuff from
> the linked lists during sleep. But if there is only one thread, that thread
> should be the only one working the lists so nothing should happen.
>
> Could it be that the original poster is reusing the same context across
> multiple threads?
I've been thinking along those lines, yes.
> If so, this is not a supported use case and we should close this issue?
The changes on that branch aren't bad, it's nice to encapsulate
linked-list management. But if we can't reproduce the original issue, I
wouldn't release this in 1.5, because it's not tested all that much. I
would merge to trunk _after_ we cut the 1.5 branch, though.
If we decide to do it this way, then all that's left are the &*#@^Y*!
error callbacks.
-- Brane
Re: Reproducing SERF-195 [was: Releasing Serf 1.5.0]
Posted by Daniel Sahlberg <da...@gmail.com>.
Den lör 13 dec. 2025 kl 20:43 skrev Branko Čibej <br...@apache.org>:
> On 13. 12. 25 20:14, Daniel Sahlberg wrote:
> > Den lör 13 dec. 2025 kl 16:39 skrev Branko Čibej<br...@apache.org>:
> >
> >> On 13. 12. 25 03:35, Branko Čibej wrote:
> >>> On 12. 12. 25 20:49, Branko Čibej wrote:
> >>>> On 12. 12. 25 20:02, Daniel Sahlberg wrote:
> >>>>> Den tors 11 dec. 2025 kl 19:43 skrev Branko Čibej<br...@apache.org>:
> >>>>>> On 11. 12. 25 19:59, Daniel Sahlberg wrote:
> >>>>>> * Issue SERF-195, which is a substantial rewrite of the request
> >>>>>> queues.
> >>>>>> The
> >>>>>>> code is in a separate branch SERF-195. The code looks good to me
> >>>>>>> but I
> >>>>>>> haven't analyzed in detail. It should be possible to merge to
> trunk.
> >>>>>> The patch works, in the sense that it causes no regressions, but I
> >>>>>> haven't been able to reproduce the reported issue. It's also makes
> the
> >>>>>> code slightly more readable, but I couldn't find the mentioned race
> >>>>>> condition during review, it looks like just a refactoring of the
> >>>>>> original code.
> >>>>>>
> >>>>>> I think we need another pair of eyes to do a careful review of the
> >>>>>> branch changes.
> >>>>>>
> >>>>> I'll try to reproduce the initial issue and see if I can figure out
> >>>>> what
> >>>>> goes wrong. Do you have any ideas where I should start? I assume a
> >>>>> multithreaded client (with multiple connections) would be needed?
> >>>> Multi-threading on the client has no effect here. Request queues are
> >>>> per-context and contexts require single-threaded access.
> >>>>
> >>>> The original issue submission just adds a sleep() in
> >>>> serf__setup_request_basic_auth(), so I guess you don't need a
> >>>> multi-threaded client, you just need a server that always requires
> >>>> authentication. That's why, e.g., linking Subversion with that and
> >>>> doing read from the Apache repos won't work. You could try to set up
> >>>> a local multi-threaded http server that requires basic auth and
> >>>> serves some data -- easy to do in Python -- then write a Serf-based
> >>>> client that sends requests in a loop. Serf-get would be a good
> >>>> starting point.
> >>>>
> >>>> I think the important points here are:
> >>>>
> >>>> * all requests to the server must require authentication;
> >>>> * the server must be able to process requests in parallel and out of
> >>>> order: this is the only way I can think of where processed
> requests
> >>>> could be removed from the middle of the queue, not the beginning
> or
> >>>> the end;
> >>>> * the server must be "slow" enough that the client request queue
> fills
> >>>> up, so it needs both an upper limit to the number of parallel
> >>>> requests and a fake latency for the responses.
> >>>>
> >>>>
> >>>> I'll give the server script a go.
> >>> See r1930478.
> >>>
> >>> Try something like this:
> >>>
> >>> $ authserver.py 2>/dev/null &
> >>> $ serf_get -n 400 -c 10 -x 20 -U user -P pass -m
> >>> HEADhttp://127.0.0.1:8087/
> >>>
> >>>
> >>> However, this still doesn't reproduce the reported issue. One possible
> >>> reason is that serf_get does *not* pipeline the requests in each
> >>> connection but only sends the next request when the previous response
> >>> has arrived. I haven't been able to track this down, don't know if
> >>> it's a problem with serf_get or authserver.py.
> >> I was mistaken about the pipelining. After adding some instrumentation
> >> to serf_get (see below), it turns out that requests are pipelined, and
> >> yet I can't reproduce the issue.
> >>
> >> This is the patch for the basic auth handler, the same as in the
> >> description of SERF-195:
> >>
> >> --- serf-trunk/auth/auth_basic.c (revision 1930477)
> >> +++ serf-trunk/auth/auth_basic.c (working copy)
> >> @@ -19,6 +19,7 @@
> >> */
> >>
> >> /*** Basic authentication ***/
> >> +#include <unistd.h>
> >>
> >> #include <serf.h>
> >> #include <serf_private.h>
> >> @@ -153,6 +154,7 @@ serf__setup_request_basic_auth(const
> serf__authn_schem
> >> basic_info = authn_info->baton;
> >>
> >> if (basic_info && basic_info->header && basic_info->value) {
> >> + usleep(75000); /* 75 milliseconds */
> >> serf_bucket_headers_setn(hdrs_bkt, basic_info->header,
> >> basic_info->value);
> >> return APR_SUCCESS;
> >>
> >>
> >> I also noticed something interesting: Each connection sends up to
> >> max-pending requests at once, but then waits for all the replies before
> >> sending the next batch. I haven't checked if this is a bug in serf_get
> >> or in Serf proper; the connection has more requests ready, so it should
> >> send new ones as soon any received response makes room in the pending
> >> requests queue. I saw this with the following invocation:
> >>
> >> serf_get -d -n 500 -c 1 -x 50 -U user -P something -m GEThttp://
> >> 127.0.0.1:8087/ >/dev/null
> >>
> >>
> >>
> >> The instrumentation patch for serf_get:
> >>
> >> --- serf-trunk/test/serf_get.c (revision 1930478)
> >> +++ serf-trunk/test/serf_get.c (working copy)
> >> @@ -590,6 +590,7 @@ int main(int argc, const char **argv)
> >> apr_getopt_t *opt;
> >> int opt_c;
> >> const char *opt_arg;
> >> + unsigned int prev_min = 0, prev_max = UINT_MAX, prev_all = 0;
> >>
> >> apr_initialize();
> >> atexit(apr_terminate);
> >> @@ -910,6 +911,24 @@ int main(int argc, const char **argv)
> >> break;
> >> }
> >> /* Debugging purposes only! */
> >> + if (debug) {
> >> + unsigned int min = UINT_MAX, max = 0, all = 0;
> >> + for (i = 0; i < conn_count; i++) {
> >> + serf_connection_t *conn = connections[i];
> >> + unsigned int pending =
> >> (serf_connection_pending_requests(conn)
> >> + -
> >> serf_connection_queued_requests(conn));
> >> + all += pending;
> >> + if (min > pending) min = pending;
> >> + if (max < pending) max = pending;
> >> + }
> >> + if (prev_min != min || prev_max != max || prev_all != all)
> {
> >> + fprintf(stderr, ">pending: %u [avg=%0.1f min=%u
> >> max=%u]\n",
> >> + all, (double)all/conn_count, min, max);
> >> + prev_min = min;
> >> + prev_max = max;
> >> + prev_all = all;
> >> + }
> >> + }
> >> serf_debug__closed_conn(app_ctx.bkt_alloc);
> >> }
> >>
> >>
> >> -- Brane
> >>
> > With a longer delay in authserver.py (10.0) and a high enough number of
> > concurrent connections (15 seems enough) and high enough outstanding
> > requests (>2), I seem to get an error:
> > [[[
> > dsg@devi-25-01:~/serf_trunk/test$ ./serf_get -d -n 150 -c 15 -x 5 -U
> foo -P
> > barhttp://127.0.0.1:8087
> > ...
> > Error running context: (70014) End of file found
> > ]]]
> >
> > With 2 outstanding requests:
> > [[[
> > dsg@devi-25-01:~/serf_trunk/test$ ./serf_get -d -n 150 -c 15 -x 2 -U
> foo -P
> > barhttp://127.0.0.1:8087
> > ...
> > 2025-12-13T19:06:06.161609+00 ERROR [l:127.0.0.1:35170 r:127.0.0.1:8087]
> > receiving raw: Error 104 while reading.
> > Error running context: (104) Connection reset by peer
> > ]]]
> >
> > If I add the usleep(), I can't reproduce these errors anymore. It seems
> to
> > be exactly opposite of what the original poster says - as far as I
> > understand they claim that the error is triggered by adding a usleep?
>
>
> SERF-195 is about memory corruption and crashes on the client. This
> appears to be something else.
>
> In any case, it would make sense to retry your test with serf_get from
> the SERF-195 branch (but note that branch needs to be sync-merged from
> trunk).
>
Tried it and it crashes the same way - which as Brane correctly notes -
isn't quite the same as described in this issue.
I've been thinking about the usleep() and it doesn't make sense that it
should introduce any problems for a single threaded application. From how I
understand the description, it seems like something is removing stuff from
the linked lists during sleep. But if there is only one thread, that thread
should be the only one working the lists so nothing should happen.
Could it be that the original poster is reusing the same context across
multiple threads? If so, this is not a supported use case and we should
close this issue?
Cheers,
Daniel
Re: Reproducing SERF-195 [was: Releasing Serf 1.5.0]
Posted by Branko Čibej <br...@apache.org>.
On 13. 12. 25 20:14, Daniel Sahlberg wrote:
> Den lör 13 dec. 2025 kl 16:39 skrev Branko Čibej<br...@apache.org>:
>
>> On 13. 12. 25 03:35, Branko Čibej wrote:
>>> On 12. 12. 25 20:49, Branko Čibej wrote:
>>>> On 12. 12. 25 20:02, Daniel Sahlberg wrote:
>>>>> Den tors 11 dec. 2025 kl 19:43 skrev Branko Čibej<br...@apache.org>:
>>>>>> On 11. 12. 25 19:59, Daniel Sahlberg wrote:
>>>>>> * Issue SERF-195, which is a substantial rewrite of the request
>>>>>> queues.
>>>>>> The
>>>>>>> code is in a separate branch SERF-195. The code looks good to me
>>>>>>> but I
>>>>>>> haven't analyzed in detail. It should be possible to merge to trunk.
>>>>>> The patch works, in the sense that it causes no regressions, but I
>>>>>> haven't been able to reproduce the reported issue. It's also makes the
>>>>>> code slightly more readable, but I couldn't find the mentioned race
>>>>>> condition during review, it looks like just a refactoring of the
>>>>>> original code.
>>>>>>
>>>>>> I think we need another pair of eyes to do a careful review of the
>>>>>> branch changes.
>>>>>>
>>>>> I'll try to reproduce the initial issue and see if I can figure out
>>>>> what
>>>>> goes wrong. Do you have any ideas where I should start? I assume a
>>>>> multithreaded client (with multiple connections) would be needed?
>>>> Multi-threading on the client has no effect here. Request queues are
>>>> per-context and contexts require single-threaded access.
>>>>
>>>> The original issue submission just adds a sleep() in
>>>> serf__setup_request_basic_auth(), so I guess you don't need a
>>>> multi-threaded client, you just need a server that always requires
>>>> authentication. That's why, e.g., linking Subversion with that and
>>>> doing read from the Apache repos won't work. You could try to set up
>>>> a local multi-threaded http server that requires basic auth and
>>>> serves some data -- easy to do in Python -- then write a Serf-based
>>>> client that sends requests in a loop. Serf-get would be a good
>>>> starting point.
>>>>
>>>> I think the important points here are:
>>>>
>>>> * all requests to the server must require authentication;
>>>> * the server must be able to process requests in parallel and out of
>>>> order: this is the only way I can think of where processed requests
>>>> could be removed from the middle of the queue, not the beginning or
>>>> the end;
>>>> * the server must be "slow" enough that the client request queue fills
>>>> up, so it needs both an upper limit to the number of parallel
>>>> requests and a fake latency for the responses.
>>>>
>>>>
>>>> I'll give the server script a go.
>>> See r1930478.
>>>
>>> Try something like this:
>>>
>>> $ authserver.py 2>/dev/null &
>>> $ serf_get -n 400 -c 10 -x 20 -U user -P pass -m
>>> HEADhttp://127.0.0.1:8087/
>>>
>>>
>>> However, this still doesn't reproduce the reported issue. One possible
>>> reason is that serf_get does *not* pipeline the requests in each
>>> connection but only sends the next request when the previous response
>>> has arrived. I haven't been able to track this down, don't know if
>>> it's a problem with serf_get or authserver.py.
>> I was mistaken about the pipelining. After adding some instrumentation
>> to serf_get (see below), it turns out that requests are pipelined, and
>> yet I can't reproduce the issue.
>>
>> This is the patch for the basic auth handler, the same as in the
>> description of SERF-195:
>>
>> --- serf-trunk/auth/auth_basic.c (revision 1930477)
>> +++ serf-trunk/auth/auth_basic.c (working copy)
>> @@ -19,6 +19,7 @@
>> */
>>
>> /*** Basic authentication ***/
>> +#include <unistd.h>
>>
>> #include <serf.h>
>> #include <serf_private.h>
>> @@ -153,6 +154,7 @@ serf__setup_request_basic_auth(const serf__authn_schem
>> basic_info = authn_info->baton;
>>
>> if (basic_info && basic_info->header && basic_info->value) {
>> + usleep(75000); /* 75 milliseconds */
>> serf_bucket_headers_setn(hdrs_bkt, basic_info->header,
>> basic_info->value);
>> return APR_SUCCESS;
>>
>>
>> I also noticed something interesting: Each connection sends up to
>> max-pending requests at once, but then waits for all the replies before
>> sending the next batch. I haven't checked if this is a bug in serf_get
>> or in Serf proper; the connection has more requests ready, so it should
>> send new ones as soon any received response makes room in the pending
>> requests queue. I saw this with the following invocation:
>>
>> serf_get -d -n 500 -c 1 -x 50 -U user -P something -m GEThttp://
>> 127.0.0.1:8087/ >/dev/null
>>
>>
>>
>> The instrumentation patch for serf_get:
>>
>> --- serf-trunk/test/serf_get.c (revision 1930478)
>> +++ serf-trunk/test/serf_get.c (working copy)
>> @@ -590,6 +590,7 @@ int main(int argc, const char **argv)
>> apr_getopt_t *opt;
>> int opt_c;
>> const char *opt_arg;
>> + unsigned int prev_min = 0, prev_max = UINT_MAX, prev_all = 0;
>>
>> apr_initialize();
>> atexit(apr_terminate);
>> @@ -910,6 +911,24 @@ int main(int argc, const char **argv)
>> break;
>> }
>> /* Debugging purposes only! */
>> + if (debug) {
>> + unsigned int min = UINT_MAX, max = 0, all = 0;
>> + for (i = 0; i < conn_count; i++) {
>> + serf_connection_t *conn = connections[i];
>> + unsigned int pending =
>> (serf_connection_pending_requests(conn)
>> + -
>> serf_connection_queued_requests(conn));
>> + all += pending;
>> + if (min > pending) min = pending;
>> + if (max < pending) max = pending;
>> + }
>> + if (prev_min != min || prev_max != max || prev_all != all) {
>> + fprintf(stderr, ">pending: %u [avg=%0.1f min=%u
>> max=%u]\n",
>> + all, (double)all/conn_count, min, max);
>> + prev_min = min;
>> + prev_max = max;
>> + prev_all = all;
>> + }
>> + }
>> serf_debug__closed_conn(app_ctx.bkt_alloc);
>> }
>>
>>
>> -- Brane
>>
> With a longer delay in authserver.py (10.0) and a high enough number of
> concurrent connections (15 seems enough) and high enough outstanding
> requests (>2), I seem to get an error:
> [[[
> dsg@devi-25-01:~/serf_trunk/test$ ./serf_get -d -n 150 -c 15 -x 5 -U foo -P
> barhttp://127.0.0.1:8087
> ...
> Error running context: (70014) End of file found
> ]]]
>
> With 2 outstanding requests:
> [[[
> dsg@devi-25-01:~/serf_trunk/test$ ./serf_get -d -n 150 -c 15 -x 2 -U foo -P
> barhttp://127.0.0.1:8087
> ...
> 2025-12-13T19:06:06.161609+00 ERROR [l:127.0.0.1:35170 r:127.0.0.1:8087]
> receiving raw: Error 104 while reading.
> Error running context: (104) Connection reset by peer
> ]]]
>
> If I add the usleep(), I can't reproduce these errors anymore. It seems to
> be exactly opposite of what the original poster says - as far as I
> understand they claim that the error is triggered by adding a usleep?
SERF-195 is about memory corruption and crashes on the client. This
appears to be something else.
In any case, it would make sense to retry your test with serf_get from
the SERF-195 branch (but note that branch needs to be sync-merged from
trunk).
-- Brane
Re: Reproducing SERF-195 [was: Releasing Serf 1.5.0]
Posted by Daniel Sahlberg <da...@gmail.com>.
Den lör 13 dec. 2025 kl 16:39 skrev Branko Čibej <br...@apache.org>:
> On 13. 12. 25 03:35, Branko Čibej wrote:
> > On 12. 12. 25 20:49, Branko Čibej wrote:
> >> On 12. 12. 25 20:02, Daniel Sahlberg wrote:
> >>> Den tors 11 dec. 2025 kl 19:43 skrev Branko Čibej<br...@apache.org>:
> >>>> On 11. 12. 25 19:59, Daniel Sahlberg wrote:
> >>>> * Issue SERF-195, which is a substantial rewrite of the request
> >>>> queues.
> >>>> The
> >>>>> code is in a separate branch SERF-195. The code looks good to me
> >>>>> but I
> >>>>> haven't analyzed in detail. It should be possible to merge to trunk.
> >>>> The patch works, in the sense that it causes no regressions, but I
> >>>> haven't been able to reproduce the reported issue. It's also makes the
> >>>> code slightly more readable, but I couldn't find the mentioned race
> >>>> condition during review, it looks like just a refactoring of the
> >>>> original code.
> >>>>
> >>>> I think we need another pair of eyes to do a careful review of the
> >>>> branch changes.
> >>>>
> >>> I'll try to reproduce the initial issue and see if I can figure out
> >>> what
> >>> goes wrong. Do you have any ideas where I should start? I assume a
> >>> multithreaded client (with multiple connections) would be needed?
> >>
> >> Multi-threading on the client has no effect here. Request queues are
> >> per-context and contexts require single-threaded access.
> >>
> >> The original issue submission just adds a sleep() in
> >> serf__setup_request_basic_auth(), so I guess you don't need a
> >> multi-threaded client, you just need a server that always requires
> >> authentication. That's why, e.g., linking Subversion with that and
> >> doing read from the Apache repos won't work. You could try to set up
> >> a local multi-threaded http server that requires basic auth and
> >> serves some data -- easy to do in Python -- then write a Serf-based
> >> client that sends requests in a loop. Serf-get would be a good
> >> starting point.
> >>
> >> I think the important points here are:
> >>
> >> * all requests to the server must require authentication;
> >> * the server must be able to process requests in parallel and out of
> >> order: this is the only way I can think of where processed requests
> >> could be removed from the middle of the queue, not the beginning or
> >> the end;
> >> * the server must be "slow" enough that the client request queue fills
> >> up, so it needs both an upper limit to the number of parallel
> >> requests and a fake latency for the responses.
> >>
> >>
> >> I'll give the server script a go.
> >
> > See r1930478.
> >
> > Try something like this:
> >
> > $ authserver.py 2>/dev/null &
> > $ serf_get -n 400 -c 10 -x 20 -U user -P pass -m
> > HEADhttp://127.0.0.1:8087/
> >
> >
> > However, this still doesn't reproduce the reported issue. One possible
> > reason is that serf_get does *not* pipeline the requests in each
> > connection but only sends the next request when the previous response
> > has arrived. I haven't been able to track this down, don't know if
> > it's a problem with serf_get or authserver.py.
>
> I was mistaken about the pipelining. After adding some instrumentation
> to serf_get (see below), it turns out that requests are pipelined, and
> yet I can't reproduce the issue.
>
> This is the patch for the basic auth handler, the same as in the
> description of SERF-195:
>
> --- serf-trunk/auth/auth_basic.c (revision 1930477)
> +++ serf-trunk/auth/auth_basic.c (working copy)
> @@ -19,6 +19,7 @@
> */
>
> /*** Basic authentication ***/
> +#include <unistd.h>
>
> #include <serf.h>
> #include <serf_private.h>
> @@ -153,6 +154,7 @@ serf__setup_request_basic_auth(const serf__authn_schem
> basic_info = authn_info->baton;
>
> if (basic_info && basic_info->header && basic_info->value) {
> + usleep(75000); /* 75 milliseconds */
> serf_bucket_headers_setn(hdrs_bkt, basic_info->header,
> basic_info->value);
> return APR_SUCCESS;
>
>
> I also noticed something interesting: Each connection sends up to
> max-pending requests at once, but then waits for all the replies before
> sending the next batch. I haven't checked if this is a bug in serf_get
> or in Serf proper; the connection has more requests ready, so it should
> send new ones as soon any received response makes room in the pending
> requests queue. I saw this with the following invocation:
>
> serf_get -d -n 500 -c 1 -x 50 -U user -P something -m GEThttp://
> 127.0.0.1:8087/ >/dev/null
>
>
>
> The instrumentation patch for serf_get:
>
> --- serf-trunk/test/serf_get.c (revision 1930478)
> +++ serf-trunk/test/serf_get.c (working copy)
> @@ -590,6 +590,7 @@ int main(int argc, const char **argv)
> apr_getopt_t *opt;
> int opt_c;
> const char *opt_arg;
> + unsigned int prev_min = 0, prev_max = UINT_MAX, prev_all = 0;
>
> apr_initialize();
> atexit(apr_terminate);
> @@ -910,6 +911,24 @@ int main(int argc, const char **argv)
> break;
> }
> /* Debugging purposes only! */
> + if (debug) {
> + unsigned int min = UINT_MAX, max = 0, all = 0;
> + for (i = 0; i < conn_count; i++) {
> + serf_connection_t *conn = connections[i];
> + unsigned int pending =
> (serf_connection_pending_requests(conn)
> + -
> serf_connection_queued_requests(conn));
> + all += pending;
> + if (min > pending) min = pending;
> + if (max < pending) max = pending;
> + }
> + if (prev_min != min || prev_max != max || prev_all != all) {
> + fprintf(stderr, ">pending: %u [avg=%0.1f min=%u
> max=%u]\n",
> + all, (double)all/conn_count, min, max);
> + prev_min = min;
> + prev_max = max;
> + prev_all = all;
> + }
> + }
> serf_debug__closed_conn(app_ctx.bkt_alloc);
> }
>
>
> -- Brane
>
With a longer delay in authserver.py (10.0) and a high enough number of
concurrent connections (15 seems enough) and high enough outstanding
requests (>2), I seem to get an error:
[[[
dsg@devi-25-01:~/serf_trunk/test$ ./serf_get -d -n 150 -c 15 -x 5 -U foo -P
bar http://127.0.0.1:8087
...
Error running context: (70014) End of file found
]]]
With 2 outstanding requests:
[[[
dsg@devi-25-01:~/serf_trunk/test$ ./serf_get -d -n 150 -c 15 -x 2 -U foo -P
bar http://127.0.0.1:8087
...
2025-12-13T19:06:06.161609+00 ERROR [l:127.0.0.1:35170 r:127.0.0.1:8087]
receiving raw: Error 104 while reading.
Error running context: (104) Connection reset by peer
]]]
If I add the usleep(), I can't reproduce these errors anymore. It seems to
be exactly opposite of what the original poster says - as far as I
understand they claim that the error is triggered by adding a usleep?
Cheers,
Daniel
Re: Reproducing SERF-195 [was: Releasing Serf 1.5.0]
Posted by Branko Čibej <br...@apache.org>.
On 13. 12. 25 03:35, Branko Čibej wrote:
> On 12. 12. 25 20:49, Branko Čibej wrote:
>> On 12. 12. 25 20:02, Daniel Sahlberg wrote:
>>> Den tors 11 dec. 2025 kl 19:43 skrev Branko Čibej<br...@apache.org>:
>>>> On 11. 12. 25 19:59, Daniel Sahlberg wrote:
>>>> * Issue SERF-195, which is a substantial rewrite of the request
>>>> queues.
>>>> The
>>>>> code is in a separate branch SERF-195. The code looks good to me
>>>>> but I
>>>>> haven't analyzed in detail. It should be possible to merge to trunk.
>>>> The patch works, in the sense that it causes no regressions, but I
>>>> haven't been able to reproduce the reported issue. It's also makes the
>>>> code slightly more readable, but I couldn't find the mentioned race
>>>> condition during review, it looks like just a refactoring of the
>>>> original code.
>>>>
>>>> I think we need another pair of eyes to do a careful review of the
>>>> branch changes.
>>>>
>>> I'll try to reproduce the initial issue and see if I can figure out
>>> what
>>> goes wrong. Do you have any ideas where I should start? I assume a
>>> multithreaded client (with multiple connections) would be needed?
>>
>> Multi-threading on the client has no effect here. Request queues are
>> per-context and contexts require single-threaded access.
>>
>> The original issue submission just adds a sleep() in
>> serf__setup_request_basic_auth(), so I guess you don't need a
>> multi-threaded client, you just need a server that always requires
>> authentication. That's why, e.g., linking Subversion with that and
>> doing read from the Apache repos won't work. You could try to set up
>> a local multi-threaded http server that requires basic auth and
>> serves some data -- easy to do in Python -- then write a Serf-based
>> client that sends requests in a loop. Serf-get would be a good
>> starting point.
>>
>> I think the important points here are:
>>
>> * all requests to the server must require authentication;
>> * the server must be able to process requests in parallel and out of
>> order: this is the only way I can think of where processed requests
>> could be removed from the middle of the queue, not the beginning or
>> the end;
>> * the server must be "slow" enough that the client request queue fills
>> up, so it needs both an upper limit to the number of parallel
>> requests and a fake latency for the responses.
>>
>>
>> I'll give the server script a go.
>
> See r1930478.
>
> Try something like this:
>
> $ authserver.py 2>/dev/null &
> $ serf_get -n 400 -c 10 -x 20 -U user -P pass -m
> HEADhttp://127.0.0.1:8087/
>
>
> However, this still doesn't reproduce the reported issue. One possible
> reason is that serf_get does *not* pipeline the requests in each
> connection but only sends the next request when the previous response
> has arrived. I haven't been able to track this down, don't know if
> it's a problem with serf_get or authserver.py.
I was mistaken about the pipelining. After adding some instrumentation
to serf_get (see below), it turns out that requests are pipelined, and
yet I can't reproduce the issue.
This is the patch for the basic auth handler, the same as in the
description of SERF-195:
--- serf-trunk/auth/auth_basic.c (revision 1930477)
+++ serf-trunk/auth/auth_basic.c (working copy)
@@ -19,6 +19,7 @@
*/
/*** Basic authentication ***/
+#include <unistd.h>
#include <serf.h>
#include <serf_private.h>
@@ -153,6 +154,7 @@ serf__setup_request_basic_auth(const serf__authn_schem
basic_info = authn_info->baton;
if (basic_info && basic_info->header && basic_info->value) {
+ usleep(75000); /* 75 milliseconds */
serf_bucket_headers_setn(hdrs_bkt, basic_info->header,
basic_info->value);
return APR_SUCCESS;
I also noticed something interesting: Each connection sends up to
max-pending requests at once, but then waits for all the replies before
sending the next batch. I haven't checked if this is a bug in serf_get
or in Serf proper; the connection has more requests ready, so it should
send new ones as soon any received response makes room in the pending
requests queue. I saw this with the following invocation:
serf_get -d -n 500 -c 1 -x 50 -U user -P something -m GEThttp://127.0.0.1:8087/ >/dev/null
The instrumentation patch for serf_get:
--- serf-trunk/test/serf_get.c (revision 1930478)
+++ serf-trunk/test/serf_get.c (working copy)
@@ -590,6 +590,7 @@ int main(int argc, const char **argv)
apr_getopt_t *opt;
int opt_c;
const char *opt_arg;
+ unsigned int prev_min = 0, prev_max = UINT_MAX, prev_all = 0;
apr_initialize();
atexit(apr_terminate);
@@ -910,6 +911,24 @@ int main(int argc, const char **argv)
break;
}
/* Debugging purposes only! */
+ if (debug) {
+ unsigned int min = UINT_MAX, max = 0, all = 0;
+ for (i = 0; i < conn_count; i++) {
+ serf_connection_t *conn = connections[i];
+ unsigned int pending = (serf_connection_pending_requests(conn)
+ - serf_connection_queued_requests(conn));
+ all += pending;
+ if (min > pending) min = pending;
+ if (max < pending) max = pending;
+ }
+ if (prev_min != min || prev_max != max || prev_all != all) {
+ fprintf(stderr, ">pending: %u [avg=%0.1f min=%u max=%u]\n",
+ all, (double)all/conn_count, min, max);
+ prev_min = min;
+ prev_max = max;
+ prev_all = all;
+ }
+ }
serf_debug__closed_conn(app_ctx.bkt_alloc);
}
-- Brane
Re: Releasing Serf 1.5.0
Posted by Branko Čibej <br...@apache.org>.
On 12. 12. 25 20:49, Branko Čibej wrote:
> On 12. 12. 25 20:02, Daniel Sahlberg wrote:
>> Den tors 11 dec. 2025 kl 19:43 skrev Branko Čibej<br...@apache.org>:
>>> On 11. 12. 25 19:59, Daniel Sahlberg wrote:
>>> * Issue SERF-195, which is a substantial rewrite of the request queues.
>>> The
>>>> code is in a separate branch SERF-195. The code looks good to me but I
>>>> haven't analyzed in detail. It should be possible to merge to trunk.
>>> The patch works, in the sense that it causes no regressions, but I
>>> haven't been able to reproduce the reported issue. It's also makes the
>>> code slightly more readable, but I couldn't find the mentioned race
>>> condition during review, it looks like just a refactoring of the
>>> original code.
>>>
>>> I think we need another pair of eyes to do a careful review of the
>>> branch changes.
>>>
>> I'll try to reproduce the initial issue and see if I can figure out what
>> goes wrong. Do you have any ideas where I should start? I assume a
>> multithreaded client (with multiple connections) would be needed?
>
> Multi-threading on the client has no effect here. Request queues are
> per-context and contexts require single-threaded access.
>
> The original issue submission just adds a sleep() in
> serf__setup_request_basic_auth(), so I guess you don't need a
> multi-threaded client, you just need a server that always requires
> authentication. That's why, e.g., linking Subversion with that and
> doing read from the Apache repos won't work. You could try to set up a
> local multi-threaded http server that requires basic auth and serves
> some data -- easy to do in Python -- then write a Serf-based client
> that sends requests in a loop. Serf-get would be a good starting point.
>
> I think the important points here are:
>
> * all requests to the server must require authentication;
> * the server must be able to process requests in parallel and out of
> order: this is the only way I can think of where processed requests
> could be removed from the middle of the queue, not the beginning or
> the end;
> * the server must be "slow" enough that the client request queue fills
> up, so it needs both an upper limit to the number of parallel
> requests and a fake latency for the responses.
>
>
> I'll give the server script a go.
See r1930478.
Try something like this:
$ authserver.py 2>/dev/null &
$ serf_get -n 400 -c 10 -x 20 -U user -P pass -m HEADhttp://127.0.0.1:8087/
However, this still doesn't reproduce the reported issue. One possible
reason is that serf_get does *not* pipeline the requests in each
connection but only sends the next request when the previous response
has arrived. I haven't been able to track this down, don't know if it's
a problem with serf_get or authserver.py.
-- Brane
Re: Releasing Serf 1.5.0
Posted by Branko Čibej <br...@apache.org>.
On 12. 12. 25 20:02, Daniel Sahlberg wrote:
> Den tors 11 dec. 2025 kl 19:43 skrev Branko Čibej<br...@apache.org>:
>
>> On 11. 12. 25 19:59, Daniel Sahlberg wrote:
>>> Hi,
>>>
>>> Prompted by Evgeny's suggestion to move forward with releasing Subversion
>>> 1.15.0[1], I'd like to suggest getting a new minor release of Serf out of
>>> the door as well - preferrably first!
>>>
>>> There are a few features in Subversion that depend on recent development
>> in
>>> Serf (for example the error reporting) and it would be nice to have it
>>> released together.
>>>
>>> Brane has made a summary in Jira, see SERF-208[2], open points copied
>> below:
>>> * Generalized error callbacks, discussed in [3]. From what I can see we
>>> have an error callback mechanism that works for SSL errors. There is also
>>> code in Subversion to support this. We need to decide if [a] We are happy
>>> with the current situation, or [b] Can someone step up to improve it.
>>> Personally, I'm leaning towards [a].
>> I'm not happy with the current custom callback just for SSL errors. I'd
>> be -0.9 to release it as a public API. I sort of dropped the ball .
>>
> In the previous thread, I proposed a patch which replaced the callback with
> a more generic serf_error_cb_t, with an addtiional error_type parameter.
> Basically:
>
> [[[
> -typedef apr_status_t (*serf_ssl_error_cb_t)(
> +typedef apr_status_t (*serf_error_cb_t)(
> + serf_error_type_t error_type,
> void *baton,
> apr_status_t status,
> const char *message);
> ]]]
>
> I did NOT implement this anywhere else apart from the ssl code but it
> should be easy enough to add to other parts of Serf as well.
>
> WDYT?
The problem is that the error callback is global, not connection- or at
least context-specific. That simply doesn't scale. Ideally, we'd have
global, context-, connection- and request-specific callbacks, all
optional, all deferring to the next up the chain if not defined.
But of course the SSL context doesn't have a reference to any of those,
so we'd have to first somehow hint to the SSL functions where they're
being used. That would require a pretty serious refactoring of the API,
which is why I've not even started.
Having only a global callback with no context is unhelpful in anything
but completely trivial use cases. And, yes, Subversion counts as a
trivial use case.
However, starting with a global callback is better than nothing,
assuming we design the API so that other, more specific callbacks can be
added later.
>> * Issue SERF-195, which is a substantial rewrite of the request queues.
>> The
>>> code is in a separate branch SERF-195. The code looks good to me but I
>>> haven't analyzed in detail. It should be possible to merge to trunk.
>> The patch works, in the sense that it causes no regressions, but I
>> haven't been able to reproduce the reported issue. It's also makes the
>> code slightly more readable, but I couldn't find the mentioned race
>> condition during review, it looks like just a refactoring of the
>> original code.
>>
>> I think we need another pair of eyes to do a careful review of the
>> branch changes.
>>
> I'll try to reproduce the initial issue and see if I can figure out what
> goes wrong. Do you have any ideas where I should start? I assume a
> multithreaded client (with multiple connections) would be needed?
Multi-threading on the client has no effect here. Request queues are
per-context and contexts require single-threaded access.
The original issue submission just adds a sleep() in
serf__setup_request_basic_auth(), so I guess you don't need a
multi-threaded client, you just need a server that always requires
authentication. That's why, e.g., linking Subversion with that and doing
read from the Apache repos won't work. You could try to set up a local
multi-threaded http server that requires basic auth and serves some
data -- easy to do in Python -- then write a Serf-based client that
sends requests in a loop. Serf-get would be a good starting point.
I think the important points here are:
* all requests to the server must require authentication;
* the server must be able to process requests in parallel and out of
order: this is the only way I can think of where processed requests
could be removed from the middle of the queue, not the beginning or
the end;
* the server must be "slow" enough that the client request queue fills
up, so it needs both an upper limit to the number of parallel
requests and a fake latency for the responses.
I'll give the server script a go.
-- Brane
Re: Releasing Serf 1.5.0
Posted by Daniel Sahlberg <da...@gmail.com>.
Den tors 11 dec. 2025 kl 19:43 skrev Branko Čibej <br...@apache.org>:
> On 11. 12. 25 19:59, Daniel Sahlberg wrote:
> > Hi,
> >
> > Prompted by Evgeny's suggestion to move forward with releasing Subversion
> > 1.15.0[1], I'd like to suggest getting a new minor release of Serf out of
> > the door as well - preferrably first!
> >
> > There are a few features in Subversion that depend on recent development
> in
> > Serf (for example the error reporting) and it would be nice to have it
> > released together.
> >
> > Brane has made a summary in Jira, see SERF-208[2], open points copied
> below:
> > * Generalized error callbacks, discussed in [3]. From what I can see we
> > have an error callback mechanism that works for SSL errors. There is also
> > code in Subversion to support this. We need to decide if [a] We are happy
> > with the current situation, or [b] Can someone step up to improve it.
> > Personally, I'm leaning towards [a].
>
> I'm not happy with the current custom callback just for SSL errors. I'd
> be -0.9 to release it as a public API. I sort of dropped the ball .
>
In the previous thread, I proposed a patch which replaced the callback with
a more generic serf_error_cb_t, with an addtiional error_type parameter.
Basically:
[[[
-typedef apr_status_t (*serf_ssl_error_cb_t)(
+typedef apr_status_t (*serf_error_cb_t)(
+ serf_error_type_t error_type,
void *baton,
apr_status_t status,
const char *message);
]]]
I did NOT implement this anywhere else apart from the ssl code but it
should be easy enough to add to other parts of Serf as well.
WDYT?
> * Issue SERF-195, which is a substantial rewrite of the request queues.
> The
> > code is in a separate branch SERF-195. The code looks good to me but I
> > haven't analyzed in detail. It should be possible to merge to trunk.
>
> The patch works, in the sense that it causes no regressions, but I
> haven't been able to reproduce the reported issue. It's also makes the
> code slightly more readable, but I couldn't find the mentioned race
> condition during review, it looks like just a refactoring of the
> original code.
>
> I think we need another pair of eyes to do a careful review of the
> branch changes.
>
I'll try to reproduce the initial issue and see if I can figure out what
goes wrong. Do you have any ideas where I should start? I assume a
multithreaded client (with multiple connections) would be needed?
>
> > * Issue SERF-209, concerning intermitent test suite failures under
> MacOS. I
> > would suggest to leave this aside for later.
>
> Yes, it's not critical. Might be something in the test code itself,
> i.e., not related to serf proper.
>
Ack!
Cheers,
Daniel
>
> > Can we get these decided/merged and roll a release? I think it would be
> > good to do this in trunk before branching.
> >
> > Kind regards,
> > Daniel
> >
> >
> >
> >
> > [1]https://lists.apache.org/thread/x0s1c8jolql5hdkq40jm3jfnpm6wjp9s
> > [2]https://issues.apache.org/jira/browse/SERF-208
> > [3]https://lists.apache.org/thread/7khn697o2srmg8wvqy4t3xyxq4cr8v8v
> >
>
Re: Releasing Serf 1.5.0
Posted by Branko Čibej <br...@apache.org>.
On 11. 12. 25 19:59, Daniel Sahlberg wrote:
> Hi,
>
> Prompted by Evgeny's suggestion to move forward with releasing Subversion
> 1.15.0[1], I'd like to suggest getting a new minor release of Serf out of
> the door as well - preferrably first!
>
> There are a few features in Subversion that depend on recent development in
> Serf (for example the error reporting) and it would be nice to have it
> released together.
>
> Brane has made a summary in Jira, see SERF-208[2], open points copied below:
> * Generalized error callbacks, discussed in [3]. From what I can see we
> have an error callback mechanism that works for SSL errors. There is also
> code in Subversion to support this. We need to decide if [a] We are happy
> with the current situation, or [b] Can someone step up to improve it.
> Personally, I'm leaning towards [a].
I'm not happy with the current custom callback just for SSL errors. I'd
be -0.9 to release it as a public API. I sort of dropped the ball .
> * Issue SERF-195, which is a substantial rewrite of the request queues. The
> code is in a separate branch SERF-195. The code looks good to me but I
> haven't analyzed in detail. It should be possible to merge to trunk.
The patch works, in the sense that it causes no regressions, but I
haven't been able to reproduce the reported issue. It's also makes the
code slightly more readable, but I couldn't find the mentioned race
condition during review, it looks like just a refactoring of the
original code.
I think we need another pair of eyes to do a careful review of the
branch changes.
> * Issue SERF-209, concerning intermitent test suite failures under MacOS. I
> would suggest to leave this aside for later.
Yes, it's not critical. Might be something in the test code itself,
i.e., not related to serf proper.
> Can we get these decided/merged and roll a release? I think it would be
> good to do this in trunk before branching.
>
> Kind regards,
> Daniel
>
>
>
>
> [1]https://lists.apache.org/thread/x0s1c8jolql5hdkq40jm3jfnpm6wjp9s
> [2]https://issues.apache.org/jira/browse/SERF-208
> [3]https://lists.apache.org/thread/7khn697o2srmg8wvqy4t3xyxq4cr8v8v
>
Re: Releasing Serf 1.5.0
Posted by Bert Huijben via dev <de...@serf.apache.org>.
+1
I wondered the same thing over the years...
And with that as only important breakage, the few users we do have can
apply an easy fix and really benefit from the added bucket features.
Bert
On Wed, Jan 21, 2026 at 11:49 AM Branko Čibej <br...@apache.org> wrote:
> On 20. 1. 26 11:22, Bert Huijben via dev wrote:
> > The issue I was thinking about how serf_buckets_are_v2() is/was used.
> >
> > But checking the current code in serf_get_type() we probably already
> worked
> > around this problem for some time...
> >
> >
> > Bert
> >
> >
> > ---
> > From trunk:
> > const serf_bucket_type_t *serf_get_type(serf_bucket_t *bucket,
> > int min_version)
> > {
> > const serf_bucket_t *r;
> >
> > switch (min_version) {
> > case 1:
> > r = bucket; /* Always supported */
> > break;
> > #if 0
> > case 3:
> > r = bucket->type->read_bucket(bucket, &v3_check);
> > break;
> > #endif
> >
> > case 2:
> > #if 0
> > /* We can't trust this check to always do the right thing as
> > there
> > can be multiple pointers to the same function...
> > but an optimizing compiler/linker can still use the hint
> if
> > it
> > is inlining this function. ### Enable?
> > */
> > if (bucket->type->read_bucket == serf_buckets_are_v2)
> > return bucket->type;
> > #endif
> >
> > r = bucket->type->read_bucket(bucket, &v2_check);
> > break;
> >
> > default:
> > abort();
> > }
> >
> > if (r != NULL)
> > return r->type;
> >
> > return &fallback_bucket_type;
> > }
>
>
>
> Just out of interest, why do we support multiple bucket type versions? I
> can understand reasons of API compatibility.
>
> But let's be realistic: the only user of Serf that we know of is
> Subversion. We may as well drop these compatibility concerns, make 1.5
> the last 1.x release and go straight to 2.0 in the next release. It's
> not as if half the internet will come crashing down if we do that.
>
> -- Brane
>
Re: Releasing Serf 1.5.0
Posted by Branko Čibej <br...@apache.org>.
On 20. 1. 26 11:22, Bert Huijben via dev wrote:
> The issue I was thinking about how serf_buckets_are_v2() is/was used.
>
> But checking the current code in serf_get_type() we probably already worked
> around this problem for some time...
>
>
> Bert
>
>
> ---
> From trunk:
> const serf_bucket_type_t *serf_get_type(serf_bucket_t *bucket,
> int min_version)
> {
> const serf_bucket_t *r;
>
> switch (min_version) {
> case 1:
> r = bucket; /* Always supported */
> break;
> #if 0
> case 3:
> r = bucket->type->read_bucket(bucket, &v3_check);
> break;
> #endif
>
> case 2:
> #if 0
> /* We can't trust this check to always do the right thing as
> there
> can be multiple pointers to the same function...
> but an optimizing compiler/linker can still use the hint if
> it
> is inlining this function. ### Enable?
> */
> if (bucket->type->read_bucket == serf_buckets_are_v2)
> return bucket->type;
> #endif
>
> r = bucket->type->read_bucket(bucket, &v2_check);
> break;
>
> default:
> abort();
> }
>
> if (r != NULL)
> return r->type;
>
> return &fallback_bucket_type;
> }
Just out of interest, why do we support multiple bucket type versions? I
can understand reasons of API compatibility.
But let's be realistic: the only user of Serf that we know of is
Subversion. We may as well drop these compatibility concerns, make 1.5
the last 1.x release and go straight to 2.0 in the next release. It's
not as if half the internet will come crashing down if we do that.
-- Brane
Re: Releasing Serf 1.5.0
Posted by Bert Huijben via dev <de...@serf.apache.org>.
The issue I was thinking about how serf_buckets_are_v2() is/was used.
But checking the current code in serf_get_type() we probably already worked
around this problem for some time...
Bert
---
From trunk:
const serf_bucket_type_t *serf_get_type(serf_bucket_t *bucket,
int min_version)
{
const serf_bucket_t *r;
switch (min_version) {
case 1:
r = bucket; /* Always supported */
break;
#if 0
case 3:
r = bucket->type->read_bucket(bucket, &v3_check);
break;
#endif
case 2:
#if 0
/* We can't trust this check to always do the right thing as
there
can be multiple pointers to the same function...
but an optimizing compiler/linker can still use the hint if
it
is inlining this function. ### Enable?
*/
if (bucket->type->read_bucket == serf_buckets_are_v2)
return bucket->type;
#endif
r = bucket->type->read_bucket(bucket, &v2_check);
break;
default:
abort();
}
if (r != NULL)
return r->type;
return &fallback_bucket_type;
}
On Tue, Jan 20, 2026 at 11:00 AM Branko Čibej <br...@apache.org> wrote:
> On 20. 1. 26 09:51, Bert Huijben via dev wrote:
> > Hi All,
> >
> > Not really active here any more. Thanks for picking up where we left
> years
> > ago. Really happy to see that you all move to get things forward again.
> >
> > +1 on what Branko mentioned here... Please be very careful with global
> > state, being used. The rest is nicely per context.
> >
> > --------------
> > One thing I still have on my mind but not written down.... and mostly out
> > of context here:
> >
> > What is not properly documented is that on Windows when you use Serf
> built
> > as DLL the comparison of the functions in one of the Serf macros is not
> > guaranteed to work... And in practice it doesn't work there, so the old
> vs
> > new bucket type detection breaks. I think currently all Windows users
> work
> > around this by using the build static. For the future I think we should
> get
> > that address via a helper function to make sure we can do the
> ==comparison
> > safely.
>
> Are you referring to things like this:
>
> extern const serf_bucket_type_t serf_bucket_type_response;
> #define SERF_BUCKET_IS_RESPONSE(b) SERF_BUCKET_CHECK((b), response)
>
>
> which seems OK to me, though I'd have done this with a function call. Or
> could you point me to the macro you had in mind?
>
> >
> > For 1.3 ignoring those code paths is mostly safe, but for 1.5+ it should
> be
> > made safe or the limitations documented
>
> Agreed.
>
>
> -- Brane
>
Re: Releasing Serf 1.5.0
Posted by Branko Čibej <br...@apache.org>.
On 20. 1. 26 09:51, Bert Huijben via dev wrote:
> Hi All,
>
> Not really active here any more. Thanks for picking up where we left years
> ago. Really happy to see that you all move to get things forward again.
>
> +1 on what Branko mentioned here... Please be very careful with global
> state, being used. The rest is nicely per context.
>
> --------------
> One thing I still have on my mind but not written down.... and mostly out
> of context here:
>
> What is not properly documented is that on Windows when you use Serf built
> as DLL the comparison of the functions in one of the Serf macros is not
> guaranteed to work... And in practice it doesn't work there, so the old vs
> new bucket type detection breaks. I think currently all Windows users work
> around this by using the build static. For the future I think we should get
> that address via a helper function to make sure we can do the ==comparison
> safely.
Are you referring to things like this:
extern const serf_bucket_type_t serf_bucket_type_response;
#define SERF_BUCKET_IS_RESPONSE(b) SERF_BUCKET_CHECK((b), response)
which seems OK to me, though I'd have done this with a function call. Or
could you point me to the macro you had in mind?
>
> For 1.3 ignoring those code paths is mostly safe, but for 1.5+ it should be
> made safe or the limitations documented
Agreed.
-- Brane
Re: Releasing Serf 1.5.0
Posted by Bert Huijben via dev <de...@serf.apache.org>.
Hi All,
Not really active here any more. Thanks for picking up where we left years
ago. Really happy to see that you all move to get things forward again.
+1 on what Branko mentioned here... Please be very careful with global
state, being used. The rest is nicely per context.
--------------
One thing I still have on my mind but not written down.... and mostly out
of context here:
What is not properly documented is that on Windows when you use Serf built
as DLL the comparison of the functions in one of the Serf macros is not
guaranteed to work... And in practice it doesn't work there, so the old vs
new bucket type detection breaks. I think currently all Windows users work
around this by using the build static. For the future I think we should get
that address via a helper function to make sure we can do the ==comparison
safely.
For 1.3 ignoring those code paths is mostly safe, but for 1.5+ it should be
made safe or the limitations documented
----
Thanks all,
Bert
On Sun, Jan 18, 2026 at 9:54 AM Branko Čibej <br...@apache.org> wrote:
> On 11. 12. 25 19:59, Daniel Sahlberg wrote:
> > Hi,
> >
> > Prompted by Evgeny's suggestion to move forward with releasing Subversion
> > 1.15.0[1], I'd like to suggest getting a new minor release of Serf out of
> > the door as well - preferrably first!
> >
> > There are a few features in Subversion that depend on recent development
> in
> > Serf (for example the error reporting) and it would be nice to have it
> > released together.
> >
> > Brane has made a summary in Jira, see SERF-208[2], open points copied
> below:
> > * Generalized error callbacks, discussed in [3]. From what I can see we
> > have an error callback mechanism that works for SSL errors. There is also
> > code in Subversion to support this. We need to decide if [a] We are happy
> > with the current situation, or [b] Can someone step up to improve it.
> > Personally, I'm leaning towards [a].
> > * Issue SERF-195, which is a substantial rewrite of the request queues.
> The
> > code is in a separate branch SERF-195. The code looks good to me but I
> > haven't analyzed in detail. It should be possible to merge to trunk.
> > * Issue SERF-209, concerning intermitent test suite failures under
> MacOS. I
> > would suggest to leave this aside for later.
> >
> > Can we get these decided/merged and roll a release? I think it would be
> > good to do this in trunk before branching.
> >
> > Kind regards,
> > Daniel
> >
> >
> >
> >
> > [1]https://lists.apache.org/thread/x0s1c8jolql5hdkq40jm3jfnpm6wjp9s
> > [2]https://issues.apache.org/jira/browse/SERF-208
> > [3]https://lists.apache.org/thread/7khn697o2srmg8wvqy4t3xyxq4cr8v8v
>
> Everything mentioned in SERF-208 is now resolved. In addition to that:
>
> * I went through all the test failures with LibreSSL. They were all
> just differences in the way errors are reported from the library, so
> I adjusted the expected test output.
> * I finally found the reason why some OCSP SSL tests failed on Fedora
> and derivatives; the test code was creating a SHA-1 signature on
> OCSP responses, but the OpenSSL shipped with Fedora forbids the use
> of SHA-1 in this context. I fixed this by using SHA-256 in the test
> code.
>
>
> With the above two changes, I built and successfully tested trunk on:
> ARM64 Windows; aarch64 Linux (Debian 12 and Fedora 43); aarch64
> (Free|Open|Net)BSD; and emulated x86-64 Haiku, just for the thrill of it.
>
> I think we're ready to release, pending API review.
>
> -- Brane
>
Re: Releasing Serf 1.5.0
Posted by Branko Čibej <br...@apache.org>.
On 11. 12. 25 19:59, Daniel Sahlberg wrote:
> Hi,
>
> Prompted by Evgeny's suggestion to move forward with releasing Subversion
> 1.15.0[1], I'd like to suggest getting a new minor release of Serf out of
> the door as well - preferrably first!
>
> There are a few features in Subversion that depend on recent development in
> Serf (for example the error reporting) and it would be nice to have it
> released together.
>
> Brane has made a summary in Jira, see SERF-208[2], open points copied below:
> * Generalized error callbacks, discussed in [3]. From what I can see we
> have an error callback mechanism that works for SSL errors. There is also
> code in Subversion to support this. We need to decide if [a] We are happy
> with the current situation, or [b] Can someone step up to improve it.
> Personally, I'm leaning towards [a].
> * Issue SERF-195, which is a substantial rewrite of the request queues. The
> code is in a separate branch SERF-195. The code looks good to me but I
> haven't analyzed in detail. It should be possible to merge to trunk.
> * Issue SERF-209, concerning intermitent test suite failures under MacOS. I
> would suggest to leave this aside for later.
>
> Can we get these decided/merged and roll a release? I think it would be
> good to do this in trunk before branching.
>
> Kind regards,
> Daniel
>
>
>
>
> [1]https://lists.apache.org/thread/x0s1c8jolql5hdkq40jm3jfnpm6wjp9s
> [2]https://issues.apache.org/jira/browse/SERF-208
> [3]https://lists.apache.org/thread/7khn697o2srmg8wvqy4t3xyxq4cr8v8v
Everything mentioned in SERF-208 is now resolved. In addition to that:
* I went through all the test failures with LibreSSL. They were all
just differences in the way errors are reported from the library, so
I adjusted the expected test output.
* I finally found the reason why some OCSP SSL tests failed on Fedora
and derivatives; the test code was creating a SHA-1 signature on
OCSP responses, but the OpenSSL shipped with Fedora forbids the use
of SHA-1 in this context. I fixed this by using SHA-256 in the test
code.
With the above two changes, I built and successfully tested trunk on:
ARM64 Windows; aarch64 Linux (Debian 12 and Fedora 43); aarch64
(Free|Open|Net)BSD; and emulated x86-64 Haiku, just for the thrill of it.
I think we're ready to release, pending API review.
-- Brane