You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "mikemccand (via GitHub)" <gi...@apache.org> on 2023/09/02 11:46:46 UTC
[GitHub] [lucene] mikemccand opened a new pull request, #12535: LockVerifyServer does not need to reuse addresses nor set accept timeout
mikemccand opened a new pull request, #12535:
URL: https://github.com/apache/lucene/pull/12535
Spinoff from [this shocking build failure](https://lists.apache.org/thread/t79ctdmql0j7v4s8gd9ovk6z7lzrl0w9).
The OS will pick a free socket address to bind to since we pass `0`, and we tell the clients to use the assigned port already, so we don't need to ask the OS to reuse the local address.
Also remove the 30 second timeout: the test can wait for crazy slow clients to wake up and connect.
I confirmed I could still run the `LockVerifyServer` and two `LockStressTest` clients from the command-line, testing `NIOFSLockFactory`. Fortunately, it passed.
Maybe we should move all this lock verifying stuff out of `core`? I don't think this is a tool users often use?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] LockVerifyServer does not need to reuse addresses nor set accept timeout [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler merged PR #12535:
URL: https://github.com/apache/lucene/pull/12535
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] uschindler commented on pull request #12535: LockVerifyServer does not need to reuse addresses nor set accept timeout
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on PR #12535:
URL: https://github.com/apache/lucene/pull/12535#issuecomment-1704953411
Hi,
I checked the code again:
I think the timeout was added to make sure the server ends if one of the clients does not start at all or fails to start. The server is written fine, it waits for exactly `n` clients to start. If one of the client will crush before connecting the server will hang forever, so there should really be a timeout! If 30 seconds is too short, I think we have to dig into the code and figure out why the client did not start. Maybe there's startup logging missing or we do not print it out.
So in my opinion, we should better figure out why the client did not start. Maybe becuase something ran out of file handles?
So my ideas:
- +1 to remove the SO_REUSEADDR
- -1 to remove timeout completely as it will cause the test server to hang if any of the clients does not successfully start
We should maybe change the Lucene Unit Test that uses the server class to better log the stdout/stderr of clients and have some more sanity checks. The 30 seconds timeout should be fine. If we figure out that in reality the clients are too slow to start, we can raise the timeout. But actually it is very unlikely that the very first client takes so long. As the server just accepts `n` clients it waits 30 seconds *per* client. Once the firts client started it is also unlikly that later ones take aditional 30 seconds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mikemccand commented on pull request #12535: LockVerifyServer does not need to reuse addresses nor set accept timeout
Posted by "mikemccand (via GitHub)" <gi...@apache.org>.
mikemccand commented on PR #12535:
URL: https://github.com/apache/lucene/pull/12535#issuecomment-1710327041
> I did notice one odd thing: on test failure, I seemed to have a leftover test.lock in the root directory of the checkout, which is very odd. The test creates a new temp directory and is supposed to use that directory to create its test lock ... so I'm not yet sure how that happened. Or maybe I am just hallucinating ...
OK nevermind -- I'm pretty sure this was leftover from me manually invoking `LockVerifyServer` and `LockStressTest` myself. Phew.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] LockVerifyServer does not need to reuse addresses nor set accept timeout [lucene]
Posted by "mikemccand (via GitHub)" <gi...@apache.org>.
mikemccand commented on PR #12535:
URL: https://github.com/apache/lucene/pull/12535#issuecomment-1881040587
Aha, thank you @uschindler!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] rmuir commented on pull request #12535: LockVerifyServer does not need to reuse addresses nor set accept timeout
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on PR #12535:
URL: https://github.com/apache/lucene/pull/12535#issuecomment-1706783612
The 500ms connect timeout on the client: extremely suspicious (as far as jenkins failure). Make it at least 3 seconds or something? Doesn't matter if its being used by jenkins or not, there's sorta no sense in extremely low timeout values like that when using TCP...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] LockVerifyServer does not need to reuse addresses nor set accept timeout [lucene]
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on PR #12535:
URL: https://github.com/apache/lucene/pull/12535#issuecomment-1881005988
Hi @mikemccand,
I merged this stale PR. Thanks for taking care. Let's see what Jenkins says.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mikemccand commented on pull request #12535: LockVerifyServer does not need to reuse addresses nor set accept timeout
Posted by "mikemccand (via GitHub)" <gi...@apache.org>.
mikemccand commented on PR #12535:
URL: https://github.com/apache/lucene/pull/12535#issuecomment-1710320639
OK I made the changes!
I also manually tested two failure modes of the clients: taking too long to initially connect, and throwing some sort of exception after testing the lock N times. In both cases the test stderr showed the stderr/out from the clients, hopefully making it easier to test in the future.
I did notice one odd thing: on test failure, I seemed to have a leftover `test.lock` in the root directory of the checkout, which is very odd. The test creates a new temp directory and is supposed to use that directory to create its test lock ... so I'm not yet sure how that happened. Or maybe I am just hallucinating ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] LockVerifyServer does not need to reuse addresses nor set accept timeout [lucene]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #12535:
URL: https://github.com/apache/lucene/pull/12535#issuecomment-1880901771
This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the dev@lucene.apache.org list. Thank you for your contribution!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] uschindler commented on pull request #12535: LockVerifyServer does not need to reuse addresses nor set accept timeout
Posted by "uschindler (via GitHub)" <gi...@apache.org>.
uschindler commented on PR #12535:
URL: https://github.com/apache/lucene/pull/12535#issuecomment-1706816121
> The 500ms connect timeout on the client: extremely suspicious (as far as jenkins failure). Make it at least 3 seconds or something? Doesn't matter if its being used by jenkins or not, there's sorta no sense in extremely low timeout values like that when using TCP...
Thanks! Maybe that is the reason for the problem, I haven't seen the timeout on client!
If one of the clients times out, the server will wait for that client forever. As we had not stackdump of the client we can't see that.
I agree:
- lets raise the connect timeout
- DO NOT remove the server timeout, because a fileing client will otherwise make the server wait forever for the missing client after accepting connections of all others
Uwe
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mikemccand commented on pull request #12535: LockVerifyServer does not need to reuse addresses nor set accept timeout
Posted by "mikemccand (via GitHub)" <gi...@apache.org>.
mikemccand commented on PR #12535:
URL: https://github.com/apache/lucene/pull/12535#issuecomment-1710238828
Thanks @uschindler and @rmuir -- I will restore the 30s timeout, try to improve logging on client and server errors (so we can figure out WTF happened that caused client not to connect in 30s window in future build failures), increase client connect timeout from 500ms -> 3s.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org