You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by GitBox <gi...@apache.org> on 2022/05/31 07:26:58 UTC
[GitHub] [qpid-dispatch] jiridanek opened a new pull request, #1567: Fixed #116: Coordinate port assignment between all parallel tests spawned by ctest -j (#123)
jiridanek opened a new pull request, #1567:
URL: https://github.com/apache/qpid-dispatch/pull/1567
* https://github.com/skupperproject/skupper-router/issue/116
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-dispatch] jiridanek commented on a diff in pull request #1567: DISPATCH-2324 - Coordinate port assignment between all parallel tests spawned by ctest -j (#123)
Posted by GitBox <gi...@apache.org>.
jiridanek commented on code in PR #1567:
URL: https://github.com/apache/qpid-dispatch/pull/1567#discussion_r885686638
##########
tests/system_test.py:
##########
@@ -765,6 +772,46 @@ def setup(self):
os.makedirs(self.directory)
os.chdir(self.directory)
+ def _next_port(self) -> int:
+ """Reads and increments value stored in self.port_file, under an exclusive file lock.
+
+ When a lock cannot be acquired immediately, fcntl.lockf blocks.
+
+ Failure possibilities:
+ File locks may not work correctly on network filesystems. We still should be no worse off than we were.
+
+ This method always unlocks the lock file, so it should not ever deadlock other tests running in parallel.
+ Even if that happened, the lock is unlocked by the OS when the file is closed, which happens automatically
+ when the process that opened and locked it ends.
+
+ Invalid content in the self.port_file will break this method. Manual intervention is then required.
+ """
+ try:
+ fcntl.flock(self.port_file, fcntl.LOCK_EX)
+
+ # read old value
Review Comment:
In the beginning (no file), I return the result of random.randint and then store port + 1 to file. Next time, I read number from file and store number + 1 back to file and so on.
If I did not do port + 1 the first time, then I'd return the same port twice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-dispatch] ganeshmurthy commented on a diff in pull request #1567: DISPATCH-2324 - Coordinate port assignment between all parallel tests spawned by ctest -j (#123)
Posted by GitBox <gi...@apache.org>.
ganeshmurthy commented on code in PR #1567:
URL: https://github.com/apache/qpid-dispatch/pull/1567#discussion_r885650797
##########
tests/system_test.py:
##########
@@ -765,6 +772,46 @@ def setup(self):
os.makedirs(self.directory)
os.chdir(self.directory)
+ def _next_port(self) -> int:
+ """Reads and increments value stored in self.port_file, under an exclusive file lock.
+
+ When a lock cannot be acquired immediately, fcntl.lockf blocks.
+
+ Failure possibilities:
+ File locks may not work correctly on network filesystems. We still should be no worse off than we were.
+
+ This method always unlocks the lock file, so it should not ever deadlock other tests running in parallel.
+ Even if that happened, the lock is unlocked by the OS when the file is closed, which happens automatically
+ when the process that opened and locked it ends.
+
+ Invalid content in the self.port_file will break this method. Manual intervention is then required.
+ """
+ try:
+ fcntl.flock(self.port_file, fcntl.LOCK_EX)
+
+ # read old value
Review Comment:
Minor suggestion -
Does `next_port = port + 1` need to be done if the file is empty.
```
# read old value
self.port_file.seek(0, os.SEEK_END)
if self.port_file.tell() != 0:
# file is not empty, read the old port and increment it to get next possible available port.
self.port_file.seek(0)
port = int(self.port_file.read())
next_port = port + 1
else:
# file is empty, so just start with a randint between 20000 and 30000
port = random.randint(self.port_range[0], self.port_range[1])
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-dispatch] jiridanek commented on a diff in pull request #1567: DISPATCH-2324 - Coordinate port assignment between all parallel tests spawned by ctest -j (#123)
Posted by GitBox <gi...@apache.org>.
jiridanek commented on code in PR #1567:
URL: https://github.com/apache/qpid-dispatch/pull/1567#discussion_r885655523
##########
tests/system_test.py:
##########
@@ -765,6 +772,46 @@ def setup(self):
os.makedirs(self.directory)
os.chdir(self.directory)
+ def _next_port(self) -> int:
+ """Reads and increments value stored in self.port_file, under an exclusive file lock.
+
+ When a lock cannot be acquired immediately, fcntl.lockf blocks.
+
+ Failure possibilities:
+ File locks may not work correctly on network filesystems. We still should be no worse off than we were.
+
+ This method always unlocks the lock file, so it should not ever deadlock other tests running in parallel.
+ Even if that happened, the lock is unlocked by the OS when the file is closed, which happens automatically
+ when the process that opened and locked it ends.
+
+ Invalid content in the self.port_file will break this method. Manual intervention is then required.
+ """
+ try:
+ fcntl.flock(self.port_file, fcntl.LOCK_EX)
+
+ # read old value
Review Comment:
i think it is needed; next_port has to be one higher than current port, and it does not matter if I get current port from file or as random number
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-dispatch] jiridanek commented on a diff in pull request #1567: DISPATCH-2324 - Coordinate port assignment between all parallel tests spawned by ctest -j (#123)
Posted by GitBox <gi...@apache.org>.
jiridanek commented on code in PR #1567:
URL: https://github.com/apache/qpid-dispatch/pull/1567#discussion_r885686638
##########
tests/system_test.py:
##########
@@ -765,6 +772,46 @@ def setup(self):
os.makedirs(self.directory)
os.chdir(self.directory)
+ def _next_port(self) -> int:
+ """Reads and increments value stored in self.port_file, under an exclusive file lock.
+
+ When a lock cannot be acquired immediately, fcntl.lockf blocks.
+
+ Failure possibilities:
+ File locks may not work correctly on network filesystems. We still should be no worse off than we were.
+
+ This method always unlocks the lock file, so it should not ever deadlock other tests running in parallel.
+ Even if that happened, the lock is unlocked by the OS when the file is closed, which happens automatically
+ when the process that opened and locked it ends.
+
+ Invalid content in the self.port_file will break this method. Manual intervention is then required.
+ """
+ try:
+ fcntl.flock(self.port_file, fcntl.LOCK_EX)
+
+ # read old value
Review Comment:
In the beginning (no file), I return the result of random.randint and then store port + 1 to file. Next time, I read number from file (to return) and store number + 1 back to file and so on.
If I did not do port + 1 the first time, then I'd return the same port twice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-dispatch] jiridanek merged pull request #1567: DISPATCH-2324 - Coordinate port assignment between all parallel tests spawned by ctest -j (#123)
Posted by GitBox <gi...@apache.org>.
jiridanek merged PR #1567:
URL: https://github.com/apache/qpid-dispatch/pull/1567
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-dispatch] ganeshmurthy commented on a diff in pull request #1567: DISPATCH-2324 - Coordinate port assignment between all parallel tests spawned by ctest -j (#123)
Posted by GitBox <gi...@apache.org>.
ganeshmurthy commented on code in PR #1567:
URL: https://github.com/apache/qpid-dispatch/pull/1567#discussion_r885680985
##########
tests/system_test.py:
##########
@@ -765,6 +772,46 @@ def setup(self):
os.makedirs(self.directory)
os.chdir(self.directory)
+ def _next_port(self) -> int:
+ """Reads and increments value stored in self.port_file, under an exclusive file lock.
+
+ When a lock cannot be acquired immediately, fcntl.lockf blocks.
+
+ Failure possibilities:
+ File locks may not work correctly on network filesystems. We still should be no worse off than we were.
+
+ This method always unlocks the lock file, so it should not ever deadlock other tests running in parallel.
+ Even if that happened, the lock is unlocked by the OS when the file is closed, which happens automatically
+ when the process that opened and locked it ends.
+
+ Invalid content in the self.port_file will break this method. Manual intervention is then required.
+ """
+ try:
+ fcntl.flock(self.port_file, fcntl.LOCK_EX)
+
+ # read old value
Review Comment:
Can you please elaborate? Just trying to understand. Doing `random.randint(self.port_range[0], self.port_range[1])` gives a random port number, why add 1 to it. Thx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org