You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/05/13 15:06:30 UTC
[GitHub] [arrow] lidavidm opened a new pull request #10316: ARROW-12760: [C++][Python][R] Allow setting I/O thread pool size
lidavidm opened a new pull request #10316:
URL: https://github.com/apache/arrow/pull/10316
This adds functions to change the pool size at runtime, but doesn't adjust the default size of 8.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou closed pull request #10316: ARROW-12760: [C++][Python][R] Allow setting I/O thread pool size
Posted by GitBox <gi...@apache.org>.
pitrou closed pull request #10316:
URL: https://github.com/apache/arrow/pull/10316
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #10316: ARROW-12760: [C++][Python][R] Allow setting I/O thread pool size
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on a change in pull request #10316:
URL: https://github.com/apache/arrow/pull/10316#discussion_r633456979
##########
File path: python/pyarrow/io.pxi
##########
@@ -42,6 +42,23 @@ cdef extern from "Python.h":
char *v, Py_ssize_t len) except NULL
+def io_thread_count():
+ """
+ Return the number of threads to use for I/O operations.
+
+ The number of threads is set to a fixed value at startup. It can
+ be modified at runtime by calling :func:`set_io_thread_count()`.
+ """
+ return GetIOThreadPoolCapacity()
+
+
+def set_io_thread_count(int count):
+ """Set the number of threads to use for I/O operations."""
Review comment:
Maybe can add a "See also" reference to `set_cpu_count` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on a change in pull request #10316: ARROW-12760: [C++][Python][R] Allow setting I/O thread pool size
Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #10316:
URL: https://github.com/apache/arrow/pull/10316#discussion_r633554950
##########
File path: python/pyarrow/io.pxi
##########
@@ -42,6 +42,23 @@ cdef extern from "Python.h":
char *v, Py_ssize_t len) except NULL
+def io_thread_count():
+ """
+ Return the number of threads to use for I/O operations.
+
+ The number of threads is set to a fixed value at startup. It can
+ be modified at runtime by calling :func:`set_io_thread_count()`.
+ """
+ return GetIOThreadPoolCapacity()
+
+
+def set_io_thread_count(int count):
+ """Set the number of threads to use for I/O operations."""
Review comment:
Done, I cross-referenced the two pairs of functions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on a change in pull request #10316: ARROW-12760: [C++][Python][R] Allow setting I/O thread pool size
Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #10316:
URL: https://github.com/apache/arrow/pull/10316#discussion_r633554950
##########
File path: python/pyarrow/io.pxi
##########
@@ -42,6 +42,23 @@ cdef extern from "Python.h":
char *v, Py_ssize_t len) except NULL
+def io_thread_count():
+ """
+ Return the number of threads to use for I/O operations.
+
+ The number of threads is set to a fixed value at startup. It can
+ be modified at runtime by calling :func:`set_io_thread_count()`.
+ """
+ return GetIOThreadPoolCapacity()
+
+
+def set_io_thread_count(int count):
+ """Set the number of threads to use for I/O operations."""
Review comment:
Done, I cross-referenced the two pairs of functions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #10316: ARROW-12760: [C++][Python][R] Allow setting I/O thread pool size
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on a change in pull request #10316:
URL: https://github.com/apache/arrow/pull/10316#discussion_r633564645
##########
File path: python/pyarrow/io.pxi
##########
@@ -42,6 +42,23 @@ cdef extern from "Python.h":
char *v, Py_ssize_t len) except NULL
+def io_thread_count():
+ """
+ Return the number of threads to use for I/O operations.
+
+ The number of threads is set to a fixed value at startup. It can
+ be modified at runtime by calling :func:`set_io_thread_count()`.
+ """
+ return GetIOThreadPoolCapacity()
+
+
+def set_io_thread_count(int count):
+ """Set the number of threads to use for I/O operations."""
Review comment:
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #10316: ARROW-12760: [C++][Python][R] Allow setting I/O thread pool size
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on a change in pull request #10316:
URL: https://github.com/apache/arrow/pull/10316#discussion_r633456979
##########
File path: python/pyarrow/io.pxi
##########
@@ -42,6 +42,23 @@ cdef extern from "Python.h":
char *v, Py_ssize_t len) except NULL
+def io_thread_count():
+ """
+ Return the number of threads to use for I/O operations.
+
+ The number of threads is set to a fixed value at startup. It can
+ be modified at runtime by calling :func:`set_io_thread_count()`.
+ """
+ return GetIOThreadPoolCapacity()
+
+
+def set_io_thread_count(int count):
+ """Set the number of threads to use for I/O operations."""
Review comment:
Maybe can add a "See also" reference to `set_cpu_count` ?
##########
File path: python/pyarrow/io.pxi
##########
@@ -42,6 +42,23 @@ cdef extern from "Python.h":
char *v, Py_ssize_t len) except NULL
+def io_thread_count():
+ """
+ Return the number of threads to use for I/O operations.
+
+ The number of threads is set to a fixed value at startup. It can
+ be modified at runtime by calling :func:`set_io_thread_count()`.
+ """
+ return GetIOThreadPoolCapacity()
+
+
+def set_io_thread_count(int count):
+ """Set the number of threads to use for I/O operations."""
Review comment:
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #10316: ARROW-12760: [C++][Python][R] Allow setting I/O thread pool size
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #10316:
URL: https://github.com/apache/arrow/pull/10316#issuecomment-840640398
https://issues.apache.org/jira/browse/ARROW-12760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org