You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "CurtHagenlocher (via GitHub)" <gi...@apache.org> on 2024/05/09 20:28:50 UTC
[I] A model for I/O utilization? [arrow-adbc]
CurtHagenlocher opened a new issue, #1848:
URL: https://github.com/apache/arrow-adbc/issues/1848
### What would you like help with?
I've been thinking a lot over the ~last week about asynchronous operation. It seems to me that the existing driver of all the language APIs is the C API, which has the drawback of being a bit "lowest common denominator" and therefore (given the current state of the world) completely synchronous. This means that there aren't any indications of which calls might more reasonably be expected to require I/O than others.
Here are the C API calls that seem to me most likely to come with an expectation of probable I/O:
ConnectionCommit
ConnectionGetInfo ?
ConnectionGetObjects
ConnectionGetTableSchema
ConnectionGetTableTypes ?
ConnectionInit
ConnectionReadPartition
ConnectionRelease
ConnectionRollback
ConnectionGetStatistics
StatementExecuteQuery
StatementExecutePartitions
StatementGetParameterSchema ?
StatementPrepare
StatementCancel ?
StatementExecuteSchema
ArrowArrayStream
I'm basing this list on a model whereby drivers, databases and statements are effectively purely in-memory. I'm not completely sure about something like SetOption; the options that are set before a connection is open, for instance, are clearly being set synchronously. But I don't know that we should make a strong statement about the behavior of something like `AdbcConnectionSetOption(ADBC_CONNECTION_OPTION_CURRENT_CATALOG, "catalogName")'; is it always just remembering a current catalog for the next request or is it allowed to change server state right away?
Curious to hear others' thoughts.
CC: #1843, #811
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] A model for I/O utilization? [arrow-adbc]
Posted by "CurtHagenlocher (via GitHub)" <gi...@apache.org>.
CurtHagenlocher commented on issue #1848:
URL: https://github.com/apache/arrow-adbc/issues/1848#issuecomment-2103470898
I guess in addition to setting the current catalog and schema, the other obvious candidate for a standard option that immediately changes server state would be autocommit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] A model for I/O utilization? [arrow-adbc]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on issue #1848:
URL: https://github.com/apache/arrow-adbc/issues/1848#issuecomment-2103404654
> But I don't know that we should make a strong statement about the behavior of something like `AdbcConnectionSetOption(ADBC_CONNECTION_OPTION_CURRENT_CATALOG, "catalogName")'; is it always just remembering a current catalog for the next request or is it allowed to change server state right away?
At least in our currently implementations, some of the options *do* change the server state right away instead of waiting for the next request. I believe both ways should be allowed, as long as the option is reflected in any subsequent queries. That would be my opinion at least
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org