You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "avm19 (via GitHub)" <gi...@apache.org> on 2025/08/20 20:50:07 UTC
[I] python/adbc_driver_manager: executemany requires argument of positive length in 1.7.0 [arrow-adbc]
avm19 opened a new issue, #3319:
URL: https://github.com/apache/arrow-adbc/issues/3319
### What happened?
Starting from 1.7.0, `cursor.executemany(operation, seq_of_parameters)` throws `adbc_driver_manager.ProgrammingError: INVALID_ARGUMENT: [libpq] Failed to execute query ...` when `seq_of_parameters` has length 0, e.g. an empty list.
This is not such a big deal, but annoying, as did not have to worry about this edge case in previous versions. I checked release notes and found no mention of this change. I assume, this was unintentional and hence a bug.
### Stack Trace
```text
Traceback (most recent call last):
File "....bug_adbc_executemany_empty.py", line 17, in <module>
cursor.executemany(
~~~~~~~~~~~~~~~~~~^
"INSERT INTO test1 (id, val) VALUES ($1, $2);",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[], # FAILS IN 1.7.0 but works in 1.5.0
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File ".../lib/python3.13/site-packages/adbc_driver_manager/dbapi.py", line 771, in executemany
self._rowcount = _blocking_call(
~~~~~~~~~~~~~~^
self._stmt.execute_update, (), {}, self._stmt.cancel
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "adbc_driver_manager/_lib.pyx", line 1674, in adbc_driver_manager._lib._blocking_call_impl
File "adbc_driver_manager/_lib.pyx", line 1667, in adbc_driver_manager._lib._blocking_call_impl
File "adbc_driver_manager/_lib.pyx", line 1391, in adbc_driver_manager._lib.AdbcStatement.execute_update
File "adbc_driver_manager/_lib.pyx", line 261, in adbc_driver_manager._lib.check_error
adbc_driver_manager.ProgrammingError: INVALID_ARGUMENT: [libpq] Failed to execute query 'INSERT INTO test1 (id, val) VALUES ($1, $2);': ERROR: there is no parameter $1
LINE 1: INSERT INTO test1 (id, val) VALUES ($1, $2);
^
. SQLSTATE: 42P02
```
### How can we reproduce the bug?
```python
with adbc_driver_postgresql.dbapi.connect(uri) as conn:
with conn.cursor() as cursor:
cursor.execute("CREATE TEMP TABLE test1 (id INTEGER, val INTEGER);")
# This works in 1.5.0, 1.6.0 and 1.7.0:
cursor.executemany("INSERT INTO test1 (id, val) VALUES ($1, $2);", [(1, 1), (2, 2)])
# This fails for adbc_driver_manager==1.7.0, but works in previous versions:
cursor.executemany("INSERT INTO test1 (id, val) VALUES ($1, $2);", [])
```
### Environment/Setup
I use Postgres, but this might be irrelevant, not sure.
Out of these three combinations of versions, only the last one shows the bug:
```bash
pip install "adbc-driver-manager==1.6.0" "adbc-driver-postgresql==1.7.0";
pip install "adbc-driver-manager==1.6.0" "adbc-driver-postgresql==1.6.0";
pip install "adbc-driver-manager==1.7.0" "adbc-driver-postgresql==1.6.0";
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] python/adbc_driver_manager: executemany requires argument of positive length in 1.7.0 [arrow-adbc]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #3319:
URL: https://github.com/apache/arrow-adbc/issues/3319#issuecomment-3208535497
Hmm, what was the behavior before? The parameters would implicitly be NULL?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] python/adbc_driver_manager: executemany requires argument of positive length in 1.7.0 [arrow-adbc]
Posted by "avm19 (via GitHub)" <gi...@apache.org>.
avm19 commented on issue #3319:
URL: https://github.com/apache/arrow-adbc/issues/3319#issuecomment-3208127593
I think this was the change:
https://github.com/apache/arrow-adbc/commit/7804b2c47debc1740981658a12039eafe4733ada#diff-aac0b59df381ba9a1f9232bd15d9f12610bb80beb238eb51901febba5fabc066R766
https://github.com/apache/arrow-adbc/blob/7804b2c47debc1740981658a12039eafe4733ada/python/adbc_driver_manager/adbc_driver_manager/dbapi.py#L759-L769
`bind()` used to be without "if".
Just thinking out loud. I guess this issue would not have arisen if
```python
elif seq_of_parameters:
```
were replaced with
```python
elif seq_of_parameters is not None:
```
I also guess that the consideration was to avoid creating an empty arrow batch. Anyways, it would be great if the library took care of the empty sequence case, so that I would not have to.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] python/adbc_driver_manager: executemany requires argument of positive length in 1.7.0 [arrow-adbc]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #3319:
URL: https://github.com/apache/arrow-adbc/issues/3319#issuecomment-3222760767
Ok, it seems the old behavior was to treat `None` and `[]` as to effectively not execute the query, while the new behavior is to treat them both as executing the query once.
I believe the "right" behavior should be that None is the same as executing the query once, and `[]` is the same as not executing the query at all.
Looking at PEP-249, it states:
> Modules are free to implement this method using multiple calls to the [.execute()](https://peps.python.org/pep-0249/#id21) method
I'm interpreting this as saying that semantically, it should be like
```python
for paramset in parameters:
self.execute(query, paramset)
```
And so an empty list corresponds to no behavior.
By the spec, I don't think `None` is actually defined. `psycopg` appears to reject None, while also treating `[]` the same as not executing the query.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] python/adbc_driver_manager: executemany requires argument of positive length in 1.7.0 [arrow-adbc]
Posted by "avm19 (via GitHub)" <gi...@apache.org>.
avm19 commented on issue #3319:
URL: https://github.com/apache/arrow-adbc/issues/3319#issuecomment-3208628889
Before, I was able to do `cursor.executemany("INSERT INTO test1 (id, val) VALUES ($1, $2);", [])` without any errors.
From looking at the old Python code, it seems that the passed empty Python list `[]` would be automatically replaced by an empty batch `pyarrow.record_batch([])` and then bound as a parameter to the prepared statement. (I have no idea what happens next at a deeper level, in C code or in libpq). In the new code no binding seems to take place, as the passed empty list `[]` is replaced by `None`.
From reading the docstring of `executemany()`, I take away that parameter value `None` has a special meaning, separate from `[]`. But `elif seq_of_parameters: ` makes no distinction between them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] python/adbc_driver_manager: executemany requires argument of positive length in 1.7.0 [arrow-adbc]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #3319:
URL: https://github.com/apache/arrow-adbc/issues/3319#issuecomment-3222373479
Ah. I think before, an empty list effectively meant that the query would silently _not be executed_. I suppose that is technically correct, and either behavior is liable to be surprising in some contexts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] python/adbc_driver_manager: executemany requires argument of positive length in 1.7.0 [arrow-adbc]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm closed issue #3319: python/adbc_driver_manager: executemany requires argument of positive length in 1.7.0
URL: https://github.com/apache/arrow-adbc/issues/3319
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org