You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "davlee1972 (via GitHub)" <gi...@apache.org> on 2025/10/27 20:29:27 UTC
[I] adbc_driver_snowflake's adbc_stmt_kwargs broken after 1.8.0 release [arrow-adbc]
davlee1972 opened a new issue, #3625:
URL: https://github.com/apache/arrow-adbc/issues/3625
### What happened?
After the 1.8 release, the adbc_stmt_kwargs options don't seem to work at all and all the options run in default mode.
Here's a screenshot with the 1.7.0 driver with:
self.cursor_kwargs["adbc_stmt_kwargs"]["adbc.snowflake.statement.ingest_copy_concurrency"] = "1"
<img width="2098" height="463" alt="Image" src="https://github.com/user-attachments/assets/a0dd858a-35a2-45e1-a8be-ce085db7caea" />
This still looks odd since the behavior of "1" appears to be behaving like "0".
With 1.8 it looks like it it running multiple ingests in parallel with overlapping start and end times even though the setting is "1".
<img width="2043" height="701" alt="Image" src="https://github.com/user-attachments/assets/c5aadc37-f23c-4278-bec4-23e1c765db03" />
### Stack Trace
_No response_
### How can we reproduce the bug?
_No response_
### Environment/Setup
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] adbc_driver_snowflake's adbc_stmt_kwargs broken after 1.8.0 release [arrow-adbc]
Posted by "Mandukhai-Alimaa (via GitHub)" <gi...@apache.org>.
Mandukhai-Alimaa commented on issue #3625:
URL: https://github.com/apache/arrow-adbc/issues/3625#issuecomment-3454031077
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] adbc_driver_snowflake's adbc_stmt_kwargs broken after 1.8.0 release [arrow-adbc]
Posted by "Mandukhai-Alimaa (via GitHub)" <gi...@apache.org>.
Mandukhai-Alimaa commented on issue #3625:
URL: https://github.com/apache/arrow-adbc/issues/3625#issuecomment-3478695269
Hi @davelee1972, I tested this using the Python driver manager with adbc_stmt_kwargs applied, and the concurrency setting appears to be working correctly.
What I’m seeing:
- With default concurrency (the default is 4), multiple COPY operations start concurrently, as expected.
<img width="1408" height="392" alt="Image" src="https://github.com/user-attachments/assets/39a6929f-f63d-4a5a-8825-bf1747334b7b" />
- When I explicitly set "adbc.snowflake.statement.ingest_copy_concurrency": "1", the COPY operations run sequentially and wait for the previous one to finish. There’s no overlap in start/end timestamps.
<img width="1394" height="206" alt="Image" src="https://github.com/user-attachments/assets/9340c118-2088-4f06-a301-b976b0be3b3d" />
here is the snippet I used to control the concurrency.
```
cursor_kwargs = {
"adbc_stmt_kwargs": {
"adbc.snowflake.statement.ingest_copy_concurrency": concurrency,
}
}
with connection.cursor(**cursor_kwargs) as cursor:
```
If you can provide further clarification, happy to investigate more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] adbc_driver_snowflake's adbc_stmt_kwargs broken after 1.8.0 release [arrow-adbc]
Posted by "davlee1972 (via GitHub)" <gi...@apache.org>.
davlee1972 commented on issue #3625:
URL: https://github.com/apache/arrow-adbc/issues/3625#issuecomment-3482596173
Closing this issue.. I couldn't reproduce the error today.. I tried 100,000, 10 million and a final run with the entire 100 million row dataset. Each time the copy into(s) ran consecutively. I didn't make any code changes other than upgrading the driver from 1.7 back to 1.8..
<img width="2331" height="995" alt="Image" src="https://github.com/user-attachments/assets/5ff8a6e4-bfec-4fec-9e03-ee4f61066a79" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] adbc_driver_snowflake's adbc_stmt_kwargs broken after 1.8.0 release [arrow-adbc]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #3625:
URL: https://github.com/apache/arrow-adbc/issues/3625#issuecomment-3453842455
@Mandukhai-Alimaa any chance you could see what's going on here? Or look at the changelog and see if there's any commits that might be responsible?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] adbc_driver_snowflake's adbc_stmt_kwargs broken after 1.8.0 release [arrow-adbc]
Posted by "davlee1972 (via GitHub)" <gi...@apache.org>.
davlee1972 closed issue #3625: [Python] adbc_driver_snowflake's adbc_stmt_kwargs broken after 1.8.0 release
URL: https://github.com/apache/arrow-adbc/issues/3625
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org