You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "mcrumiller (via GitHub)" <gi...@apache.org> on 2024/10/24 15:42:22 UTC
[I] Expected PGCOPY signature of 11 bytes at beginning of stream but found -1 bytes of input [arrow-adbc]
mcrumiller opened a new issue, #2276:
URL: https://github.com/apache/arrow-adbc/issues/2276
### What happened?
This is a semi-duplicate of #2133. That issue was closed by a PR that apparently improves the error message, but it seems as though it doesn't resolve the issue itself in that instance.
In my case, I'd like to diagnose what's causing the error (if possible) as I'd like to actually get this working--if there's a workaround possible on my end I would like to pursue that possibility before waiting for a new version of ADBC. I'm not sure where the ADBC log file are stored and I can't find in the documentation where I might look.
When I run my query (via polars), I get:
```
IO: [libpq] ReadHeader failed: Expected PGCOPY signature of 11 bytes at beginning of stream but found -1 bytes of input
```
If I add a `LIMIT 10000` the query succeeds, so the issue is either in a later record in the data, or something else that I can't think of. I expect 1,219,228 total records.. Can someone possibly help me diagnose the issue?
### Stack Trace
```
File "C:\Projects\project-cqn\.venv_cqn\Lib\site-packages\adbc_driver_manager\_reader.pyx", line 89, in adbc_driver_manager._reader.AdbcRecordBatchReader.read_all
return self._reader.read_all()
File "C:\Projects\project-cqn\.venv_cqn\Lib\site-packages\pyarrow\ipc.pxi", line 762, in pyarrow.lib.RecordBatchReader.read_all
check_status(self.reader.get().ToTable().Value(&table))
File "C:\Projects\project-cqn\.venv_cqn\Lib\site-packages\pyarrow\error.pxi", line 92, in pyarrow.lib.check_status
raise convert_status(status)
OSError: [libpq] ReadHeader failed: Expected PGCOPY signature of 11 bytes at beginning of stream but found -1 bytes of input
During handling of the above exception, another exception occurred:
File "C:\Projects\project-cqn\.venv_cqn\Lib\site-packages\adbc_driver_manager\_reader.pyx", line 41, in adbc_driver_manager._reader._AdbcErrorHelper.check_error
raise exc from None
File "C:\Projects\project-cqn\.venv_cqn\Lib\site-packages\adbc_driver_manager\_reader.pyx", line 91, in adbc_driver_manager._reader.AdbcRecordBatchReader.read_all
self._helper.check_error(e)
File "C:\Projects\project-cqn\.venv_cqn\Lib\site-packages\adbc_driver_manager\_lib.pyx", line 1590, in adbc_driver_manager._lib._blocking_call
return func(*args, **kwargs)
File "C:\Projects\project-cqn\.venv_cqn\Lib\site-packages\adbc_driver_manager\dbapi.py", line 1197, in fetch_arrow_table
return _blocking_call(self._reader.read_all, (), {}, self._stmt.cancel)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Projects\project-cqn\.venv_cqn\Lib\site-packages\adbc_driver_manager\dbapi.py", line 1088, in fetch_arrow_table
return self._results.fetch_arrow_table()
```
### How can we reproduce the bug?
It's a fairly complex query. I could perhaps work to reproduce but since it works on 10k records, it may be difficult to make a repro.
### Environment/Setup
```
greenplum/postgres PostgreSQL 9.4.26 (Greenplum Database 6.24.3 build commit:25d3498a400ca5230e81abb94861f23389315213) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on May 3 2023 20:34:57
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Expected PGCOPY signature of 11 bytes at beginning of stream but found -1 bytes of input [arrow-adbc]
Posted by "mcrumiller (via GitHub)" <gi...@apache.org>.
mcrumiller commented on issue #2276:
URL: https://github.com/apache/arrow-adbc/issues/2276#issuecomment-2436530704
Is that some anonymous third-party website providing builds? I'd rather not go through that route. I'll do more investigating on my side. Thank you though!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Expected PGCOPY signature of 11 bytes at beginning of stream but found -1 bytes of input [arrow-adbc]
Posted by "mcrumiller (via GitHub)" <gi...@apache.org>.
mcrumiller commented on issue #2276:
URL: https://github.com/apache/arrow-adbc/issues/2276#issuecomment-2436257985
Or actually, is there a pip release for the nightlies?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Expected PGCOPY signature of 11 bytes at beginning of stream but found -1 bytes of input [arrow-adbc]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #2276:
URL: https://github.com/apache/arrow-adbc/issues/2276#issuecomment-2642002204
1.4.0 is available, does that version work or show any more diagnostic info?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Expected PGCOPY signature of 11 bytes at beginning of stream but found -1 bytes of input [arrow-adbc]
Posted by "mcrumiller (via GitHub)" <gi...@apache.org>.
mcrumiller commented on issue #2276:
URL: https://github.com/apache/arrow-adbc/issues/2276#issuecomment-2436257408
Sorry, I'm unable to build the nightlies at work. Unsure if you want me to re-open after the next release if I still see the issue, or if you want to just leave this one in limbo until then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Expected PGCOPY signature of 11 bytes at beginning of stream but found -1 bytes of input [arrow-adbc]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm closed issue #2276: Expected PGCOPY signature of 11 bytes at beginning of stream but found -1 bytes of input
URL: https://github.com/apache/arrow-adbc/issues/2276
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Expected PGCOPY signature of 11 bytes at beginning of stream but found -1 bytes of input [arrow-adbc]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on issue #2276:
URL: https://github.com/apache/arrow-adbc/issues/2276#issuecomment-2436192187
Does this still happen with the nightly Python builds? Since the last release we did a heavy refactor of some of those internals to make those types of errors report better/go away:
```
pip install \
--pre \
--index-url https://repo.fury.io/arrow-adbc-nightlies \
adbc-driver-postgresql
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Expected PGCOPY signature of 11 bytes at beginning of stream but found -1 bytes of input [arrow-adbc]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on issue #2276:
URL: https://github.com/apache/arrow-adbc/issues/2276#issuecomment-2436301534
I think there's wheels! For me it resolves to `https://repo.fury.io/arrow-adbc-nightlies/-/ver_1QpKJK/adbc_driver_postgresql-1.3.0-py3-none-macosx_11_0_arm64.whl` but no idea if that will work for you. Installing dev versions is not great right now, sorry!
If you're up for another approach, figuring out which row/column causes the issue by manipulating the LIMIT clause might help pinpoint the issue. The error that was obscured by the last report of this was one where a computation failed on a specific value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Expected PGCOPY signature of 11 bytes at beginning of stream but found -1 bytes of input [arrow-adbc]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #2276:
URL: https://github.com/apache/arrow-adbc/issues/2276#issuecomment-4933496937
(Issue is stale, but please feel free to re-open if you are still experiencing this.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org