You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/05/26 16:13:19 UTC
[GitHub] [arrow] Mgmaplus opened a new issue #10408: ipc attribute is not found
Mgmaplus opened a new issue #10408:
URL: https://github.com/apache/arrow/issues/10408
Hi,
I'm using the bigquery_storage_v1 and I have an error when it calls read_schema.
It seems this function is in pyarrow.lib
I have pyarrow 4.0 installed which makes me think I need to install an earlier release.
Any help is appreciated.
<!--StartFragment-->
[ERROR] AttributeError: module 'pyarrow' has no attribute 'ipc'Traceback (most recent call last): File "/var/task/lambda_handler.py", line 89, in lambda_handler raise err File "/var/task/lambda_handler.py", line 80, in lambda_handler report_json = fetchPlatform(job['platform'], job['credentials'], job['ranges'], date_ranges, job['labels_to_add']) File "/var/task/generators/gen_reports_rev.py", line 81, in fetchPlatform listOfFigs = BQ_report_builder(cred_dict, listOfDateRanges[i][0], listOfDateRanges[i][1], labelsToAdd) File "/var/task/generators/gen_reports_rev.py", line 583, in BQ_report_builder df = BIGQ_downloadLastDay(json_cert, projectID, bucketID, params, date = end) File "/var/task/generators/data_download.py", line 604, in BIGQ_downloadLastDay dataframe = (bqclient.query(query_string).result().to_datafram
e()) File "/mnt/efs/__python_packages__/google/cloud/bigquery/table.py", line 1852, in to_dataframe record_batch = self.to_arrow( File "/mnt/efs/__python_packages__/google/cloud/bigquery/table.py", line 1667, in to_arrow for record_batch in self._to_arrow_iterable( File "/mnt/efs/__python_packages__/google/cloud/bigquery/table.py", line 1567, in _to_page_iterable yield from result_pages File "/mnt/efs/__python_packages__/google/cloud/bigquery/_pandas_helpers.py", line 717, in _download_table_bqstorage future.result() File "/var/lang/lib/python3.8/concurrent/futures/_base.py", line 437, in result return self.__get_result() File "/var/lang/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result raise self._exception File "/var/lang/lib/python3.8/concurrent/fut
ures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/mnt/efs/__python_packages__/google/cloud/bigquery/_pandas_helpers.py", line 598, in _download_table_bqstorage_stream item = page_to_item(page) File "/mnt/efs/__python_packages__/google/cloud/bigquery/_pandas_helpers.py", line 581, in _bqstorage_page_to_arrow return page.to_arrow() File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 453, in to_arrow return self._stream_parser.to_arrow(self._message) File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 626, in to_arrow return self._parse_arrow_message(message) File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 650, in _parse_arrow_message self._parse_arrow_schema()&
nbsp; File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 661, in _parse_arrow_schema self._schema = pyarrow.ipc.read_schema( | [ERROR] AttributeError: module 'pyarrow' has no attribute 'ipc' Traceback (most recent call last): File "/var/task/lambda_handler.py", line 89, in lambda_handler raise err File "/var/task/lambda_handler.py", line 80, in lambda_handler report_json = fetchPlatform(job['platform'], job['credentials'], job['ranges'], date_ranges, job['labels_to_add']) File "/var/task/generators/gen_reports_rev.py", line 81, in fetchPlatform listOfFigs = BQ_report_builder(cred_dict, listOfDateRanges[i][0], listOfDateRanges[i][1], labelsToAdd) File "/var/task/generators/gen_reports_rev.py", line 583, in BQ_report_builder df = BIGQ_downloadLastDay(json_cert, projectID, bucketID,
params, date = end) File "/var/task/generators/data_download.py", line 604, in BIGQ_downloadLastDay dataframe = (bqclient.query(query_string).result().to_dataframe()) File "/mnt/efs/__python_packages__/google/cloud/bigquery/table.py", line 1852, in to_dataframe record_batch = self.to_arrow( File "/mnt/efs/__python_packages__/google/cloud/bigquery/table.py", line 1667, in to_arrow for record_batch in self._to_arrow_iterable( File "/mnt/efs/__python_packages__/google/cloud/bigquery/table.py", line 1567, in _to_page_iterable yield from result_pages File "/mnt/efs/__python_packages__/google/cloud/bigquery/_pandas_helpers.py", line 717, in _download_table_bqstorage future.result() File "/var/lang/lib/python3.8/concurrent/futures/_base.py", line 437, in result return self.__ge
t_result() File "/var/lang/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result raise self._exception File "/var/lang/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/mnt/efs/__python_packages__/google/cloud/bigquery/_pandas_helpers.py", line 598, in _download_table_bqstorage_stream item = page_to_item(page) File "/mnt/efs/__python_packages__/google/cloud/bigquery/_pandas_helpers.py", line 581, in _bqstorage_page_to_arrow return page.to_arrow() File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 453, in to_arrow return self._stream_parser.to_arrow(self._message) File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 626, in to_arrow &nb
sp;return self._parse_arrow_message(message) File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 650, in _parse_arrow_message self._parse_arrow_schema() File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 661, in _parse_arrow_schema self._schema = pyarrow.ipc.read_schema(
-- | --
<!--EndFragment-->
STOP! Are you reporting a bug, a possible bug, or requesting a
feature? If so, please report under the ARROW project on the ASF JIRA
server https://issues.apache.org/jira/browse/ARROW. This JIRA server
is free to use and open to the public, but you must create an account
if it is your first time.
See our contribution guidelines for more information:
http://arrow.apache.org/docs/developers/contributing.html
We have GitHub issues available as a way for new contributors and
passers-by who are unfamiliar with Apache Software Foundation projects
to ask questions and interact with the project. Do not be surprised if
the first response is to open a JIRA issue or to write an e-mail to
one of the public mailing lists:
* Development discussions: dev@arrow.apache.org (first subscribe by
sending an e-mail to dev-subscribe@arrow.apache.org).
* User discussions: user@arrow.apache.org (first subscribe by
sending an e-mail to user-subscribe@arrow.apache.org).
Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] Mgmaplus commented on issue #10408: ipc attribute is not found
Posted by GitBox <gi...@apache.org>.
Mgmaplus commented on issue #10408:
URL: https://github.com/apache/arrow/issues/10408#issuecomment-851558679
It was just a matter of reinstalling the dependecies something in the env was not right
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on issue #10408: ipc attribute is not found
Posted by GitBox <gi...@apache.org>.
wesm commented on issue #10408:
URL: https://github.com/apache/arrow/issues/10408#issuecomment-850513744
Hi, it looks like you may have a file named `pyarrow.py` in your project. Can you confirm?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] Mgmaplus closed issue #10408: ipc attribute is not found
Posted by GitBox <gi...@apache.org>.
Mgmaplus closed issue #10408:
URL: https://github.com/apache/arrow/issues/10408
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] Mgmaplus removed a comment on issue #10408: ipc attribute is not found
Posted by GitBox <gi...@apache.org>.
Mgmaplus removed a comment on issue #10408:
URL: https://github.com/apache/arrow/issues/10408#issuecomment-851558679
It was just a matter of reinstalling the dependecies something in the env was not right
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] Mgmaplus commented on issue #10408: ipc attribute is not found
Posted by GitBox <gi...@apache.org>.
Mgmaplus commented on issue #10408:
URL: https://github.com/apache/arrow/issues/10408#issuecomment-851558916
It was just a matter of reinstalling the dependecies something in the env was not right
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org