You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "mitchej123 (via GitHub)" <gi...@apache.org> on 2024/02/23 03:22:25 UTC
[I] [Bug]: Pytest fails on a pipeline when importing from apache_beam.io.gcp.bigquery [beam]
mitchej123 opened a new issue, #30392:
URL: https://github.com/apache/beam/issues/30392
### What happened?
When running Python 3.11.8 and `pytest==8.0.1` the rest runner errors out with the following stacktrace:
```python
_______ ERROR collecting tests/application/unit/beam/test_preprocess_pipeline.py _______
../../.virtualenvs/project/lib/python3.11/site-packages/_pytest/python.py:538: in importtestmodule
mod = import_path(path, mode=importmode, root=config.rootpath)
../../.virtualenvs/project/lib/python3.11/site-packages/_pytest/pathlib.py:566: in import_path
importlib.import_module(module_name)
/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py:126: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1204: in _gcd_import
???
<frozen importlib._bootstrap>:1176: in _find_and_load
???
<frozen importlib._bootstrap>:1147: in _find_and_load_unlocked
???
<frozen importlib._bootstrap>:690: in _load_unlocked
???
../../.virtualenvs/project/lib/python3.11/site-packages/_pytest/assertion/rewrite.py:178: in exec_module
exec(co, module.__dict__)
tests/application/unit/beam/test_preprocess_pipeline.py:23: in <module>
from project.beam.common.pipeline_config import PipelineConfig
src/project/beam/common/pipeline_config.py:6: in <module>
from apache_beam import Create, WindowInto, window
../../.virtualenvs/project/lib/python3.11/site-packages/apache_beam/__init__.py:88: in <module>
from apache_beam import io
../../.virtualenvs/project/lib/python3.11/site-packages/apache_beam/io/__init__.py:36: in <module>
from apache_beam.io.gcp.bigquery import *
E File "/Users/jason/.virtualenvs/project/lib/python3.11/site-packages/apache_beam/io/gcp/bigquery.py", line 2640
E """Read data from BigQuery.
E ^^^^^^^
E SyntaxError: invalid escape sequence '\#'
```
The exact line it's failing on is [here](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/bigquery.py#L2709): `https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro\#avro_conversions`
This was introduced with pull request #29578 that made the lines longer. It appears it was originally wrapped and a continuation escape `\` was added at the end of the line, however when it was lengthened this was not removed. I'm unclear why it only seems to fail when running pytest, or what other versions of packages are required to reproduce this; however the fix is to remove the stray `\`.
### Issue Priority
Priority: 3 (minor)
### Issue Components
- [X] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Pytest fails on a pipeline when importing from apache_beam.io.gcp.bigquery [beam]
Posted by "mitchej123 (via GitHub)" <gi...@apache.org>.
mitchej123 commented on issue #30392:
URL: https://github.com/apache/beam/issues/30392#issuecomment-1962227841
.close-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Pytest fails on a pipeline when importing from apache_beam.io.gcp.bigquery [beam]
Posted by "mitchej123 (via GitHub)" <gi...@apache.org>.
mitchej123 commented on issue #30392:
URL: https://github.com/apache/beam/issues/30392#issuecomment-1960680818
.take-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Pytest fails on a pipeline when importing from apache_beam.io.gcp.bigquery [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #30392: [Bug]: Pytest fails on a pipeline when importing from apache_beam.io.gcp.bigquery
URL: https://github.com/apache/beam/issues/30392
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org