You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/09/08 15:26:53 UTC
[GitHub] [beam] diRblo opened a new issue, #23090: [Bug] Python Mongodb IO error when loading collection with single document
diRblo opened a new issue, #23090:
URL: https://github.com/apache/beam/issues/23090
**What happened?**
When I tried to read from a mongodb collection with only a single document in it, the worker returns "float division by zero" error. The source database is deployed in mongodb atlas and the pipeline is deployed on gcp dataflow runner. Below is the code we used and the error message got:
**Code**:
`pipeline | ReadFromMongoDB(uri=..., db=..., coll=..., bucket_auto=True)`
**ERROR**:
> Error message from worker: Traceback (most recent call last):
> File "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line 646, in do_work
> work_executor.execute()
> File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py", line 208, in execute
> op.start()
> File "dataflow_worker/native_operations.py", line 38, in dataflow_worker.native_operations.NativeReadOperation.start
> File "dataflow_worker/native_operations.py", line 40, in dataflow_worker.native_operations.NativeReadOperation.start
> File "dataflow_worker/native_operations.py", line 44, in dataflow_worker.native_operations.NativeReadOperation.start
> File "dataflow_worker/native_operations.py", line 49, in dataflow_worker.native_operations.NativeReadOperation.start
> File "/usr/local/lib/python3.7/site-packages/dataflow_worker/workercustomsources.py", line 112, in get_progress
> fraction = self._range_tracker.fraction_consumed()
> File "/usr/local/lib/python3.7/site-packages/apache_beam/io/range_trackers.py", line 286, in fraction_consumed
> self._last_claim, self._start_position, self._stop_position)
> File "/usr/local/lib/python3.7/site-packages/apache_beam/io/range_trackers.py", line 432, in position_to_fraction
> return float(ikey - istart) / (iend - istart)
> ZeroDivisionError: float division by zero
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #23090: [Bug] Python Mongodb IO error when loading collection with single document
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #23090:
URL: https://github.com/apache/beam/issues/23090#issuecomment-1246818173
.add-labels python, io, mongodb, 'awaiting triage'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #23090: [Bug] Python Mongodb IO error when loading collection with single document
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #23090:
URL: https://github.com/apache/beam/issues/23090#issuecomment-1258451057
Could not reproduce when reading with single document by running
`python apache_beam/io/mongodbio_it_test.py --num_documents=1` or `python apache_beam/io/mongodbio_it_test.py --num_documents=1 --batch_size=1`
setting break point at range tracker [here](https://github.com/apache/beam/blob/3a4d57eb8976c5f503b32d478a80b1800490f66f/sdks/python/apache_beam/io/mongodbio.py#L367) get the following logging:
```
raise RuntimeError(str(start_position) + " " + str(stop_position))
6331ef8116e9cab1252f4558 6331ef8116e9cab1252f4559 [while running 'ReadFromMongoDB/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction']
```
As can be seen start_positin and stop_position still differ by 1 when there is single document.
@diRblo Is it possible to provide a minimum working example that reproduces the issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #23090: [Bug] Python Mongodb IO error when loading collection with single document
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #23090:
URL: https://github.com/apache/beam/issues/23090#issuecomment-1242083831
Could you please provide the version of Beam SDK used? Also if possible check if the latest version has the same issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on issue #23090: [Bug] Python Mongodb IO error when loading collection with single document
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #23090:
URL: https://github.com/apache/beam/issues/23090#issuecomment-1246961646
Label cannot be managed because it does not exist in the repo. Please check your spelling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #23090: [Bug] Python Mongodb IO error when loading collection with single document
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #23090:
URL: https://github.com/apache/beam/issues/23090#issuecomment-1242085778
.add-labels "python", "io", "mongodb", "awaiting triage"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #23090: [Bug] Python Mongodb IO error when loading collection with single document
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #23090:
URL: https://github.com/apache/beam/issues/23090#issuecomment-1242161143
add-labels python, io, mongodb, 'awaiting triage'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on issue #23090: [Bug] Python Mongodb IO error when loading collection with single document
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #23090:
URL: https://github.com/apache/beam/issues/23090#issuecomment-1242179160
Label "python" cannot be managed because it does not exist in the repo. Please check your spelling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] diRblo commented on issue #23090: [Bug] Python Mongodb IO error when loading collection with single document
Posted by GitBox <gi...@apache.org>.
diRblo commented on issue #23090:
URL: https://github.com/apache/beam/issues/23090#issuecomment-1246913017
> Could you please provide the version of Beam SDK used? Also if possible check if the latest version has the same issue?
We are using the latest version 2.41.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org