[jira] [Updated] (BEAM-9380) Python: ReadFromDatastore Embedded Entities

["Jeff Webb (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-9380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jeff Webb updated BEAM-9380:
----------------------------
    Status: Open  (was: Triage Needed)

> Python: ReadFromDatastore Embedded Entities
> -------------------------------------------
>
>                 Key: BEAM-9380
>                 URL: https://issues.apache.org/jira/browse/BEAM-9380
>             Project: Beam
>          Issue Type: Bug
>          Components: io-py-gcp
>    Affects Versions: 2.17.0, 2.18.0, 2.19.0
>            Reporter: Quentin
>            Priority: P3
>
> [Issue 8405|https://issues.apache.org/jira/browse/BEAM-8405] discussed the possibility to support embedded entities when using the conversion methods to/from of the client entity type.
> This feature was added in [PR 9805|[https://github.com/apache/beam/pull/9805]], which was shipped in 2.17.
> However, there seems to be an issue when Datastore embedded entities do not have a key.
>  Keys in embedded entities are optional.
> Because of this, when using the apache_beam.io.gcp.datastore.v1new.datastoreio.ReadFromDatastore transform, the pipeline fails with the following stacktrace:
> {code:java}
>   File "apache_beam/runners/common.py", line 780, in apache_beam.runners.common.DoFnRunner.process
>   File "apache_beam/runners/common.py", line 440, in apache_beam.runners.common.SimpleInvoker.invoke_process
>   File "apache_beam/runners/common.py", line 895, in apache_beam.runners.common._OutputProcessor.process_outputs
>   File "/Users/quentin/Work/sensome/dev/senback/dataflow/senback-reporting-db-import-flow/venv/lib/python3.7/site-packages/apache_beam/io/gcp/datastore/v1new/datastoreio.py", line 264, in process
>     yield types.Entity.from_client_entity(client_entity)
>   File "/Users/quentin/Work/sensome/dev/senback/dataflow/senback-reporting-db-import-flow/venv/lib/python3.7/site-packages/apache_beam/io/gcp/datastore/v1new/types.py", line 225, in from_client_entity
>     value = Entity.from_client_entity(value)
>   File "/Users/quentin/Work/sensome/dev/senback/dataflow/senback-reporting-db-import-flow/venv/lib/python3.7/site-packages/apache_beam/io/gcp/datastore/v1new/types.py", line 219, in from_client_entity
>     Key.from_client_key(client_entity.key),
>   File "/Users/quentin/Work/sensome/dev/senback/dataflow/senback-reporting-db-import-flow/venv/lib/python3.7/site-packages/apache_beam/io/gcp/datastore/v1new/types.py", line 156, in from_client_key
>     return Key(client_key.flat_path, project=client_key.project,
> AttributeError: 'NoneType' object has no attribute 'flat_path' [while running 'Read from Datastore/Read']
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)