[GitHub] [beam] damccorm opened a new issue, #20057: Python: ReadFromDatastore Embedded Entities

[GitBox <gi...@apache.org>]

damccorm opened a new issue, #20057:
URL: https://github.com/apache/beam/issues/20057

   [Issue 8405](https://issues.apache.org/jira/browse/BEAM-8405) discussed the possibility to support embedded entities when using the conversion methods to/from of the client entity type.
   
   This feature was added in [PR 9805]([https://github.com/apache/beam/pull/9805)], which was shipped in 2.17.
   
   However, there seems to be an issue when Datastore embedded entities do not have a key.
    Keys in embedded entities are optional.
   
   Because of this, when using the apache_beam.io.gcp.datastore.v1new.datastoreio.ReadFromDatastore transform, the pipeline fails with the following stacktrace:
   ```
   
     File "apache_beam/runners/common.py", line 780, in apache_beam.runners.common.DoFnRunner.process
   
    File "apache_beam/runners/common.py", line 440, in apache_beam.runners.common.SimpleInvoker.invoke_process
   
    File "apache_beam/runners/common.py", line 895, in apache_beam.runners.common._OutputProcessor.process_outputs
   
    File "/Users/quentin/Work/sensome/dev/senback/dataflow/senback-reporting-db-import-flow/venv/lib/python3.7/site-packages/apache_beam/io/gcp/datastore/v1new/datastoreio.py",
   line 264, in process
       yield types.Entity.from_client_entity(client_entity)
     File "/Users/quentin/Work/sensome/dev/senback/dataflow/senback-reporting-db-import-flow/venv/lib/python3.7/site-packages/apache_beam/io/gcp/datastore/v1new/types.py",
   line 225, in from_client_entity
       value = Entity.from_client_entity(value)
     File "/Users/quentin/Work/sensome/dev/senback/dataflow/senback-reporting-db-import-flow/venv/lib/python3.7/site-packages/apache_beam/io/gcp/datastore/v1new/types.py",
   line 219, in from_client_entity
       Key.from_client_key(client_entity.key),
     File "/Users/quentin/Work/sensome/dev/senback/dataflow/senback-reporting-db-import-flow/venv/lib/python3.7/site-packages/apache_beam/io/gcp/datastore/v1new/types.py",
   line 156, in from_client_key
       return Key(client_key.flat_path, project=client_key.project,
   AttributeError:
   'NoneType' object has no attribute 'flat_path' [while running 'Read from Datastore/Read']
   
   ```
   
    
   
   Imported from Jira [BEAM-9380](https://issues.apache.org/jira/browse/BEAM-9380). Original Jira may contain additional context.
   Reported by: Cavalié.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org