[GitHub] [hudi] gtwuser commented on issue #5891: How to read existing hoodie written data from `S3` using `AWS Glue DynamicFrame` class. Failing with error with below error: An error occurred while calling o84.getDynamicFrame. s3://xxxx/.hoodie/202212312312.commit is not a Parquet file. expected magic number at tail

[GitBox <gi...@apache.org>]

gtwuser commented on issue #5891:
URL: https://github.com/apache/hudi/issues/5891#issuecomment-1161388537

   > @gtwuser I am not very familiar with Glue APIs. But, can you share the full stacktrace? Also, don't you need to give `connection_type` while creating dataframe? I am referring to https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-crawler-pyspark-extensions-dynamic-frame-reader.html#aws-glue-api-crawler-pyspark-extensions-dynamic-frame-reader-from_options
   
   @codope sorry for the delay and thanks for your response, also you are **right**  we do need `connection_type` but thats when we are creating `DynamicFrame` instance using `from_option` method, here im using `from_catalog` so we wont need it. [Ref_link](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-crawler-pyspark-extensions-dynamic-frame-reader.html#aws-glue-api-crawler-pyspark-extensions-dynamic-frame-reader-from_catalog)
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org