You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/04 01:54:36 UTC

[GitHub] [beam] kennknowles opened a new issue, #19499: HCatalogTableProvider supports filter pushdown

kennknowles opened a new issue, #19499:
URL: https://github.com/apache/beam/issues/19499

   Hi,
   
   I'm using HCatalogTableProvider while doing SqlTransform.query. The query is something like "select * from `hive`.`table_name` limit 10". Despite of the limit clause, the data source still reads much more rows (the data of Hive table are files on S3), even more than the number of rows in one file (or partition).
   
    
   
   Some more details:
    - It is running on Flink.
    - I actually implemented my own HiveTableProvider because HCatalogBeamSchema only supports primitive types. However, the table provider works when I query a small table with ~1k rows.
   
   Imported from Jira [BEAM-6874](https://issues.apache.org/jira/browse/BEAM-6874). Original Jira may contain additional context.
   Reported by: Near.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org