You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/08/17 03:39:07 UTC
[GitHub] [hudi] h117561964 opened a new issue #1973: Incremental query on COW table returns empty records.
h117561964 opened a new issue #1973:
URL: https://github.com/apache/hudi/issues/1973
Before doing incremental query, I can get 360 records from COW table with below Hive sql.
`select count(1) from poc_attribute where `_hoodie_commit_time` > '20200814235710';`
But after I execute Hive commands to do incremental query with the same sql, the same commit time, I get empty return.
`set hoodie.poc_attribute.consume.start.timestamp=20200814235710;`
`set hoodie.poc_attribute.consume.mode=INCREMENTAL;`
`set hoodie.poc_attribute.consume.max.commits=3;`
I am wondering if my understanding about incremental query is correct, so please guide me what I am doing wrong here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] satishkotha commented on issue #1973: [SUPPORT] Incremental query on COW table returns empty records.
Posted by GitBox <gi...@apache.org>.
satishkotha commented on issue #1973:
URL: https://github.com/apache/hudi/issues/1973#issuecomment-770583049
@nsivabalan see here for hive config attributes: https://github.com/apache/hudi/blob/master/hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieHiveUtils.java#L40
The one you pointed out is for spark datasource API.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] bvaradar commented on issue #1973: [SUPPORT] Incremental query on COW table returns empty records.
Posted by GitBox <gi...@apache.org>.
bvaradar commented on issue #1973:
URL: https://github.com/apache/hudi/issues/1973#issuecomment-678239716
Added https://issues.apache.org/jira/browse/HUDI-1210
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on issue #1973: [SUPPORT] Incremental query on COW table returns empty records.
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on issue #1973:
URL: https://github.com/apache/hudi/issues/1973#issuecomment-770580791
@satishkotha : I am trying to fix all documentation related jiras. Can you throw some light on which config is this exactly. All I could see is "hoodie.datasource.read.begin.instanttime" [here](https://hudi.apache.org/docs/configurations.html#read-options). But guess you mentioned about some other config. Can you point me in code where this config is used.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] bhasudha commented on issue #1973: [SUPPORT] Incremental query on COW table returns empty records.
Posted by GitBox <gi...@apache.org>.
bhasudha commented on issue #1973:
URL: https://github.com/apache/hudi/issues/1973#issuecomment-675060988
@h117561964 This should work. Wondering what is wrong here. What version of hudi are you using ? Could you paste your beeline or hive cli config / command. And is it possible for you to get the hive log ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] satishkotha commented on issue #1973: [SUPPORT] Incremental query on COW table returns empty records.
Posted by GitBox <gi...@apache.org>.
satishkotha commented on issue #1973:
URL: https://github.com/apache/hudi/issues/1973#issuecomment-675087028
@h117561964 Just fyi, specified start.timestamp is excluded for incremental queries. Try
set hoodie.poc_attribute.consume.start.timestamp=202008142357**09**;
I debugged similar issue earlier. This is not documented afaik. Feel free to send PR with updates to documentation if this works for you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] bvaradar closed issue #1973: [SUPPORT] Incremental query on COW table returns empty records.
Posted by GitBox <gi...@apache.org>.
bvaradar closed issue #1973:
URL: https://github.com/apache/hudi/issues/1973
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org