You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "pravin1406 (via GitHub)" <gi...@apache.org> on 2023/03/14 18:07:53 UTC
[GitHub] [hudi] pravin1406 opened a new issue, #8185: [SUPPORT] Getting older data using trino hudi connector, for real time MOR table .. whereas getting correct data in hive for same mor_rt table
pravin1406 opened a new issue, #8185:
URL: https://github.com/apache/hudi/issues/8185
**_Tips before filing an issue_**
- Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)?
- yes
- Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.
- If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly.
**Describe the problem you faced**
A clear and concise description of the problem.
I upserted some data into a MOR table using spark (java code) with hive sync enabled using HMS. When i read this data on spark shell using output path or db.employee_test_mor_rt , i get the correct output, records are read from .log files as expected and same from hive.
But on reading this data from trino for employee_test_mor_rt table, i get the output from last commits only and not from .log files of MOR table.
**To Reproduce**
Steps to reproduce the behavior:
1. Upsert data using spark multiple times , so there are multiple .log files with hive sync enabled using HMS/jdbc
2. Read through trino 410 (latest) using hudi connector
**Expected behavior**
A clear and concise description of what you expected to happen.
**Environment Description**
* Hudi version : 0.12.2
* Spark version : 3.2.0
* Hive version : 3.1.2_1
* Hadoop version : Hadoop 3.2.1
* Storage (HDFS/S3/GCS..) : HDFS
* Running on Docker? (yes/no) : no
**Additional context**
Add any other context about the problem here.
**Stacktrace**
```Add the stacktrace of the error.```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pravin1406 commented on issue #8185: [SUPPORT] Getting older data using trino hudi connector, for real time MOR table .. whereas getting correct data in hive for same mor_rt table
Posted by "pravin1406 (via GitHub)" <gi...@apache.org>.
pravin1406 commented on issue #8185:
URL: https://github.com/apache/hudi/issues/8185#issuecomment-1704374193
Hi @codope When is this support expected ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on issue #8185: [SUPPORT] Getting older data using trino hudi connector, for real time MOR table .. whereas getting correct data in hive for same mor_rt table
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on issue #8185:
URL: https://github.com/apache/hudi/issues/8185#issuecomment-1470363966
@pravin1406 The MOR snapshot query (_rt table) is not yet supported in the trino-hudi connector. We have an active PR which is expected to merge soon - https://github.com/trinodb/trino/pull/14786
Stay tuned!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on issue #8185: [SUPPORT] Getting older data using trino hudi connector, for real time MOR table .. whereas getting correct data in hive for same mor_rt table
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on issue #8185:
URL: https://github.com/apache/hudi/issues/8185#issuecomment-1704580224
Hi @pravin1406 , please check my comment here https://github.com/trinodb/trino/pull/14786#issuecomment-1683432646
I think we will have to wait for a couple of months more to get this out. Hudi 0.14.0 is still in the RC phase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pravin1406 commented on issue #8185: [SUPPORT] Getting older data using trino hudi connector, for real time MOR table .. whereas getting correct data in hive for same mor_rt table
Posted by "pravin1406 (via GitHub)" <gi...@apache.org>.
pravin1406 commented on issue #8185:
URL: https://github.com/apache/hudi/issues/8185#issuecomment-1470416379
Ohkay, Will wait for the support ! Thanks for the quick response.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on issue #8185: [SUPPORT] Getting older data using trino hudi connector, for real time MOR table .. whereas getting correct data in hive for same mor_rt table
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on issue #8185:
URL: https://github.com/apache/hudi/issues/8185#issuecomment-1469461153
A known issue, @codope , what is the approximate fix time ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pravin1406 commented on issue #8185: [SUPPORT] Getting older data using trino hudi connector, for real time MOR table .. whereas getting correct data in hive for same mor_rt table
Posted by "pravin1406 (via GitHub)" <gi...@apache.org>.
pravin1406 commented on issue #8185:
URL: https://github.com/apache/hudi/issues/8185#issuecomment-1470408996
@codope Thanks for that. But hive connector works right ? I tried using that as well but the result still remains the same.
Please see below for the output when using hive connector
<img width="1355" alt="Screenshot 2023-03-15 at 10 27 31 PM" src="https://user-images.githubusercontent.com/25177655/225384585-f8fde30b-213b-4c05-9147-5aee18cf5262.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pravin1406 closed issue #8185: [SUPPORT] Getting older data using trino hudi connector, for real time MOR table .. whereas getting correct data in hive for same mor_rt table
Posted by "pravin1406 (via GitHub)" <gi...@apache.org>.
pravin1406 closed issue #8185: [SUPPORT] Getting older data using trino hudi connector, for real time MOR table .. whereas getting correct data in hive for same mor_rt table
URL: https://github.com/apache/hudi/issues/8185
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pravin1406 commented on issue #8185: [SUPPORT] Getting older data using trino hudi connector, for real time MOR table .. whereas getting correct data in hive for same mor_rt table
Posted by "pravin1406 (via GitHub)" <gi...@apache.org>.
pravin1406 commented on issue #8185:
URL: https://github.com/apache/hudi/issues/8185#issuecomment-1469477267
Adding output of trino for proofs!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on issue #8185: [SUPPORT] Getting older data using trino hudi connector, for real time MOR table .. whereas getting correct data in hive for same mor_rt table
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on issue #8185:
URL: https://github.com/apache/hudi/issues/8185#issuecomment-1470414149
Sorry, the Hive connector also does not support yet in Trino. The support matrix for different query engines is documented at https://hudi.apache.org/docs/querying_data#support-matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org