You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/06/02 07:58:50 UTC
[GitHub] [iceberg] ymZhao1001 opened a new issue, #4940: Caching Data In Memory
ymZhao1001 opened a new issue, #4940:
URL: https://github.com/apache/iceberg/issues/4940
**Is your feature request related to a problem? Please describe this feature request :**
In some multi-table associative query scenarios, the data volume of the dimension table is small and the update is slow. We can cache the dimension table in memory to improve query performance
**Describe the solution you'd like :**
Add a cache attribute to the table, then we use spark.catalog.cacheTable("tableName") or implements the LookupTableSource in flink connector to cache table in Spark and Flink, for example
@rdblue @kbendick what do you thinik of it ? It would be a pleasure to hear your thoughts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ymZhao1001 commented on issue #4940: Caching Data In Memory
Posted by GitBox <gi...@apache.org>.
ymZhao1001 commented on issue #4940:
URL: https://github.com/apache/iceberg/issues/4940#issuecomment-1147120701
> Is it not already possible to use `spark.table("iceberg.tableName").cache` using the normal Spark cache?
i am not sure about this. The key point is it necessary to cache data? if it is necessary , it depends on iceberg or the compute engine? Looking forward to your reply
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ymZhao1001 closed issue #4940: Caching Data In Memory
Posted by GitBox <gi...@apache.org>.
ymZhao1001 closed issue #4940: Caching Data In Memory
URL: https://github.com/apache/iceberg/issues/4940
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] kbendick commented on issue #4940: Caching Data In Memory
Posted by GitBox <gi...@apache.org>.
kbendick commented on issue #4940:
URL: https://github.com/apache/iceberg/issues/4940#issuecomment-1145122782
Is it not already possible to use `spark.table("iceberg.tableName").cache` using the normal Spark cache?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ymZhao1001 commented on issue #4940: Caching Data In Memory
Posted by GitBox <gi...@apache.org>.
ymZhao1001 commented on issue #4940:
URL: https://github.com/apache/iceberg/issues/4940#issuecomment-1149679107
closed by
https://github.com/apache/iceberg/pull/4649
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org