You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2019/10/23 22:52:48 UTC
[GitHub] [spark] HeartSaVioR edited a comment on issue #25943:
[WIP][SPARK-29261][SQL][CORE] Support recover live entities from KVStore
for (SQL)AppStatusListener
HeartSaVioR edited a comment on issue #25943: [WIP][SPARK-29261][SQL][CORE] Support recover live entities from KVStore for (SQL)AppStatusListener
URL: https://github.com/apache/spark/pull/25943#issuecomment-545668898
@Ngone51
Sorry I revisit too late. We've had some sort of a stuck moment since compatibility was raised as a major problem. I've discussed with @vanzin and @squito offline, and we all agreed about allowing no-compatibility on live entities/snapshots for now. (The goal of SPARK-28870 is changed to not allow deleting old event logs.)
Based on the agreement, I feel we could just try to serialize live entities by itself (although we should have to recover references between them) instead of recreating them from KVStore entities which seem to be too complicated and likely lossy.
Could you update your patch to simplify based on this agreement? Actually I have a POC commit to deal with this, and got a feedback, too verbose (like let jackson works for me).
https://github.com/HeartSaVioR/spark/commit/946bca40f66aa5a1709f04e67a5d61b6b7351a6b
I'd be really appreciate if you could take this up and simplify the code change. No need to retain authorship - it's yours if you are willing to take up.
Apologize of the change of direction. I'm really sorry about this for leading wrong direction. Please let me know if you have any question or uncertain. Thanks in advance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org