You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/07/29 11:51:27 UTC

[GitHub] [spark] xuanyuanking opened a new pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

xuanyuanking opened a new pull request #33578:
URL: https://github.com/apache/spark/pull/33578


   ### What changes were proposed in this pull request?
   As the discussion in https://github.com/apache/spark/pull/32928/files#r654049392, after confirming the compatibility, we can use a newer RocksDB version for the state store implementation.
   
   ### Why are the changes needed?
   For further ARM support and leverage the bug fix for the newer version.
   
   ### Does this PR introduce _any_ user-facing change?
   No.
   
   ### How was this patch tested?
   Existing tests.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889201506


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141842/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889366620


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141845/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889345646


   **[Test build #141845 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141845/testReport)** for PR 33578 at commit [`b83d73f`](https://github.com/apache/spark/commit/b83d73fee4ddd26e053d01ad36404aeee65ce208).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889078538


   **[Test build #141842 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141842/testReport)** for PR 33578 at commit [`b83d73f`](https://github.com/apache/spark/commit/b83d73fee4ddd26e053d01ad36404aeee65ce208).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889329569


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46360/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889078538


   **[Test build #141842 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141842/testReport)** for PR 33578 at commit [`b83d73f`](https://github.com/apache/spark/commit/b83d73fee4ddd26e053d01ad36404aeee65ce208).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889354037


   Thank you, @xuanyuanking , @srowen , @viirya .
   
   Merged to master/branch-3.2.
   
   cc @gengliangwang 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun closed pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #33578:
URL: https://github.com/apache/spark/pull/33578


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889245167


   **[Test build #141845 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141845/testReport)** for PR 33578 at commit [`b83d73f`](https://github.com/apache/spark/commit/b83d73fee4ddd26e053d01ad36404aeee65ce208).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889305795


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46360/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] srowen commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889236136


   FWIW, I think Databricks has updated RocksDB to 6.20.x internally to fix some things too. (Don't know firsthand, just remember a thread about it) So should be OK pending tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889284244


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46360/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889245167


   **[Test build #141845 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141845/testReport)** for PR 33578 at commit [`b83d73f`](https://github.com/apache/spark/commit/b83d73fee4ddd26e053d01ad36404aeee65ce208).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889200799






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889366620


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141845/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889275746


   If this is tested by Databricks internally, should be okay. CI pending.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] xuanyuanking commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889052840


   cc @dongjoon-hyun @viirya @HeartSaVioR  


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889199770


   **[Test build #141842 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141842/testReport)** for PR 33578 at commit [`b83d73f`](https://github.com/apache/spark/commit/b83d73fee4ddd26e053d01ad36404aeee65ce208).
    * This patch **fails PySpark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889200760


   Kubernetes integration test unable to build dist.
   
   exiting with code: 1
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46357/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889200799


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46357/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] srowen commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889236411


   Jenkins retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889329569


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46360/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] xuanyuanking commented on pull request #33578: [SPARK-36347][SS] Upgrade the RocksDB version to 6.20.3

Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on pull request #33578:
URL: https://github.com/apache/spark/pull/33578#issuecomment-889670438


   Thanks for the quick review and merging! @dongjoon-hyun @viirya @srowen 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org