You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "anishshri-db (via GitHub)" <gi...@apache.org> on 2023/07/04 01:02:58 UTC
[GitHub] [spark] anishshri-db opened a new pull request, #41840: [SPARK-44288] Set the column family options before passing to DBOptions in RocksDB state store provider
anishshri-db opened a new pull request, #41840:
URL: https://github.com/apache/spark/pull/41840
### What changes were proposed in this pull request?
Set the column family options before passing to DBOptions in RocksDB state store provider
### Why are the changes needed?
Address bug fix to ensure column family options around memory usage are passed correctly to dbOptions
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] anishshri-db commented on pull request #41840: [SPARK-44288] Set the column family options before passing to DBOptions in RocksDB state store provider
Posted by "anishshri-db (via GitHub)" <gi...@apache.org>.
anishshri-db commented on PR #41840:
URL: https://github.com/apache/spark/pull/41840#issuecomment-1619314640
cc - @HeartSaVioR - pls take a look at this small change. Thx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #41840: [SPARK-44288][SS] Set the column family options before passing to DBOptions in RocksDB state store provider
Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR commented on PR #41840:
URL: https://github.com/apache/spark/pull/41840#issuecomment-1619578400
Yeah... Looks like affected version is marked incorrectly. Let me change it instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #41840: [SPARK-44288][SS] Set the column family options before passing to DBOptions in RocksDB state store provider
Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR commented on PR #41840:
URL: https://github.com/apache/spark/pull/41840#issuecomment-1619444780
@anishshri-db Looks like there is merge conflict with branch-3.4. Could you please submit a backport PR for 3.4? Thanks in advance!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #41840: [SPARK-44288][SS] Set the column family options before passing to DBOptions in RocksDB state store provider
Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR commented on PR #41840:
URL: https://github.com/apache/spark/pull/41840#issuecomment-1619443007
Thanks! Merging to master/3.4.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] anishshri-db commented on pull request #41840: [SPARK-44288][SS] Set the column family options before passing to DBOptions in RocksDB state store provider
Posted by "anishshri-db (via GitHub)" <gi...@apache.org>.
anishshri-db commented on PR #41840:
URL: https://github.com/apache/spark/pull/41840#issuecomment-1619576207
@HeartSaVioR - I checked, but it seems like we added the memory usage related changes even for setting write buffer max size and max num of write buffers after 3.4. I think those options are not added for this release, so maybe we can skip ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR closed pull request #41840: [SPARK-44288][SS] Set the column family options before passing to DBOptions in RocksDB state store provider
Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR closed pull request #41840: [SPARK-44288][SS] Set the column family options before passing to DBOptions in RocksDB state store provider
URL: https://github.com/apache/spark/pull/41840
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #41840: [SPARK-44288] Set the column family options before passing to DBOptions in RocksDB state store provider
Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR commented on PR #41840:
URL: https://github.com/apache/spark/pull/41840#issuecomment-1619323622
I'll merge once CI passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org