[jira] [Assigned] (SPARK-41524) Expose SQL confs and extraOptions separately in o.a.s.sql.execution.streaming.state.RocksDBConf

["Apache Spark (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-41524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-41524:
------------------------------------

    Assignee: Apache Spark

> Expose SQL confs and extraOptions separately in o.a.s.sql.execution.streaming.state.RocksDBConf
> -----------------------------------------------------------------------------------------------
>
>                 Key: SPARK-41524
>                 URL: https://issues.apache.org/jira/browse/SPARK-41524
>             Project: Spark
>          Issue Type: Improvement
>          Components: Structured Streaming
>    Affects Versions: 3.4.0
>            Reporter: Huanli Wang
>            Assignee: Apache Spark
>            Priority: Minor
>
> Currently the usage of *StateStoreConf* is via [*{{confs}}*|https://github.com/huanliwang-db/spark/blob/7671bc975f2d88ab929e4982abfe3e166fa72e35/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStoreConf.scala#L77-L78], which composes both SQL confs and extraOptions into one. The name of config for extraOptions shouldn't have to follow the name prefix of SQL conf, because it's not bound to the context of SQL conf.
> After differentiate SQL conf and extraOptions in {*}StateStoreConf{*}, we should be able to adopt more use case on operator level configs by using the {*}extraOptions{*}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org