[jira] [Commented] (KAFKA-10630) State Directory config could be improved

["Dongjin Lee (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-10630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17283068#comment-17283068 ] 

Dongjin Lee commented on KAFKA-10630:
-------------------------------------

Hi [~vvcephei], I inspected this issue a little bit.

If I understood correctly, this issue addresses:
 # State directory is not created lazily, nor deleted.
 # So many tests are repeatedly creating test state directory with {{TestUtil#tempDirectory}}, degrading test.
 # {{StreamsConfig.STATE_DIR_CONFIG}} config is os-specific.

1 is addressed in KAFKA-10585, and 3 is resolved with KAFKA-10604. So, it seems like the only remaining problem is 2, which does not require a KIP. How do you think?

 

> State Directory config could be improved
> ----------------------------------------
>
>                 Key: KAFKA-10630
>                 URL: https://issues.apache.org/jira/browse/KAFKA-10630
>             Project: Kafka
>          Issue Type: Task
>          Components: streams
>            Reporter: John Roesler
>            Priority: Minor
>              Labels: needs-kip
>
> During [https://github.com/apache/kafka/pull/9477,] I noticed that many tests wind up providing a state directory config purely to ensure a unique temp directory for the test. Since TopologyTestDriver and MockProcessorContext tests are typically unit tests, it would be more convenient to initialize those components with their own unique temp state directory, following the universal pattern from such tests:
> {code:java}
> props.setProperty(StreamsConfig.STATE_DIR_CONFIG, TestUtils.tempDirectory().getAbsolutePath()); {code}
> Note that this literal setting is not ideal, since it actually creates a directory regardless of whether the application needs one. Instead, we should create a new TestUtil method to lazily generate a temp directory _name_ and then register a shutdown handler to delete it if it exists. Then, Streams would only create the directory if it actually needs persistence.
> Also, the default value for that config is not platform independent. It is simply: {color:#067d17}"/tmp/kafka-streams"{color}. Perhaps instead we should set the default to something like "unset" or "" or "none". Then, instead of reading the property directly, when Streams actually needs the state directory, it could log a warning that the state directory config is not set and call the platform-independent Java api for creating a temporary directory.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)