[jira] [Updated] (KAFKA-17251) KafkaStreams.cleanup() semantics unclear

["Matthias J. Sax (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthias J. Sax updated KAFKA-17251:
------------------------------------
    Description: 
`KafkaStreams#cleanup()` method is designed to delete an instance local state directory, including the `app.dir` itself if it's empty (cf corresponding unit tests in StateDirectoryTest.java). 

If the top level `app.dir` could not be deleted, a WARN is logged.

However, in a later version we started to persist the `processId` what implies that the state directory won't be empty for many cases, because the `StateDirectory#clean()` method does not explicitly delete the processId-file.

It's unclear right now, if `clean()` should actually try to delete the `processId` file, too, and it's a bug that it does not, or if there are cases for which the `processId` should actually be preserved (for this case, we should not log a WARN as it's expected that the `app.dir` is not deleted).

Maybe there is not even a strict yes/no answer to this question, but we should extend `KafkaStreams#cleanup()` with a parameter and let users pick? (This would require a KIP.)

  was:
`KafkaStreams#cleanup()` method is designed to delete an instance local state directory, including the `app.dir` itself if it's empty (cf corresponding unit tests in StateDirectoryTest.java). 

If the top level `app.dir` could not be deleted, a WARN is logged.

However, in a later version we started to persist the `processId` what implies that the state directory won't be empty for many cases, because the `StateDirectory#clean()` method does not explicitly delete it.

It's unclear right now, if `clean()` should actually try to delete the `processId` file, too, and it's a bug that it does not, or if there are cases for which the `processId` should actually be preserved (for this case, we should not log a WARN as it's expected that the `app.dir` is not deleted).

Maybe there is not even a strict yes/no answer to this question, but we should extend `KafkaStreams#cleanup()` with a parameter and let users pick? (This would require a KIP.)


> KafkaStreams.cleanup() semantics unclear
> ----------------------------------------
>
>                 Key: KAFKA-17251
>                 URL: https://issues.apache.org/jira/browse/KAFKA-17251
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Matthias J. Sax
>            Priority: Major
>              Labels: needs-kip
>
> `KafkaStreams#cleanup()` method is designed to delete an instance local state directory, including the `app.dir` itself if it's empty (cf corresponding unit tests in StateDirectoryTest.java). 
> If the top level `app.dir` could not be deleted, a WARN is logged.
> However, in a later version we started to persist the `processId` what implies that the state directory won't be empty for many cases, because the `StateDirectory#clean()` method does not explicitly delete the processId-file.
> It's unclear right now, if `clean()` should actually try to delete the `processId` file, too, and it's a bug that it does not, or if there are cases for which the `processId` should actually be preserved (for this case, we should not log a WARN as it's expected that the `app.dir` is not deleted).
> Maybe there is not even a strict yes/no answer to this question, but we should extend `KafkaStreams#cleanup()` with a parameter and let users pick? (This would require a KIP.)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)