[jira] [Resolved] (KAFKA-8774) Connect REST API exposes plaintext secrets in tasks endpoint if config value contains additional characters

["Randall Hauch (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Randall Hauch resolved KAFKA-8774.
----------------------------------
       Resolution: Fixed
         Reviewer: Randall Hauch
    Fix Version/s: 2.3.1
                   2.4.0
                   2.2.2
                   2.1.2
                   2.0.2

[~wicknicks] did a great job identifying the root cause, which involved how the AbstractHerder to correctly identifies task configs that contain variables for externalized secrets. The original method incorrectly used `matcher.matches()` instead of `matcher.find()`. The former method expects the entire string to match the regex, whereas the second one can find a pattern anywhere within the input string (which fits this use case more correctly).

This is why the problem is only in the tasks endpoint (no other endpoints) when connector configs contain externalized secret variables _plus additional characters_. If a config value contains only the variable, the secret is not exposed on this task endpoint. 

Arjun added unit tests to cover various cases of a config with externalized secrets, and updated system tests to cover case where config value contains additional characters besides secret that requires regex pattern to be found anywhere in the string (as opposed to complete match).

Merged back to the `2.0` branch, which was when [KIP-297 and externalized secrets](https://cwiki.apache.org/confluence/display/KAFKA/KIP-297%3A+Externalizing+Secrets+for+Connect+Configurations) were introduced.


> Connect REST API exposes plaintext secrets in tasks endpoint if config value contains additional characters
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-8774
>                 URL: https://issues.apache.org/jira/browse/KAFKA-8774
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 2.3.0
>            Reporter: Oleksandr Diachenko
>            Assignee: Arjun Satish
>            Priority: Critical
>             Fix For: 2.0.2, 2.1.2, 2.2.2, 2.4.0, 2.3.1
>
>
> I have configured a Connector to use externalized secrets, and the following endpoint returns secrets in the externalized form: 
> {code:java}
> curl localhost:8083/connectors/foobar|jq
> {code}
> {code:java}
> {
> "name": "foobar",
> "config": {
> "connector.class": "io.confluent.connect.s3.S3SinkConnector",
> ...
> "consumer.override.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"${file:/some/secret/path/secrets.properties:kafka.api.key}\" password=\"${file:/some/secret/path/secrets.properties:kafka.api.secret}\";",
> "admin.override.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"${file:/some/secret/path/secrets.properties:kafka.api.key}\" password=\"${file:/some/secret/path/secrets.properties:kafka.api.secret}\";",
> "consumer.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"${file:/some/secret/path/secrets.properties:kafka.api.key}\" password=\"${file:/some/secret/path/secrets.properties:kafka.api.secret}\";",
> "producer.override.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"${file:/some/secret/path/secrets.properties:kafka.api.key}\" password=\"${file:/some/secret/path/secrets.properties:kafka.api.secret}\";",
> "producer.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"${file:/some/secret/path/secrets.properties:kafka.api.key}\" password=\"${file:/some/secret/path/secrets.properties:kafka.api.secret}\";",
> ...
> },
> "tasks": [
> { "connector": "foobar", "task": 0 }
> ],
> "type": "sink"
> }{code}
> But another endpoint returns secrets in plain text:
> {code:java}
> curl localhost:8083/connectors/foobar/tasks|jq
> {code}
> {code:java}
> [
>   {
>     "id": {
>       "connector": "lcc-kgkpm",
>       "task": 0
>     },
>     "config": {
>       "connector.class": "io.confluent.connect.s3.S3SinkConnector",
>       ...
>       "errors.log.include.messages": "true",
>       "flush.size": "1000",
>       "consumer.override.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"OOPS\" password=\"SURPRISE\";",
>       "admin.override.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"OOPS\" password=\"SURPRISE\";",
>       "consumer.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"OOPS\" password=\"SURPRISE\";",
>       "producer.override.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"OOPS\" password=\"SURPRISE\";",
>       "producer.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"OOPS\" password=\"SURPRISE\";",
>       ...
>     }
>   }
> ]
> {code}
>  
> EDIT: This bug only shows up if the secrets are a substring in the config value. If they form the entirety of the config value, then the secrets are hidden at the /tasks endpoints.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)