You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/09/30 15:49:01 UTC
[GitHub] [kafka] heritamas opened a new pull request #11371: MINOR: fix of possible java.nio.file.AccessDeniedException during Con…
heritamas opened a new pull request #11371:
URL: https://github.com/apache/kafka/pull/11371
…nect plugin directory scan
org.apache.kafka.connect.runtime.isolation.PluginUtils.pluginUrls scans a path and collects plugin candidates from there. However, if a directory is not readable, it fails with AccessDeniedException instead of skipping it. This commit fixes this minor problem with the change of plugin path filter used during scanning.
I've also added a unit test for proof.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] akatona84 commented on pull request #11371: KAFKA-13337: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
akatona84 commented on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-959759146
IMHO we could enhance the code to log those which are not eligible for the conditions (readable, directory or jar, whatever) if that helps.
The problem is that if it's not readable, it'll fail to load ANY plugins. Not just skipping the problematic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] akatona84 edited a comment on pull request #11371: KAFKA-13337: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
akatona84 edited a comment on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-959759146
IMHO we could enhance the code to log those which are not eligible for the conditions (readable, directory or jar, whatever) if that helps.
The problem is that if it's not readable, it'll fail to load ANY plugins. Not just skipping the problematic, no plugin will be available/loaded at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] akatona84 commented on pull request #11371: KAFKA-13337: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
akatona84 commented on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-959759146
IMHO we could enhance the code to log those which are not eligible for the conditions (readable, directory or jar, whatever) if that helps.
The problem is that if it's not readable, it'll fail to load ANY plugins. Not just skipping the problematic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mimaison commented on pull request #11371: KAFKA-13337: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
mimaison commented on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-959671382
Sorry I'm still not quite sure I fully understand the issue you're describing.
If the path is not readable, yes Connect won't load plugins in it. But it's printing an error message `Could not get listing for plugin path: <PATH>. Ignoring.` so it's easy to spot there's an issue with permissions.
As far as I can tell, with this PR, we don't get a message anymore. I think this could make debugger harder.
When there's an unreadable folder in my `plugin.path`, I don't see any issues with Connect. It just skipped that folder but otherwise works just fine. Can you explain why yours failed/stopped?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] akatona84 commented on pull request #11371: MINOR: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
akatona84 commented on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-931982306
@heritamas pls change the title MINOR part to be KAFKA-13337
Hey @kkonstantine , could you take a look, pls?
Connect dies during startup if it encounters unreadable plugin dir, minor fix, but we created a ticket for that, maybe others would hit it too and they could find jira about it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] akatona84 edited a comment on pull request #11371: KAFKA-13337: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
akatona84 edited a comment on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-959759146
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] akatona84 commented on pull request #11371: KAFKA-13337: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
akatona84 commented on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-959759146
IMHO we could enhance the code to log those which are not eligible for the conditions (readable, directory or jar, whatever) if that helps.
The problem is that if it's not readable, it'll fail to load ANY plugins. Not just skipping the problematic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] akatona84 commented on pull request #11371: KAFKA-13337: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
akatona84 commented on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-1014579238
@mimaison , np, thx for following up :)
steps to fail:
1. create - for example - an environment variable config provider, create a jar from it
2. put it into the plugins directory
3. configure this config provider in connect-distributed.properties and also use it at another config entry. e.g.
```
something=${env:MY_ENVIRONMENT_VARIABLE}
```
4. create a directory within the plugins dir and make it unreadable via chmod (chmod 0000 would do the trick)
5. try to start connect
It proves that connect won't load any plugin not just skipping the non-readable one. (And the server dies because of the missing config provider.)
I hope it helps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] akatona84 edited a comment on pull request #11371: KAFKA-13337: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
akatona84 edited a comment on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-959759146
IMHO we could enhance the code to log those which are not eligible for the conditions (readable, directory or jar, whatever) if that helps.
The problem is that if it's not readable, it'll fail to load ANY plugins. Not just skipping the problematic, no plugin will be available/loaded at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mimaison commented on pull request #11371: KAFKA-13337: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
mimaison commented on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-959671382
Sorry I'm still not quite sure I fully understand the issue you're describing.
If the path is not readable, yes Connect won't load plugins in it. But it's printing an error message `Could not get listing for plugin path: <PATH>. Ignoring.` so it's easy to spot there's an issue with permissions.
As far as I can tell, with this PR, we don't get a message anymore. I think this could make debugger harder.
When there's an unreadable folder in my `plugin.path`, I don't see any issues with Connect. It just skipped that folder but otherwise works just fine. Can you explain why yours failed/stopped?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mimaison commented on pull request #11371: KAFKA-13337: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
mimaison commented on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-950757901
Thanks for the PR!
I played a bit locally and while Connect logs a `AccessDeniedException` exception if a file/directory is not readable, Connect seems to work fine and in the logs I can see Connect is ignoring the directory:
```
[2021-10-25 11:43:59,745] ERROR Could not get listing for plugin path: /tmp/connect. Ignoring. (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:257)
java.nio.file.AccessDeniedException: /tmp/connect/private
```
[KAFKA-13337](https://issues.apache.org/jira/browse/KAFKA-13337) mentions `Connect will fail with an AccessDeniedException.`, is there another issue or is the ticket just about the exception/stacktrace? (It's still valuable to address the stacktrace but I just want to be sure there's not another issue)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] akatona84 commented on pull request #11371: KAFKA-13337: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
akatona84 commented on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-950937973
@mimaison , correct, connect could work, but the plugins are not loaded. My connect stopped because an extension was supposed to be loaded and a `ConnectException` was thrown and connect died.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mimaison commented on pull request #11371: KAFKA-13337: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
mimaison commented on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-959671382
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] akatona84 commented on pull request #11371: KAFKA-13337: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
akatona84 commented on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-959759146
IMHO we could enhance the code to log those which are not eligible for the conditions (readable, directory or jar, whatever) if that helps.
The problem is that if it's not readable, it'll fail to load ANY plugins. Not just skipping the problematic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mimaison commented on pull request #11371: KAFKA-13337: fix of possible java.nio.file.AccessDeniedException during Con…
Posted by GitBox <gi...@apache.org>.
mimaison commented on pull request #11371:
URL: https://github.com/apache/kafka/pull/11371#issuecomment-1014401089
@akatona84 Sorry this PR felt through the cracks. If I remember correctly, last time I was looking at it I had trouble replicating the issue. Can you provide steps to reproduce?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org