You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "janjwerner-confluent (via GitHub)" <gi...@apache.org> on 2024/02/29 17:52:09 UTC
[PR] remove azure-sdk from ranger-extension (druid)
janjwerner-confluent opened a new pull request, #16011:
URL: https://github.com/apache/druid/pull/16011
Fixes # size blowup regression introduced in https://github.com/apache/druid/pull/15443
### Description
This PR removes the transitive dependency of ranger-plugins-audit to reduce the size of the compiled artifacts
This PR has:
- [ ] been self-reviewed.
- [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] a release note entry in the PR description.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
- [ ] added integration tests.
- [ ] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] remove aws-sdk from ranger-extension (druid)
Posted by "janjwerner-confluent (via GitHub)" <gi...@apache.org>.
janjwerner-confluent commented on code in PR #16011:
URL: https://github.com/apache/druid/pull/16011#discussion_r1517763906
##########
extensions-core/druid-ranger-security/pom.xml:
##########
@@ -193,7 +193,14 @@
</exclusions>
<scope>compile</scope>
</dependency>
-
+ <!-- This is a transitive dependency of ranger-plugins-audit
+ added here to replace bloated bundle sdk, remove this if /when
+ apache ranger replaces bundle-skd with logs-sdk -->
+ <dependency>
+ <groupId>com.amazonaws</groupId>
+ <artifactId>aws-java-sdk-logs</artifactId>
+ <version>1.12.669</version>
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] remove aws-sdk from ranger-extension (druid)
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl merged PR #16011:
URL: https://github.com/apache/druid/pull/16011
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] remove azure-sdk from ranger-extension (druid)
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl commented on code in PR #16011:
URL: https://github.com/apache/druid/pull/16011#discussion_r1509304747
##########
extensions-core/druid-ranger-security/pom.xml:
##########
@@ -193,7 +193,14 @@
</exclusions>
<scope>compile</scope>
</dependency>
-
+ <!-- This is a transitive dependency of ranger-plugins-audit
+ added here to replace bloated bundle sdk, remove this if /when
+ apache ranger replaces bundle-skd with logs-sdk -->
+ <dependency>
+ <groupId>com.amazonaws</groupId>
+ <artifactId>aws-java-sdk-logs</artifactId>
+ <version>1.12.669</version>
Review Comment:
nit, any reason we don't use ${aws.sdk.version} here and align the versions across the board?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] remove azure-sdk from ranger-extension (druid)
Posted by "abhishekagarwal87 (via GitHub)" <gi...@apache.org>.
abhishekagarwal87 commented on PR #16011:
URL: https://github.com/apache/druid/pull/16011#issuecomment-1974782159
why does the description says "azure-sdk"?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] remove azure-sdk from ranger-extension (druid)
Posted by "janjwerner-confluent (via GitHub)" <gi...@apache.org>.
janjwerner-confluent commented on code in PR #16011:
URL: https://github.com/apache/druid/pull/16011#discussion_r1508130853
##########
extensions-core/druid-ranger-security/pom.xml:
##########
@@ -178,6 +178,10 @@
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>com.amazonaws</groupId>
+ <artifactId>aws-java-sdk-bundle</artifactId>
+ </exclusion>
Review Comment:
AWS-sdk-bundle is used only in CloudWatch audit [destination](https://github.com/apache/ranger/blob/master/agents-audit/src/main/java/org/apache/ranger/audit/destination/AmazonCloudWatchAuditDestination.java), and I believe that codepath is not touched by ranger extension for Druid,
I will add [aws-java-sdk-logs](https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-logs/1.12.669) package as explicit dependency of this module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] remove aws-sdk from ranger-extension (druid)
Posted by "janjwerner-confluent (via GitHub)" <gi...@apache.org>.
janjwerner-confluent commented on PR #16011:
URL: https://github.com/apache/druid/pull/16011#issuecomment-1979288898
> why does the description says "azure-sdk"?
brain freeze I guess, fixed it :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] remove azure-sdk from ranger-extension (druid)
Posted by "LakshSingla (via GitHub)" <gi...@apache.org>.
LakshSingla commented on PR #16011:
URL: https://github.com/apache/druid/pull/16011#issuecomment-1971767296
Is it safe to exclude the dependency? Last I checked, the ranger did rely on AWS Cloudwatch for some things. Do we not use that functionality in Druid?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] remove azure-sdk from ranger-extension (druid)
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl commented on code in PR #16011:
URL: https://github.com/apache/druid/pull/16011#discussion_r1508246591
##########
extensions-core/druid-ranger-security/pom.xml:
##########
@@ -193,7 +198,13 @@
</exclusions>
<scope>compile</scope>
</dependency>
-
+ <!-- This is a transitive dependency of ranger-plugins-audit
+ added here to replace bloated bundle sdk -->
+ <dependency>
+ <groupId>com.amazonaws</groupId>
+ <artifactId>aws-java-sdk-logs</artifactId>
+ <version>1.12.669</version>
Review Comment:
can we put a note next to the ranger plugin version to update the aws-sdk version when the ranger plugin gets updated?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] remove azure-sdk from ranger-extension (druid)
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl commented on code in PR #16011:
URL: https://github.com/apache/druid/pull/16011#discussion_r1508112657
##########
extensions-core/druid-ranger-security/pom.xml:
##########
@@ -178,6 +178,10 @@
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>com.amazonaws</groupId>
+ <artifactId>aws-java-sdk-bundle</artifactId>
+ </exclusion>
Review Comment:
it'd be nice to add some comments to explain how those are safe to exclude in our case and why we are excluding them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org