You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/08/21 14:56:07 UTC
[GitHub] [flink] bzhaoopenstack opened a new pull request, #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
bzhaoopenstack opened a new pull request, #20642:
URL: https://github.com/apache/flink/pull/20642
Shade the fabric8 kubernetes deps with `org.apache.flink.shaded` prefix in flink-kubernetes
## What is the purpose of the change
For supporting stepDecorators SPI(pluginable decorators), we propose to package the implementation class and associated dependencies into a plugin jar.
So we need to load the said dependencies from parent class loader, as the most part / all of plugin decorators depend on the fabric8 kubernetes dependency, such as replies on the kubernetes models/client from fabric8.
So we need to shade all the said classes in flink-kubernetes and flink-dist.
## Brief change log
- Shade the `io.fabric.kubernetes` with `org.apache.flink.shaded` prefix.
- Exclude the original left dependencies / classes / SPI files in flink-dist.
## Verifying this change
Build the all flink packages, and verify the flink-kubernetes jar and flink-dist jar had re-leverage the affected dependecies, classes and files.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): ( no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] bzhaoopenstack commented on pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
bzhaoopenstack commented on PR #20642:
URL: https://github.com/apache/flink/pull/20642#issuecomment-1221672835
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] bzhaoopenstack commented on pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
bzhaoopenstack commented on PR #20642:
URL: https://github.com/apache/flink/pull/20642#issuecomment-1236886852
Sorry for late...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
flinkbot commented on PR #20642:
URL: https://github.com/apache/flink/pull/20642#issuecomment-1221562438
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "78eb40c702450a389ccae05f9cab415344f707bc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "78eb40c702450a389ccae05f9cab415344f707bc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 78eb40c702450a389ccae05f9cab415344f707bc UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wangyang0918 commented on a diff in pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on code in PR #20642:
URL: https://github.com/apache/flink/pull/20642#discussion_r963603495
##########
flink-dist/pom.xml:
##########
@@ -715,6 +715,8 @@ under the License.
<artifact>*</artifact>
<excludes>
<exclude>org/apache/flink/runtime/util/bash/BashJavaUtils.class</exclude>
+ <exclude>io/fabric8/**</exclude>
Review Comment:
What do we actually exclude here? Should this happen in the `flink-kubernetes/pom.xml`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] bzhaoopenstack commented on pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
bzhaoopenstack commented on PR #20642:
URL: https://github.com/apache/flink/pull/20642#issuecomment-1221774647
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas commented on a diff in pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on code in PR #20642:
URL: https://github.com/apache/flink/pull/20642#discussion_r952506273
##########
flink-kubernetes/pom.xml:
##########
@@ -167,6 +167,10 @@ under the License.
<pattern>dk.brics.automaton</pattern>
<shadedPattern>org.apache.flink.kubernetes.shaded.dk.brics.automaton</shadedPattern>
</relocation>
+ <relocation>
+ <pattern>io.fabric8.kubernetes</pattern>
+ <shadedPattern>org.apache.flink.shaded.io.fabric8.kubernetes</shadedPattern>
Review Comment:
```suggestion
<shadedPattern>org.apache.flink.kubernetes.shaded.io.fabric8.kubernetes</shadedPattern>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wangyang0918 commented on pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on PR #20642:
URL: https://github.com/apache/flink/pull/20642#issuecomment-1229832844
I will be great if we also shade the `io.fabric8:zjsonpatch`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] bzhaoopenstack commented on a diff in pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
bzhaoopenstack commented on code in PR #20642:
URL: https://github.com/apache/flink/pull/20642#discussion_r964871297
##########
flink-dist/pom.xml:
##########
@@ -715,6 +715,8 @@ under the License.
<artifact>*</artifact>
<excludes>
<exclude>org/apache/flink/runtime/util/bash/BashJavaUtils.class</exclude>
+ <exclude>io/fabric8/**</exclude>
Review Comment:
My bad, this is mvn version dismatch. Will refresh this part. Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] bzhaoopenstack commented on pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
bzhaoopenstack commented on PR #20642:
URL: https://github.com/apache/flink/pull/20642#issuecomment-1223433751
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] bzhaoopenstack commented on pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
bzhaoopenstack commented on PR #20642:
URL: https://github.com/apache/flink/pull/20642#issuecomment-1236886590
Ok, I will shade the said package `io.fabric8:zjsonpatch` with org.apache.flink.shaded prefix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] bzhaoopenstack commented on a diff in pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
bzhaoopenstack commented on code in PR #20642:
URL: https://github.com/apache/flink/pull/20642#discussion_r952515872
##########
flink-kubernetes/pom.xml:
##########
@@ -167,6 +167,10 @@ under the License.
<pattern>dk.brics.automaton</pattern>
<shadedPattern>org.apache.flink.kubernetes.shaded.dk.brics.automaton</shadedPattern>
</relocation>
+ <relocation>
+ <pattern>io.fabric8.kubernetes</pattern>
+ <shadedPattern>org.apache.flink.shaded.io.fabric8.kubernetes</shadedPattern>
Review Comment:
Addressed. Updating.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] bzhaoopenstack commented on pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
bzhaoopenstack commented on PR #20642:
URL: https://github.com/apache/flink/pull/20642#issuecomment-1241408388
@wangyang0918 Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wangyang0918 merged pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
wangyang0918 merged PR #20642:
URL: https://github.com/apache/flink/pull/20642
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas commented on pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on PR #20642:
URL: https://github.com/apache/flink/pull/20642#issuecomment-1226755242
@wangyang0918 , PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] gyfora commented on pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
gyfora commented on PR #20642:
URL: https://github.com/apache/flink/pull/20642#issuecomment-1230529865
Could we simply shade everything under `io.fabric8.` @wangyang0918 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wangyang0918 commented on pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on PR #20642:
URL: https://github.com/apache/flink/pull/20642#issuecomment-1231222159
@gyfora Yes. We could shade all the dependencies under `io.fabric8`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] bzhaoopenstack commented on a diff in pull request #20642: [FLINK-29047][k8s] Shade fabric8 kubernetes with org.apache.flink.shaded prefix in flink-kubernetes
Posted by GitBox <gi...@apache.org>.
bzhaoopenstack commented on code in PR #20642:
URL: https://github.com/apache/flink/pull/20642#discussion_r964728414
##########
flink-dist/pom.xml:
##########
@@ -715,6 +715,8 @@ under the License.
<artifact>*</artifact>
<excludes>
<exclude>org/apache/flink/runtime/util/bash/BashJavaUtils.class</exclude>
+ <exclude>io/fabric8/**</exclude>
Review Comment:
The files of `io.fabric8` directory are `zjsonpatch` and `kubernetes`.
I think I agree with you before I add this exclude option. But when I removed this line in pom from flink-dist and package the whole project in my env. The `io.fabric8` actually is packaged into the flink-dist.jar . The same behavior with the following line for "META-INF/services"(it will contain the shaded service file and non-shaded ones). If we remove both lines in flink-dist pom, the packaged flink-dist.jar will contain `io.fabric8` and `org.apache.flink.kubernetes.shaded.io.fabric8`, which contains the same class files. But the flink-kubernetes will package as wish. So I think the behavior might be related with the shade of flink-dist.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org