You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Lior Liviev <Li...@earnix.com> on 2021/12/21 14:53:25 UTC
Class loader
Hello, I wanted to know if I have my user code Jar in Flink, and I'm running it 3 times, will the class loader take the same classes at every execution?
Re: Class loader
Posted by David Morávek <dm...@apache.org>.
to answer that, I need a better understanding of how exactly you deploy the
jobs. Please keep the conversation focused in the second thread, that
discusses the problem
D.
On Tue, Dec 21, 2021 at 5:09 PM Lior Liviev <Li...@earnix.com> wrote:
> I don’t have the Jar in Flinks folder yet it’s just that I don’t know if
> it will resolve my oom metaspace problem
>
> Get Outlook for iOS <https://aka.ms/o0ukef>
> ------------------------------
> *From:* David Morávek <dm...@apache.org>
> *Sent:* Tuesday, December 21, 2021 5:59:05 PM
> *To:* Lior Liviev <Li...@earnix.com>; user <us...@flink.apache.org>
> *Subject:* Re: Class loader
>
>
> *CAUTION*: external source
>
> if I have my user code Jar in Flink
>
>
> I assume this means that the user code is already on Flink's class path
> (eg. /lib folder).
>
> Then it depends on how the class-loader is set up [1] and on how you
> submit the job. For session cluster, each job is isolated in a separate
> classloader and by default Flink uses the child-first resolution strategy,
> that first tries to load classes from the submitted job JAR.
>
> [1]
> https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#classloader-resolve-order
> <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnightlies.apache.org%2Fflink%2Fflink-docs-master%2Fdocs%2Fdeployment%2Fconfig%2F%23classloader-resolve-order&data=04%7C01%7CLior.Liviev%40earnix.com%7C8ffb4451f7f34e2fa20708d9c49ad907%7Cae9992508a9f4ae58a5dce9de7084b84%7C0%7C0%7C637756991625887288%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=oh1bSzwAADQNvJkUMkKFfvzeoe%2Bh6UpqT6ZbD1n8P68%3D&reserved=0>
>
> On Tue, Dec 21, 2021 at 4:50 PM Lior Liviev <Li...@earnix.com>
> wrote:
>
> Actually no, it's a separate question
> ------------------------------
> *From:* David Morávek <dm...@apache.org>
> *Sent:* Tuesday, December 21, 2021 5:45 PM
> *To:* Lior Liviev <Li...@earnix.com>
> *Cc:* user@flink.apache.org <us...@flink.apache.org>
> *Subject:* Re: Class loader
>
>
> *CAUTION*: external source
> I assume this is a duplicate of the previous thread [1]
>
> [1] https://lists.apache.org/thread/16kxytrqycolfwfmr5tv0g6bq9m2wvof
> <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2F16kxytrqycolfwfmr5tv0g6bq9m2wvof&data=04%7C01%7CLior.Liviev%40earnix.com%7C8ffb4451f7f34e2fa20708d9c49ad907%7Cae9992508a9f4ae58a5dce9de7084b84%7C0%7C0%7C637756991625887288%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=JXWLOZWI4C7r8tyGIJlb%2B%2BtYB9iU7Jz4%2FVAFTT%2FbVTE%3D&reserved=0>
>
> Best,
> D.
>
> On Tue, Dec 21, 2021 at 3:53 PM Lior Liviev <Li...@earnix.com>
> wrote:
>
> Hello, I wanted to know if I have my user code Jar in Flink, and I'm
> running it 3 times, will the class loader take the same classes at every
> execution?
>
> Do not click on links or open attachments unless you recognize the sender.
> Please use the report button if you believe this email is suspicious.
>
> Do not click on links or open attachments unless you recognize the sender.
> Please use the report button if you believe this email is suspicious.
>
Re: Class loader
Posted by David Morávek <dm...@apache.org>.
>
> if I have my user code Jar in Flink
>
I assume this means that the user code is already on Flink's class path
(eg. /lib folder).
Then it depends on how the class-loader is set up [1] and on how you submit
the job. For session cluster, each job is isolated in a separate
classloader and by default Flink uses the child-first resolution strategy,
that first tries to load classes from the submitted job JAR.
[1]
https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#classloader-resolve-order
On Tue, Dec 21, 2021 at 4:50 PM Lior Liviev <Li...@earnix.com> wrote:
> Actually no, it's a separate question
> ------------------------------
> *From:* David Morávek <dm...@apache.org>
> *Sent:* Tuesday, December 21, 2021 5:45 PM
> *To:* Lior Liviev <Li...@earnix.com>
> *Cc:* user@flink.apache.org <us...@flink.apache.org>
> *Subject:* Re: Class loader
>
>
> *CAUTION*: external source
> I assume this is a duplicate of the previous thread [1]
>
> [1] https://lists.apache.org/thread/16kxytrqycolfwfmr5tv0g6bq9m2wvof
> <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2F16kxytrqycolfwfmr5tv0g6bq9m2wvof&data=04%7C01%7CLior.Liviev%40earnix.com%7C0f9e647c31e5479f463708d9c498e187%7Cae9992508a9f4ae58a5dce9de7084b84%7C0%7C0%7C637756983169060952%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Qm64dJs77uzjLjTumSUUZU9%2FTw%2FjwWZ%2Bz0Oesd04itI%3D&reserved=0>
>
> Best,
> D.
>
> On Tue, Dec 21, 2021 at 3:53 PM Lior Liviev <Li...@earnix.com>
> wrote:
>
> Hello, I wanted to know if I have my user code Jar in Flink, and I'm
> running it 3 times, will the class loader take the same classes at every
> execution?
>
> Do not click on links or open attachments unless you recognize the sender.
> Please use the report button if you believe this email is suspicious.
>
Re: Class loader
Posted by David Morávek <dm...@apache.org>.
I assume this is a duplicate of the previous thread [1]
[1] https://lists.apache.org/thread/16kxytrqycolfwfmr5tv0g6bq9m2wvof
Best,
D.
On Tue, Dec 21, 2021 at 3:53 PM Lior Liviev <Li...@earnix.com> wrote:
> Hello, I wanted to know if I have my user code Jar in Flink, and I'm
> running it 3 times, will the class loader take the same classes at every
> execution?
>