You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by forideal <fs...@163.com> on 2020/11/04 06:14:51 UTC
JM upload files to blob server is slow
Hello my friend：
My line of code runs very slowly. What are the possibilities?
code：
CompletableFuture<JobGraph> jarUploadFuture = jobGraphFuture.thenCombine(blobServerPortFuture, (jobGraph, blobServerPort) -> {
final InetSocketAddress address = new InetSocketAddress(gateway.getHostname(), blobServerPort);
try {
ClientUtils.extractAndUploadJobGraphFiles(jobGraph, () -> new BlobClient(address, configuration));
} catch (FlinkException e) {
throw new CompletionException(e);
}
return jobGraph;
});
code link：https://github.com/apache/flink/blob/release-1.10.0/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/JarRunHandler.java#L93
Through jstack, we can also find that the thread Flink-DispatcherRestEndpoint-thread-1 has been running for 5 minutes.
"Flink-DispatcherRestEndpoint-thread-1" #82 daemon prio=5 os_prio=0 tid=0x00007f590c03c800 nid=0x179 runnable [0x00007f5a34165000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at org.apache.flink.runtime.blob.BlobOutputStream.write(BlobOutputStream.java:88)
at org.apache.flink.util.IOUtils.copyBytes(IOUtils.java:65)
at org.apache.flink.runtime.blob.BlobClient.putInputStream(BlobClient.java:368)
at org.apache.flink.runtime.blob.BlobClient.uploadFile(BlobClient.java:428)
at org.apache.flink.runtime.client.ClientUtils.uploadUserJars(ClientUtils.java:102)
at org.apache.flink.runtime.client.ClientUtils.uploadAndSetUserJars(ClientUtils.java:95)
at org.apache.flink.runtime.client.ClientUtils.uploadJobGraphFiles(ClientUtils.java:77)
at org.apache.flink.runtime.client.ClientUtils.extractAndUploadJobGraphFiles(ClientUtils.java:57)
at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$1(RedJarRunHandler.java:92)
at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler$$Lambda$770/941040494.apply(Unknown Source)
at java.util.concurrent.CompletableFuture.biApply(CompletableFuture.java:1105)
at java.util.concurrent.CompletableFuture$BiApply.tryFire(CompletableFuture.java:1070)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1595)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Thank you very much for your reply, forideal
Re:Re: JM upload files to blob server is slow
Posted by forideal <fs...@163.com>.
Hi Arvid Heise,
Thank you for your reply.
Yes,my connection to the JM is bad !!!
Best wishes,forideal
At 2020-11-04 15:32:38, "Arvid Heise" <ar...@ververica.com> wrote:
A jar upload shouldn't take minutes. There are two possibilities that likely co-occured:
- your jar is much bigger than needed. Did you make sure that you don't put Flink into the fatjar? That's counterproductive on many levels. Please check the jar size.
- your connection to the JM is bad. Where is your Flink cluster running? If connection is superslow, it's often better to build the jar on the respective server and submit on command line or through REST.
On Wed, Nov 4, 2020 at 7:15 AM forideal <fs...@163.com> wrote:
Hello my friend：
My line of code runs very slowly. What are the possibilities?
code：
CompletableFuture<JobGraph> jarUploadFuture = jobGraphFuture.thenCombine(blobServerPortFuture, (jobGraph, blobServerPort) -> {
final InetSocketAddress address = new InetSocketAddress(gateway.getHostname(), blobServerPort);
try {
ClientUtils.extractAndUploadJobGraphFiles(jobGraph, () -> new BlobClient(address, configuration));
} catch (FlinkException e) {
throw new CompletionException(e);
}
return jobGraph;
});
code link：https://github.com/apache/flink/blob/release-1.10.0/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/JarRunHandler.java#L93
Through jstack, we can also find that the thread Flink-DispatcherRestEndpoint-thread-1 has been running for 5 minutes.
"Flink-DispatcherRestEndpoint-thread-1" #82 daemon prio=5 os_prio=0 tid=0x00007f590c03c800 nid=0x179 runnable [0x00007f5a34165000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at org.apache.flink.runtime.blob.BlobOutputStream.write(BlobOutputStream.java:88)
at org.apache.flink.util.IOUtils.copyBytes(IOUtils.java:65)
at org.apache.flink.runtime.blob.BlobClient.putInputStream(BlobClient.java:368)
at org.apache.flink.runtime.blob.BlobClient.uploadFile(BlobClient.java:428)
at org.apache.flink.runtime.client.ClientUtils.uploadUserJars(ClientUtils.java:102)
at org.apache.flink.runtime.client.ClientUtils.uploadAndSetUserJars(ClientUtils.java:95)
at org.apache.flink.runtime.client.ClientUtils.uploadJobGraphFiles(ClientUtils.java:77)
at org.apache.flink.runtime.client.ClientUtils.extractAndUploadJobGraphFiles(ClientUtils.java:57)
at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$1(RedJarRunHandler.java:92)
at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler$$Lambda$770/941040494.apply(Unknown Source)
at java.util.concurrent.CompletableFuture.biApply(CompletableFuture.java:1105)
at java.util.concurrent.CompletableFuture$BiApply.tryFire(CompletableFuture.java:1070)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1595)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Thank you very much for your reply, forideal
--
Arvid Heise| Senior Java Developer
Follow us @VervericaData
--
Join Flink Forward - The Apache Flink Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
Re: JM upload files to blob server is slow
Posted by Arvid Heise <ar...@ververica.com>.
A jar upload shouldn't take minutes. There are two possibilities that
likely co-occured:
- your jar is much bigger than needed. Did you make sure that you don't put
Flink into the fatjar? That's counterproductive on many levels. Please
check the jar size.
- your connection to the JM is bad. Where is your Flink cluster running? If
connection is superslow, it's often better to build the jar on the
respective server and submit on command line or through REST.
On Wed, Nov 4, 2020 at 7:15 AM forideal <fs...@163.com> wrote:
> Hello my friend：
> My line of code runs very slowly. What are the possibilities?
> code：
>
> CompletableFuture<JobGraph> jarUploadFuture = jobGraphFuture.thenCombine(blobServerPortFuture, (jobGraph, blobServerPort) -> {
> final InetSocketAddress address = new InetSocketAddress(gateway.getHostname(), blobServerPort);
> try {
> ClientUtils.extractAndUploadJobGraphFiles(jobGraph, () -> new BlobClient(address, configuration));
> } catch (FlinkException e) {
> throw new CompletionException(e);
> }
> return jobGraph;
> });
>
> code link：
> https://github.com/apache/flink/blob/release-1.10.0/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/JarRunHandler.java#L93
>
> Through jstack, we can also find that the thread
> Flink-DispatcherRestEndpoint-thread-1 has been running for 5 minutes.
> "Flink-DispatcherRestEndpoint-thread-1" #82 daemon prio=5 os_prio=0
> tid=0x00007f590c03c800 nid=0x179 runnable [0x00007f5a34165000]
> java.lang.Thread.State: RUNNABLE
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
> at
> org.apache.flink.runtime.blob.BlobOutputStream.write(BlobOutputStream.java:88)
> at org.apache.flink.util.IOUtils.copyBytes(IOUtils.java:65)
> at
> org.apache.flink.runtime.blob.BlobClient.putInputStream(BlobClient.java:368)
> at org.apache.flink.runtime.blob.BlobClient.uploadFile(BlobClient.java:428)
> at
> org.apache.flink.runtime.client.ClientUtils.uploadUserJars(ClientUtils.java:102)
> at
> org.apache.flink.runtime.client.ClientUtils.uploadAndSetUserJars(ClientUtils.java:95)
> at
> org.apache.flink.runtime.client.ClientUtils.uploadJobGraphFiles(ClientUtils.java:77)
> at
> org.apache.flink.runtime.client.ClientUtils.extractAndUploadJobGraphFiles(ClientUtils.java:57)
> at
> org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$1(RedJarRunHandler.java:92)
> at
> org.apache.flink.runtime.webmonitor.handlers.JarRunHandler$$Lambda$770/941040494.apply(Unknown
> Source)
> at
> java.util.concurrent.CompletableFuture.biApply(CompletableFuture.java:1105)
> at
> java.util.concurrent.CompletableFuture$BiApply.tryFire(CompletableFuture.java:1070)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1595)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
> Thank you very much for your reply, forideal
>
>
>
>
--
Arvid Heise | Senior Java Developer
<https://www.ververica.com/>
Follow us @VervericaData
--
Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Toni) Cheng