You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "zhou (Jira)" <ji...@apache.org> on 2022/07/08 06:27:00 UTC
[jira] [Updated] (FLINK-28455) pyflink tableResult collect to local timeout
[ https://issues.apache.org/jira/browse/FLINK-28455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhou updated FLINK-28455:
-------------------------
Description:
when I used pyflink do this:
??with party_enter_final_result.execute().collect() as results:??
?? for result in results:??????
sometimes TimeoutException occured，the Exception as following:
[2022-07-07 01:18:55,843] \{bash.py:173} INFO - Job has been submitted with JobID 017de55acf2a71552fc293626cfbbe67
[2022-07-07 01:20:02,384] \{bash.py:173} INFO - Traceback (most recent call last):
[2022-07-07 01:20:02,384] \{bash.py:173} INFO - File "/opt/airflow/data/repo/dags/chloe/chloe_counter.py", line 80, in <module>
[2022-07-07 01:20:02,384] \{bash.py:173} INFO - main(date)
[2022-07-07 01:20:02,384] \{bash.py:173} INFO - File "/opt/airflow/data/repo/dags/chloe/chloe_counter.py", line 53, in main
[2022-07-07 01:20:02,384] \{bash.py:173} INFO - for result in results:
[2022-07-07 01:20:02,384] \{bash.py:173} INFO - File "/space/flink/opt/python/pyflink.zip/pyflink/table/table_result.py", line 236, in __next__
[2022-07-07 01:20:02,384] \{bash.py:173} INFO - File "/space/flink/opt/python/py4j-0.10.8.1-src.zip/py4j/java_gateway.py", line 1285, in __call__
[2022-07-07 01:20:02,384] \{bash.py:173} INFO - File "/space/flink/opt/python/pyflink.zip/pyflink/util/exceptions.py", line 147, in deco
[2022-07-07 01:20:02,384] \{bash.py:173} INFO - File "/space/flink/opt/python/py4j-0.10.8.1-src.zip/py4j/protocol.py", line 326, in get_return_value
[2022-07-07 01:20:02,384] \{bash.py:173} INFO - py4j.protocol.Py4JJavaError: An error occurred while calling o66.hasNext.
[2022-07-07 01:20:02,384] \{bash.py:173} INFO - : java.lang.RuntimeException: Failed to fetch next result
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:109)
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:80)
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:370)
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - at java.lang.reflect.Method.invoke(Method.java:498)
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.api.python.shaded.py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.api.python.shaded.py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.api.python.shaded.py4j.Gateway.invoke(Gateway.java:282)
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.api.python.shaded.py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.api.python.shaded.py4j.commands.CallCommand.execute(CallCommand.java:79)
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.api.python.shaded.py4j.GatewayConnection.run(GatewayConnection.java:238)
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - at java.lang.Thread.run(Thread.java:748)
[2022-07-07 01:20:02,385] \{bash.py:173} INFO - Caused by: java.io.IOException: Failed to fetch job execution result
[2022-07-07 01:20:02,386] \{bash.py:173} INFO - at org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.getAccumulatorResults(CollectResultFetcher.java:177)
[2022-07-07 01:20:02,386] \{bash.py:173} INFO - at org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.next(CollectResultFetcher.java:120)
[2022-07-07 01:20:02,386] \{bash.py:173} INFO - at org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:106)
[2022-07-07 01:20:02,386] \{bash.py:173} INFO - ... 13 more
[2022-07-07 01:20:02,386] \{bash.py:173} INFO - Caused by: java.util.concurrent.TimeoutException
[2022-07-07 01:20:02,386] \{bash.py:173} INFO - at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)
[2022-07-07 01:20:02,386] \{bash.py:173} INFO - at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
[2022-07-07 01:20:02,386] \{bash.py:173} INFO - at org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.getAccumulatorResults(CollectResultFetcher.java:175)
[2022-07-07 01:20:02,386] \{bash.py:173} INFO - ... 15 more
[2022-07-07 01:20:02,386] \{bash.py:173} INFO -
[2022-07-07 01:20:02,450] \{bash.py:173} INFO - org.apache.flink.client.program.ProgramAbortException: java.lang.RuntimeException: Python process exits with code: 1
> pyflink tableResult collect to local timeout
> ---------------------------------------------
>
> Key: FLINK-28455
> URL: https://issues.apache.org/jira/browse/FLINK-28455
> Project: Flink
> Issue Type: Bug
> Components: API / Core
> Affects Versions: 1.13.0
> Reporter: zhou
> Priority: Major
>
> when I used pyflink do this:
> ??with party_enter_final_result.execute().collect() as results:??
> ?? for result in results:??????
> sometimes TimeoutException occured，the Exception as following:
> [2022-07-07 01:18:55,843] \{bash.py:173} INFO - Job has been submitted with JobID 017de55acf2a71552fc293626cfbbe67
> [2022-07-07 01:20:02,384] \{bash.py:173} INFO - Traceback (most recent call last):
> [2022-07-07 01:20:02,384] \{bash.py:173} INFO - File "/opt/airflow/data/repo/dags/chloe/chloe_counter.py", line 80, in <module>
> [2022-07-07 01:20:02,384] \{bash.py:173} INFO - main(date)
> [2022-07-07 01:20:02,384] \{bash.py:173} INFO - File "/opt/airflow/data/repo/dags/chloe/chloe_counter.py", line 53, in main
> [2022-07-07 01:20:02,384] \{bash.py:173} INFO - for result in results:
> [2022-07-07 01:20:02,384] \{bash.py:173} INFO - File "/space/flink/opt/python/pyflink.zip/pyflink/table/table_result.py", line 236, in __next__
> [2022-07-07 01:20:02,384] \{bash.py:173} INFO - File "/space/flink/opt/python/py4j-0.10.8.1-src.zip/py4j/java_gateway.py", line 1285, in __call__
> [2022-07-07 01:20:02,384] \{bash.py:173} INFO - File "/space/flink/opt/python/pyflink.zip/pyflink/util/exceptions.py", line 147, in deco
> [2022-07-07 01:20:02,384] \{bash.py:173} INFO - File "/space/flink/opt/python/py4j-0.10.8.1-src.zip/py4j/protocol.py", line 326, in get_return_value
> [2022-07-07 01:20:02,384] \{bash.py:173} INFO - py4j.protocol.Py4JJavaError: An error occurred while calling o66.hasNext.
> [2022-07-07 01:20:02,384] \{bash.py:173} INFO - : java.lang.RuntimeException: Failed to fetch next result
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:109)
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:80)
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:370)
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - at java.lang.reflect.Method.invoke(Method.java:498)
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.api.python.shaded.py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.api.python.shaded.py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.api.python.shaded.py4j.Gateway.invoke(Gateway.java:282)
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.api.python.shaded.py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.api.python.shaded.py4j.commands.CallCommand.execute(CallCommand.java:79)
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - at org.apache.flink.api.python.shaded.py4j.GatewayConnection.run(GatewayConnection.java:238)
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - at java.lang.Thread.run(Thread.java:748)
> [2022-07-07 01:20:02,385] \{bash.py:173} INFO - Caused by: java.io.IOException: Failed to fetch job execution result
> [2022-07-07 01:20:02,386] \{bash.py:173} INFO - at org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.getAccumulatorResults(CollectResultFetcher.java:177)
> [2022-07-07 01:20:02,386] \{bash.py:173} INFO - at org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.next(CollectResultFetcher.java:120)
> [2022-07-07 01:20:02,386] \{bash.py:173} INFO - at org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:106)
> [2022-07-07 01:20:02,386] \{bash.py:173} INFO - ... 13 more
> [2022-07-07 01:20:02,386] \{bash.py:173} INFO - Caused by: java.util.concurrent.TimeoutException
> [2022-07-07 01:20:02,386] \{bash.py:173} INFO - at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)
> [2022-07-07 01:20:02,386] \{bash.py:173} INFO - at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
> [2022-07-07 01:20:02,386] \{bash.py:173} INFO - at org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.getAccumulatorResults(CollectResultFetcher.java:175)
> [2022-07-07 01:20:02,386] \{bash.py:173} INFO - ... 15 more
> [2022-07-07 01:20:02,386] \{bash.py:173} INFO -
> [2022-07-07 01:20:02,450] \{bash.py:173} INFO - org.apache.flink.client.program.ProgramAbortException: java.lang.RuntimeException: Python process exits with code: 1
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)