You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Reinier Kip (JIRA)" <ji...@apache.org> on 2017/08/31 11:56:00 UTC
[jira] [Updated] (BEAM-2831) Possible bug in Beam+Flink memory
management, disk spillover
[ https://issues.apache.org/jira/browse/BEAM-2831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Reinier Kip updated BEAM-2831:
------------------------------
Description:
I’ve been running a Beam pipeline on Flink. Depending on the dataset size and the heap memory configuration of the jobmanager and taskmanager, I may run into an EOFException, which causes the job to fail.
As [discussed on Flink's mailinglist|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/EOFException-related-to-memory-segments-during-run-of-Beam-pipeline-on-Flink-td15255.html] (stacktrace enclosed), Flink catches these EOFExceptions and activates disk spillover. Because Beam wraps these exceptions, this mechanism fails, the exception travels up the stack, and the job aborts.
Hopefully this is enough information and this is something that can be adjusted for in Beam. I'd be glad to provide more information where needed.
was:
I’ve been running a Beam pipeline on Flink. Depending on the dataset size and the heap memory configuration of the jobmanager and taskmanager, I may run into an EOFException, which causes the job to fail.
As [discussed on Flink's mailinglist|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/EOFException-related-to-memory-segments-during-run-of-Beam-pipeline-on-Flink-td15255.html] (stacktrace enclosed), Flink catches these EOFExceptions and activates disk spillover. Because Beam wraps these exceptions, this mechanism fails, the exception travels up the stack, and the job aborts.
EOFException occurs with fairly small datasets (tens of megabytes), 768MiB of Flink heap memory, Beam 2.0.0/2.1.0, Flink 1.2.1/1.3.0, Java 8.
Hopefully this is enough information and this is something that can be adjusted for in Beam. I'd be glad to provide more information where needed.
> Possible bug in Beam+Flink memory management, disk spillover
> ------------------------------------------------------------
>
> Key: BEAM-2831
> URL: https://issues.apache.org/jira/browse/BEAM-2831
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Affects Versions: 2.0.0, 2.1.0
> Environment: Flink 1.2.1 and 1.3.0, Java 8, macOS 10.12.6 and RedHat 6
> Reporter: Reinier Kip
> Assignee: Aljoscha Krettek
>
> I’ve been running a Beam pipeline on Flink. Depending on the dataset size and the heap memory configuration of the jobmanager and taskmanager, I may run into an EOFException, which causes the job to fail.
> As [discussed on Flink's mailinglist|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/EOFException-related-to-memory-segments-during-run-of-Beam-pipeline-on-Flink-td15255.html] (stacktrace enclosed), Flink catches these EOFExceptions and activates disk spillover. Because Beam wraps these exceptions, this mechanism fails, the exception travels up the stack, and the job aborts.
> Hopefully this is enough information and this is something that can be adjusted for in Beam. I'd be glad to provide more information where needed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)