You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Ismaël Mejía (Jira)" <ji...@apache.org> on 2020/03/05 14:54:00 UTC

[jira] [Updated] (BEAM-9451) Optimize translation when Schema information is available in Spark Structured Streaming runner

     [ https://issues.apache.org/jira/browse/BEAM-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ismaël Mejía updated BEAM-9451:
-------------------------------
    Summary: Optimize translation when Schema information is available in Spark Structured Streaming runner  (was: Optimize translations when Schema information is available in Spark Structured Streaming runner)

> Optimize translation when Schema information is available in Spark Structured Streaming runner
> ----------------------------------------------------------------------------------------------
>
>                 Key: BEAM-9451
>                 URL: https://issues.apache.org/jira/browse/BEAM-9451
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-spark
>            Reporter: Ismaël Mejía
>            Priority: Major
>              Labels: structured-streaming
>
> Spark Structured Streaming runner supports Datasets that already have Schema information. This is used by Spark to optimize jobs (via Catalyst). This issue is to implement optimized transforms for the runner so we can benefit of the performance improvements internally done by Spark.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)