You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2025/11/21 21:49:00 UTC

[jira] [Assigned] (SPARK-53797) File stream source maxBytesPerTrigger is incredibly slow

     [ https://issues.apache.org/jira/browse/SPARK-53797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dongjoon Hyun reassigned SPARK-53797:
-------------------------------------

    Assignee: Adam Binford

> File stream source maxBytesPerTrigger is incredibly slow
> --------------------------------------------------------
>
>                 Key: SPARK-53797
>                 URL: https://issues.apache.org/jira/browse/SPARK-53797
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL, Structured Streaming
>    Affects Versions: 4.0.1
>            Reporter: Adam Binford
>            Assignee: Adam Binford
>            Priority: Major
>              Labels: pull-request-available
>
> Using the new maxBytesPerTrigger setting for file stream sources is incredibly slow in building the batch for sources with a large number of files



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org