Spark Streaming Latency in practice

[xweb <as...@gmail.com>]

What kind of latency are people in achieving in production using spark
streaming? Is it in 1 second+ range. or have people been able to achieve
latency in say 250 ms range. 

Any best practices on achieving sub second latency if even possible?



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Latency-in-practice-tp25038.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: Spark Streaming Latency in practice

[Ilya Ganelin <il...@gmail.com>]

Others feel free to chime in here, the recommendation I've seen is to not
reduce the batch size below 500ms since the Spark machinery gets in the way
below that point.

If you're looking for latency at the millisecond scale then I'd recommend
checking out Apache Apex, Storm, and Flink. All of those are capable of
single digit millisecond processing.
On Mon, Oct 12, 2015 at 3:35 PM xweb <as...@gmail.com> wrote:

> What kind of latency are people in achieving in production using spark
> streaming? Is it in 1 second+ range. or have people been able to achieve
> latency in say 250 ms range.
>
> Any best practices on achieving sub second latency if even possible?
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Latency-in-practice-tp25038.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>