You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by kant kodali <ka...@gmail.com> on 2016/08/24 21:48:04 UTC
How to compute a net (difference) given a bi-directional stream of
numbers using spark streaming?
Hi Guys,
I am new to spark but I am wondering how do I compute the difference given a
bidirectional stream of numbers using spark streaming? To put it more concrete
say Bank A is sending money to Bank B and Bank B is sending money to Bank A
throughout the day such that at any given time we want to compute the difference
to find out who owe's who and how much?
The Algorithm can simply be breaking the stream into small intervals (which
spark streaming looks like it does) and then computing the difference in each
time interval and summing the results of each time interval (assuming the
numbers in one interval doesn't overlap with other intervals).
I would like to know what is the best way to achieve this using spark streaming?
I am also open to any other suggestions in the Algorithm.
Thanks!