You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Elias Levy <fe...@gmail.com> on 2016/10/08 03:03:41 UTC
Convert a KStream to KTable
I am correct in assuming there is no way to convert a KStream into a
KTable, similar to KTable.toStream() but in the reverse direction, other
than using KSteam.reduceByKey and a Reducer or looping back through Kafka
and using KStreamBuilder.table?
Re: Convert a KStream to KTable
Posted by Michael Noll <mi...@confluent.io>.
Elias,
yes, that is correct.
I also want to explain why:
One can always convert a KTable to a KStream (note: this is the opposite
direction of what you want to do) because one only needs to iterate through
the table to generate the stream.
To convert a KStream into a KTable (what you want to do) the user must tell
Kafka how multiple values of the same key should be processed to end up
with just a single value for that key (a KTable has only a single value for
each key). This step -- the semantics desired by the user of how to
"squash" multiple values into a single value -- is something that Kafka
cannot second-guess. Some users want to compute a sum and thus add all the
values for a key into a final "sum" value; others may want to compute
MAX(), and so on. This explains why there's no KStream#toTable()
equivalent to KTable#toStream().
Hope this helps!
Michael
On Sat, Oct 8, 2016 at 5:03 AM, Elias Levy <fe...@gmail.com>
wrote:
> I am correct in assuming there is no way to convert a KStream into a
> KTable, similar to KTable.toStream() but in the reverse direction, other
> than using KSteam.reduceByKey and a Reducer or looping back through Kafka
> and using KStreamBuilder.table?
>
Re: Convert a KStream to KTable
Posted by "Matthias J. Sax" <ma...@confluent.io>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
That is correct.
On 10/7/16 8:03 PM, Elias Levy wrote:
> I am correct in assuming there is no way to convert a KStream into
> a KTable, similar to KTable.toStream() but in the reverse
> direction, other than using KSteam.reduceByKey and a Reducer or
> looping back through Kafka and using KStreamBuilder.table?
>
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - https://gpgtools.org
iQIcBAEBCgAGBQJX+IiTAAoJECnhiMLycopPqtcQAKbGoBBtwEU98oGHvcPH6ZDF
dz2Htcn8E2GreDTIZCcNQmcx2oBLrrjl1W2YVbiqFsjOgTk+JqOcjxQEi/39FeA4
q8+f1TLCbSP7l3VDIElQearb0QMEMqCTtu64WKhahLaH3t7yIhxgoIU7TB2wjdBs
skYVbY+cwIx4lAvXxJDqDTsWJO7chbL1m402Gd+r1Pb3JNHSTra1jPrha09SVVIJ
Y0PWhAN34md7USfAQE5Xph6ybhrlom8HAoKPzugsZ7WSJriWlke8nnzJzh3cgDNd
aT7oLA49I4Fuvg9y78Er7XS3jWQ5QwR747UxcgGw0AoC2FfvQh5z1vccHAZEi6WE
aaCfzaXjHLI7JJWwWQ3/KZNAbEjqTtQz5k7ufmZLzh5AAjLvhYBff1a2EfQ2+X+1
5m5PN2AkpFj/STvL4rm8sPC2xiF6rUYMBMjnkF2EncLuPEv1F0eqzJIzY7BMfmpC
4S5nwhdltVzQlMh1EAkDLunfBjgVp0hCa0f4y52xmzLGh47GeXTeExyp82fgE4U9
bC9a8kao0B4WEgrr2iUDtTq8EfBr8w4C96IcnQKeON1gl9Cwua4zHIFpWO7CxHRx
vofRQiPYTSjbFn/cSu8K6wCu3W9C39218Z01MV9Xedwf5vl/Aj/mzqdjUrV1zTmF
QfCFGyG49f48z5i5b1rX
=34cP
-----END PGP SIGNATURE-----