Kafka Stream vs Spark

[Mina Aslani <as...@gmail.com>]

Hi,

I created a kafka stream app and as I was informed I created a docker image
with the app and launched it as a container. However, I have couple of
questions:

- Would every Kafka streaming job require a new docker image and deployment
of the container/service?
- How should I structure things differently if I had more than one Kafka
streaming app/job?
- What are the advantages of using Kafka streaming over Spark streaming?
I'm asking b/c with Spark streaming I don't need to create and deploy a new
docker image every time I added or changed an/a app/job.

Best regards,
Mina

Re: Kafka Stream vs Spark

[David Garcia <da...@spiceworks.com>]

Unlike spark, you don’t need an entire framework to deploy your job.  With Kstreams, you just start up an application and go.  You don’t need docker either…although containerizing your stuff is probably a good strategy for the purposes of deployment management (something you get with Yarn or a spark Cluster)…but you’re not tied to any one framework (e.g. you can use kubernetes, mesos, Yarn, or anything else) 

On 4/27/17, 10:52 AM, "Mina Aslani" <as...@gmail.com> wrote:

    Hi,
    
    I created a kafka stream app and as I was informed I created a docker image
    with the app and launched it as a container. However, I have couple of
    questions:
    
    - Would every Kafka streaming job require a new docker image and deployment
    of the container/service?
    - How should I structure things differently if I had more than one Kafka
    streaming app/job?
    - What are the advantages of using Kafka streaming over Spark streaming?
    I'm asking b/c with Spark streaming I don't need to create and deploy a new
    docker image every time I added or changed an/a app/job.
    
    Best regards,
    Mina