RE: steps path to kafka mastery

[S Ahmed <sa...@gmail.com>]

Hello,

This may be a silly question for some but here goes :)

Without real production experience, what steps do you suggest one take to
really have some solid skillz in kafka?

I tend to learn in a structured way, but it just seems that since kafka is
a general purpose tool there isn't really a course per say that teaches you
all things kafka.

There are books but the one I read was more of a tutorial on certain
aspects of kafka but it doesn't give you the insights of building a real
production system.

Any suggestions or tips?

Re: steps path to kafka mastery

[Todd Snyder <ts...@blackberry.com>]

First, find a use that makes sense to you. Are you a developer and want to focus on building applications and tools that interact with and through kafka?

Are you a platform person, who wants to figure out how to build a data ingest platform and connect various systems and data types through kafka?

Are you a data consumer, who wants to learn how to leverage kafka to improve the access to your data and the rate at which you can work with it?

from my perspective, I'd suggest standing up a simple three node cluster and using logstash and elk to play with moving data around and connecting various sources.   Then maybe try setting up the confluent stack, and looking at a more end.to end solution.

There is a whole ecosystem being built around kafka to suit all the varied interests and needs - find the ones that suit yours and start poking at it.

Cheers

Todd.

Sent from my BlackBerry 10 smartphone on the TELUS network.
  Original Message
From: S Ahmed
Sent: Monday, March 28, 2016 10:40 PM
To: kafka-users@incubator.apache.org
Reply To: users@kafka.apache.org
Subject: RE: steps path to kafka mastery


Hello,

This may be a silly question for some but here goes :)

Without real production experience, what steps do you suggest one take to
really have some solid skillz in kafka?

I tend to learn in a structured way, but it just seems that since kafka is
a general purpose tool there isn't really a course per say that teaches you
all things kafka.

There are books but the one I read was more of a tutorial on certain
aspects of kafka but it doesn't give you the insights of building a real
production system.

Any suggestions or tips?

Re: steps path to kafka mastery

[Muqtafi Akhmad <mu...@traveloka.com>]

Hello Ahmed,

I am new with Kafka myself, from my experience there are some things you
need to do before deploy Kafka in production
1. read Kafka documentation : key concepts, its architecture, etc
2. learn how to deploy Kafka cluster, this is additional but there are some
management tools that you might want to use to ease Kafka deployment e.g.
Ambari
3. learn how to configure Kafka producer, consumer, and broker properly,
e.g.
    in producer : use send asynchronous and buffering for better write
performance, etc
    in consumer : don't use auto offset commit, etc
    in broker : set 2 or 3 as default topic replication factor, set unclean
leader election enable to false, etc

These information are just a glimpse of minimal knowledge you need to run
Kafka in production, you can find articles that can help you to find more
comprehensive tips.

Regards,


On Tue, Mar 29, 2016 at 9:55 AM, Rahul Jain <ra...@gmail.com> wrote:

> You can certainly try out things in a development environment. You could
> also think of contributing to the project itself as that will encourage you
> to develop deeper skills.
>
> But you are right. Kafka is pretty straightforward to setup and get
> running. The real skill IMO is in the ability to troubleshoot when things
> go wrong. And that comes only when you see it running in a production
> environment.
> On 29 Mar 2016 8:10 a.m., "S Ahmed" <sa...@gmail.com> wrote:
>
> > Hello,
> >
> > This may be a silly question for some but here goes :)
> >
> > Without real production experience, what steps do you suggest one take to
> > really have some solid skillz in kafka?
> >
> > I tend to learn in a structured way, but it just seems that since kafka
> is
> > a general purpose tool there isn't really a course per say that teaches
> you
> > all things kafka.
> >
> > There are books but the one I read was more of a tutorial on certain
> > aspects of kafka but it doesn't give you the insights of building a real
> > production system.
> >
> > Any suggestions or tips?
> >
>



-- 
Muqtafi Akhmad
Software Engineer
Traveloka

RE: steps path to kafka mastery

[Rahul Jain <ra...@gmail.com>]

You can certainly try out things in a development environment. You could
also think of contributing to the project itself as that will encourage you
to develop deeper skills.

But you are right. Kafka is pretty straightforward to setup and get
running. The real skill IMO is in the ability to troubleshoot when things
go wrong. And that comes only when you see it running in a production
environment.
On 29 Mar 2016 8:10 a.m., "S Ahmed" <sa...@gmail.com> wrote:

> Hello,
>
> This may be a silly question for some but here goes :)
>
> Without real production experience, what steps do you suggest one take to
> really have some solid skillz in kafka?
>
> I tend to learn in a structured way, but it just seems that since kafka is
> a general purpose tool there isn't really a course per say that teaches you
> all things kafka.
>
> There are books but the one I read was more of a tutorial on certain
> aspects of kafka but it doesn't give you the insights of building a real
> production system.
>
> Any suggestions or tips?
>

Re: steps path to kafka mastery

[S Ahmed <sa...@gmail.com>]

The book says Feb. 2016 release date, looks like the producer is slower
than the consumers  (sorry couldn't resist)

On Tue, Mar 29, 2016 at 11:56 AM, S Ahmed <sa...@gmail.com> wrote:

> It was some pakt pub one.
>
> Yeah I am waiting for that book to be released!
>
> On Tue, Mar 29, 2016 at 6:11 AM, Ben Stopford <be...@confluent.io> wrote:
>
>> Not sure which book you read but, based on the first few chapters, this
>> book <http://shop.oreilly.com/product/0636920044123.do> is an worthy
>> investment.
>>
>> B
>> > On 29 Mar 2016, at 03:40, S Ahmed <sa...@gmail.com> wrote:
>> >
>> > Hello,
>> >
>> > This may be a silly question for some but here goes :)
>> >
>> > Without real production experience, what steps do you suggest one take
>> to
>> > really have some solid skillz in kafka?
>> >
>> > I tend to learn in a structured way, but it just seems that since kafka
>> is
>> > a general purpose tool there isn't really a course per say that teaches
>> you
>> > all things kafka.
>> >
>> > There are books but the one I read was more of a tutorial on certain
>> > aspects of kafka but it doesn't give you the insights of building a real
>> > production system.
>> >
>> > Any suggestions or tips?
>>
>>
>

Re: steps path to kafka mastery

[S Ahmed <sa...@gmail.com>]

It was some pakt pub one.

Yeah I am waiting for that book to be released!

On Tue, Mar 29, 2016 at 6:11 AM, Ben Stopford <be...@confluent.io> wrote:

> Not sure which book you read but, based on the first few chapters, this
> book <http://shop.oreilly.com/product/0636920044123.do> is an worthy
> investment.
>
> B
> > On 29 Mar 2016, at 03:40, S Ahmed <sa...@gmail.com> wrote:
> >
> > Hello,
> >
> > This may be a silly question for some but here goes :)
> >
> > Without real production experience, what steps do you suggest one take to
> > really have some solid skillz in kafka?
> >
> > I tend to learn in a structured way, but it just seems that since kafka
> is
> > a general purpose tool there isn't really a course per say that teaches
> you
> > all things kafka.
> >
> > There are books but the one I read was more of a tutorial on certain
> > aspects of kafka but it doesn't give you the insights of building a real
> > production system.
> >
> > Any suggestions or tips?
>
>

Re: steps path to kafka mastery

[Ben Stopford <be...@confluent.io>]

Not sure which book you read but, based on the first few chapters, this book <http://shop.oreilly.com/product/0636920044123.do> is an worthy investment. 

B
> On 29 Mar 2016, at 03:40, S Ahmed <sa...@gmail.com> wrote:
> 
> Hello,
> 
> This may be a silly question for some but here goes :)
> 
> Without real production experience, what steps do you suggest one take to
> really have some solid skillz in kafka?
> 
> I tend to learn in a structured way, but it just seems that since kafka is
> a general purpose tool there isn't really a course per say that teaches you
> all things kafka.
> 
> There are books but the one I read was more of a tutorial on certain
> aspects of kafka but it doesn't give you the insights of building a real
> production system.
> 
> Any suggestions or tips?