You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Tomasz Rojek <to...@collibra.com> on 2017/04/11 13:52:49 UTC
Kafka on windows
Hi All,
We want to choose provider of messaging system in our company, one of
possible choices is Apache Kafka. One of operating system that will host
brokers is windows, according to documentation:
https://kafka.apache.org/documentation.html#os
"*We have seen a few issues running on Windows and Windows is not currently
a well supported platform though we would be happy to change that.*"
Can you please elaborate more on this. What exactly potential issues are we
talking about? What functionalities of kafka are influenced by this? Maybe
it occurs only on specific version of windows?
Thank you in advance for any information.
With Regards
Tomasz Rojek
Java Engineer
Re: Kafka on windows
Posted by Daniel Hinojosa <dh...@evolutionnext.com>.
Kafka uses Linux OS page cache and flushing techniques as well as in-sync
replicas to bring down latency since storing to the hard drive is likely
the slowest part. It seems it would be defeating the point if you were to
install it on Windows, just like everyone here mentioned. ;)
Just wanted to add some links for your reference, the second one is pretty
good:
https://kafka.apache.org/08/documentation.html#linuxflush
https://www.quora.com/Kafka-writes-every-message-to-broker-disk-Still-performance-wise-it-is-better-than-some-of-the-in-memory-message-storing-message-queues-Why-is-that
On Fri, Apr 14, 2017 at 10:13 AM, David Garcia <da...@spiceworks.com>
wrote:
> If you want reduce maintenance headaches, I highly recommend not running
> brokers on windows (or in vm’s for that matter). It really isn’t
> supported. Kafka needs highly performant OS primitives. For example, XFS,
> and ext4 are recommended filesystems for reasons specific to Kafka. If
> this is just for experimentation/curiosity, then go all out…but production
> systems should stick with the documentation’s recommendations.
>
> On 4/12/17, 12:13 AM, "David Luu" <da...@mistsys.com> wrote:
>
> I'm curious as well. That doc blurb doesn't give specifics. How is
> kafka
> run (or tested) on Windows? Natively via the command line shell, etc.
> or
> via cygwin, within a *nix VM on Windows, or via Windows 10's Ubuntu
> Linux
> Bash shell? Would be interesting to see how each method I listed
> performs,
> maybe the Windows 10 bash shell method might be most optimal among the
> list?
>
> On Tue, Apr 11, 2017 at 9:41 PM, David Garcia <da...@spiceworks.com>
> wrote:
>
> > One issue is that Kafka leverage some very specific features of the
> linux
> > kernel that are probably far different from Windows, so I imagine the
> > performance profile is likewise much different.
> >
> > On 4/11/17, 8:52 AM, "Tomasz Rojek" <to...@collibra.com>
> wrote:
> >
> > Hi All,
> >
> > We want to choose provider of messaging system in our company,
> one of
> > possible choices is Apache Kafka. One of operating system that
> will
> > host
> > brokers is windows, according to documentation:
> >
> > https://kafka.apache.org/documentation.html#os
> > "*We have seen a few issues running on Windows and Windows is not
> > currently
> > a well supported platform though we would be happy to change
> that.*"
> >
> > Can you please elaborate more on this. What exactly potential
> issues
> > are we
> > talking about? What functionalities of kafka are influenced by
> this?
> > Maybe
> > it occurs only on specific version of windows?
> >
> > Thank you in advance for any information.
> >
> > With Regards
> > Tomasz Rojek
> > Java Engineer
> >
> >
> >
>
>
> --
> David Luu
> Member of Technical Staff
> Mist Systems, Inc.
> 1601 S. De Anza Blvd. #248
> Cupertino, CA 95014
>
>
>
Re: Kafka on windows
Posted by David Garcia <da...@spiceworks.com>.
If you want reduce maintenance headaches, I highly recommend not running brokers on windows (or in vm’s for that matter). It really isn’t supported. Kafka needs highly performant OS primitives. For example, XFS, and ext4 are recommended filesystems for reasons specific to Kafka. If this is just for experimentation/curiosity, then go all out…but production systems should stick with the documentation’s recommendations.
On 4/12/17, 12:13 AM, "David Luu" <da...@mistsys.com> wrote:
I'm curious as well. That doc blurb doesn't give specifics. How is kafka
run (or tested) on Windows? Natively via the command line shell, etc. or
via cygwin, within a *nix VM on Windows, or via Windows 10's Ubuntu Linux
Bash shell? Would be interesting to see how each method I listed performs,
maybe the Windows 10 bash shell method might be most optimal among the list?
On Tue, Apr 11, 2017 at 9:41 PM, David Garcia <da...@spiceworks.com> wrote:
> One issue is that Kafka leverage some very specific features of the linux
> kernel that are probably far different from Windows, so I imagine the
> performance profile is likewise much different.
>
> On 4/11/17, 8:52 AM, "Tomasz Rojek" <to...@collibra.com> wrote:
>
> Hi All,
>
> We want to choose provider of messaging system in our company, one of
> possible choices is Apache Kafka. One of operating system that will
> host
> brokers is windows, according to documentation:
>
> https://kafka.apache.org/documentation.html#os
> "*We have seen a few issues running on Windows and Windows is not
> currently
> a well supported platform though we would be happy to change that.*"
>
> Can you please elaborate more on this. What exactly potential issues
> are we
> talking about? What functionalities of kafka are influenced by this?
> Maybe
> it occurs only on specific version of windows?
>
> Thank you in advance for any information.
>
> With Regards
> Tomasz Rojek
> Java Engineer
>
>
>
--
David Luu
Member of Technical Staff
Mist Systems, Inc.
1601 S. De Anza Blvd. #248
Cupertino, CA 95014
Re: Kafka on windows
Posted by David Luu <da...@mistsys.com>.
I'm curious as well. That doc blurb doesn't give specifics. How is kafka
run (or tested) on Windows? Natively via the command line shell, etc. or
via cygwin, within a *nix VM on Windows, or via Windows 10's Ubuntu Linux
Bash shell? Would be interesting to see how each method I listed performs,
maybe the Windows 10 bash shell method might be most optimal among the list?
On Tue, Apr 11, 2017 at 9:41 PM, David Garcia <da...@spiceworks.com> wrote:
> One issue is that Kafka leverage some very specific features of the linux
> kernel that are probably far different from Windows, so I imagine the
> performance profile is likewise much different.
>
> On 4/11/17, 8:52 AM, "Tomasz Rojek" <to...@collibra.com> wrote:
>
> Hi All,
>
> We want to choose provider of messaging system in our company, one of
> possible choices is Apache Kafka. One of operating system that will
> host
> brokers is windows, according to documentation:
>
> https://kafka.apache.org/documentation.html#os
> "*We have seen a few issues running on Windows and Windows is not
> currently
> a well supported platform though we would be happy to change that.*"
>
> Can you please elaborate more on this. What exactly potential issues
> are we
> talking about? What functionalities of kafka are influenced by this?
> Maybe
> it occurs only on specific version of windows?
>
> Thank you in advance for any information.
>
> With Regards
> Tomasz Rojek
> Java Engineer
>
>
>
--
David Luu
Member of Technical Staff
Mist Systems, Inc.
1601 S. De Anza Blvd. #248
Cupertino, CA 95014
Re: Kafka on windows
Posted by David Garcia <da...@spiceworks.com>.
One issue is that Kafka leverage some very specific features of the linux kernel that are probably far different from Windows, so I imagine the performance profile is likewise much different.
On 4/11/17, 8:52 AM, "Tomasz Rojek" <to...@collibra.com> wrote:
Hi All,
We want to choose provider of messaging system in our company, one of
possible choices is Apache Kafka. One of operating system that will host
brokers is windows, according to documentation:
https://kafka.apache.org/documentation.html#os
"*We have seen a few issues running on Windows and Windows is not currently
a well supported platform though we would be happy to change that.*"
Can you please elaborate more on this. What exactly potential issues are we
talking about? What functionalities of kafka are influenced by this? Maybe
it occurs only on specific version of windows?
Thank you in advance for any information.
With Regards
Tomasz Rojek
Java Engineer