You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by harish jadhav <ha...@yahoo.com.INVALID> on 2017/07/14 20:24:16 UTC
Windows OS platform support
Hello Team,
I am exploring Apache Kafka and found that one of the best MQ I have encountered. I was exploring option to use it in Windows machine and started some kind of proof of concept work referring installation section on windows and it work perfectly. Later realized that Kafka documentation says under Hardware and OS section " We have seen a few issues running on Windows and Windows is not currently a well supported platform though we would be happy to change that. "
I am curios to know is there any actual issue running Kafka in windows OS as whichever feature I am using with single instance ( Producer-Consumer) works perfectly in windows test bed. Can I use it production windows machines?
Please advice.
Thanks
Harish
Re: Windows OS platform support
Posted by Matt Andruff <ma...@gmail.com>.
This is a really common issue with windows. You can try turning off
backups/virus scanning of that folder and sometimes this relieves the
issue. (I assume there is a file attribute you can alter.) It's probably
the case that Kafka itself has the file open, as linux does not have issues
if you move a file while it's being written to or open.
If you are interested in the locking process (and tools that help you
delete or 'unlock the file' on windows see this post
<https://superuser.com/questions/117902/find-out-which-process-is-locking-a-file-or-folder-in-windows>.)
Might also help provide enough information for you to file a bug in jira
against the issue.
Hope this helps.
On Mon, 17 Jul 2017 at 04:45 M. Manna <ma...@gmail.com> wrote:
> Hi,
>
> First I apologise and should just correct myself. I should use "Disk/File
> level operations" instead of using "I/O" since that refers to different
> things.
>
> What I mean to say that file deletion/renaming doesn't work even when you
> provide full permissions in Windows. But Unix/Linux based system it's okay.
> This can be called as "Deletion/Renaming of shared files" i.e. which are
> already opened for appending/editing.
>
> As an example try to run the uni tests for Kafka on Windows where file
> renaming/deletion occurs as part of the tests.They will fail. But same
> process works all good on Linux.
>
> From the official documentation, Kafka has already made a disclaimer that
> Windows Implementation is not fully vetted (in progress). I believe all the
> top examples from LinkedIn are based on Unix/Linux. You are actually okay
> to do PoC activities on windows - log retention/rename/deletion will work
> fine.
>
> I hope this helps you understand.
>
> KR,
>
> On 17 July 2017 at 09:22, harish jadhav <ha...@yahoo.com>
> wrote:
>
> > Hi M.Manna,
> >
> >
> > Thanks for the update.
> >
> > If possible, can you please share which admin I/O level functionality
> will
> > impact? Do you mean something related to log files renaming?
> >
> > In documentation, there is no pointer with respect to I/O.
> >
> > Kindly advice.
> >
> > Thanks
> > Harish
> >
> >
> >
> > On Saturday, July 15, 2017, 1:59:58 PM GMT+5:30, M. Manna <
> > manmedia@gmail.com> wrote:
> >
> >
> > Kafka has some issues wuth IO level admin functionalities e.g. file
> > deletion/renaming doesn't work out of the box like Linux. However, user
> > base is growing and there's a momentum to support Windows OS seamlessly
> > like Linux.
> >
> > I have been running some stress tests on Windows and the performance is
> > quite consistent. If you have any findings you would like to share,
> please
> > publish them.
> >
> > Please read Kafka documentation where there are some disclaimers
> regarding
> > OS support.
> >
> > On 15 Jul 2017 1:27 am, "Eno Thereska" <en...@gmail.com> wrote:
> >
> > Hi Harish,
> >
> > I believe many people/orgs use it on Windows. We rely on the community to
> > test/fix/answer any Windows questions, same as with Linux or MacOS.
> > However, based on what I've observed, perhaps there are more people
> > answering Linux-related questions.
> >
> > Eno
> >
> > > On 14 Jul 2017, at 13:24, harish jadhav <harishjadhav1979@yahoo.com.
> > INVALID> wrote:
> > >
> > > Hello Team,
> > >
> > > I am exploring Apache Kafka and found that one of the best MQ I have
> > encountered. I was exploring option to use it in Windows machine and
> > started some kind of proof of concept work referring installation section
> > on windows and it work perfectly. Later realized that Kafka documentation
> > says under Hardware and OS section " We have seen a few issues running on
> > Windows and Windows is not currently a well supported platform though we
> > would be happy to change that. "
> > >
> > > I am curios to know is there any actual issue running Kafka in windows
> > OS as whichever feature I am using with single instance (
> > Producer-Consumer) works perfectly in windows test bed. Can I use it
> > production windows machines?
> > >
> > > Please advice.
> > >
> > > Thanks
> > > Harish
> >
> >
>
Re: Windows OS platform support
Posted by "M. Manna" <ma...@gmail.com>.
I have never said that that we shouldn't use Kafka officially for Windows
Production Environment. I am running a major PoC which involves developing
solution on Windows 10 - but my production docker containers are using
Alpine Linux so I won't have any issues for the final integration pipeline.
What I would like to clarify is that if you are going for a full Windows
pipeline for your implementation (i.e. dev+delivery both on Windows
platoform) - you might want to do more research. This user group is awesome
and everyone is very active in getting all the issues resolved asap.
We have seen good performance on Windows and they are more or less
consistent with all the metrics published in Kafka - if you don't have a
strict requirement to do log cleanup you can use the workaround I
mentioned, or just clean the logs yourself.
What is your implementation requirement, exactly? Do you need to preserve
all the messages (i.e. ability to replay)? Or would you be happy to delete
them during a scheduled maintenance window ?
KR,
On 17 July 2017 at 16:19, harish jadhav <ha...@yahoo.com> wrote:
> Hello M.Manna,
>
> Do you mean we should not use Kafka in windows production until the disk
> I/O issues(For which you have given workaround) are resolved officilally?
>
>
> Thanks
> Harish
>
> On Monday, July 17, 2017, 5:31:11 PM GMT+5:30, M. Manna <
> manmedia@gmail.com> wrote:
>
>
> The workaround is to download the sources and create a wrapper around
> method forceUnmap in LogSegment.scala file. Please do not take it a
> solution as no one has approved this yet on KAFKA-1194 ticket. but it works.
>
> I tried to update the scala sources to run the unit tests - but it fails
> all the time. Howerver, the fix works. Also, the log files are actually
> okay, it's the index and timeIndex files which throws RTE.
>
> Do the following (Prerequisite - you have to follow the src download and
> setup instruction from Kafka website - Also use 0.10.2.1 or higher):
>
> 1) Create a new wrapper in AbstractIndex.scala directly under *def
> forceUnmap*
>
> // Forces unmapping of multibyte map files on Windows
> def forceUnmapOnWindows() {
> if (OperatingSystem.IS_WINDOWS)
> forceUnmap(mmap)
> }
>
> 2. Find the method changeFileSuffixes and invoke the above method (i.e.
> forceUnmapOnWindows() ) for index and timeIndex files - just before:
>
> i) The try catch block with index file
> ii) The try catch block with timeIndex file.
>
> So it will become the following:
>
> def changeFileSuffixes(oldSuffix: String, newSuffix: String) {
> def kafkaStorageException(fileType: String, e: IOException) =
> new KafkaStorageException(s"Failed to change the $fileType file
> suffix from $oldSuffix to $newSuffix for log segment $baseOffset", e)
> try log.renameTo(new File(CoreUtils.replaceSuffix(log.file.getPath,
> oldSuffix, newSuffix)))
> catch {
> case e: IOException => throw kafkaStorageException("log", e)
> }
> index.forceUnmapOnWindows()
> try index.renameTo(new File(CoreUtils.replaceSuffix(index.file.getPath,
> oldSuffix, newSuffix)))
> catch {
> case e: IOException => throw kafkaStorageException("index", e)
> }
> timeIndex.forceUnmapOnWindows()
> try timeIndex.renameTo(new File(CoreUtils.replaceSuffix(timeIndex.file.getPath,
> oldSuffix, newSuffix)))
> catch {
> case e: IOException => throw kafkaStorageException("timeindex", e)
> }
> }
>
>
> 3. Execute gradlew -jar --debug (REMEMBER - you need to setup everything
> with the project beforehand, otherwise it won't work).
>
> Once the build is complete - get the core jar and dump it in your kafka
> broker lib folder (it will replace the old one).
>
> 4. Redeploy and check if the cleanup is working.
>
> KR,
>
> On 17 July 2017 at 12:21, harish jadhav <ha...@yahoo.com>
> wrote:
>
> Hello M.Manna,
>
>
> Thanks for your quick and valuable response.
>
> Actually I am getting following error -
>
> Caused by: java.nio.file. FileSystemException: \tmp\kafka-logs\*TopicName*-2\
> 00000000000000007262.log -> \tmp\kafka-logs\TopicName
> -2\00000000000000007262.log. deleted: The process cannot access the file
> because it is being used by another process.
>
> Uncaught exception in scheduled task 'kafka-log-retention'
> (kafka.utils.KafkaScheduler)
>
> kafka.common. KafkaStorageException: Failed to change the log file suffix
> from to .deleted for log segment 7262
>
> There must be some workaround available for above issues. However I don't
> know what other potential issues can come in future. Hence need your
> suggestion on whether to continue to use Kafka in windows production system.
>
> Please guide us on this.
>
> Thanks
> Harish
>
>
> On Monday, July 17, 2017, 2:15:15 PM GMT+5:30, M. Manna <
> manmedia@gmail.com> wrote:
>
>
> Hi,
>
> First I apologise and should just correct myself. I should use "Disk/File
> level operations" instead of using "I/O" since that refers to different
> things.
>
> What I mean to say that file deletion/renaming doesn't work even when you
> provide full permissions in Windows. But Unix/Linux based system it's okay.
> This can be called as "Deletion/Renaming of shared files" i.e. which are
> already opened for appending/editing.
>
> As an example try to run the uni tests for Kafka on Windows where file
> renaming/deletion occurs as part of the tests.They will fail. But same
> process works all good on Linux.
>
> From the official documentation, Kafka has already made a disclaimer that
> Windows Implementation is not fully vetted (in progress). I believe all the
> top examples from LinkedIn are based on Unix/Linux. You are actually okay
> to do PoC activities on windows - log retention/rename/deletion will work
> fine.
>
> I hope this helps you understand.
>
> KR,
>
> On 17 July 2017 at 09:22, harish jadhav <ha...@yahoo.com>
> wrote:
>
> Hi M.Manna,
>
>
> Thanks for the update.
>
> If possible, can you please share which admin I/O level functionality will
> impact? Do you mean something related to log files renaming?
>
> In documentation, there is no pointer with respect to I/O.
>
> Kindly advice.
>
> Thanks
> Harish
>
>
>
> On Saturday, July 15, 2017, 1:59:58 PM GMT+5:30, M. Manna <
> manmedia@gmail.com> wrote:
>
>
> Kafka has some issues wuth IO level admin functionalities e.g. file
> deletion/renaming doesn't work out of the box like Linux. However, user
> base is growing and there's a momentum to support Windows OS seamlessly
> like Linux.
>
> I have been running some stress tests on Windows and the performance is
> quite consistent. If you have any findings you would like to share, please
> publish them.
>
> Please read Kafka documentation where there are some disclaimers regarding
> OS support.
>
> On 15 Jul 2017 1:27 am, "Eno Thereska" <en...@gmail.com> wrote:
>
> Hi Harish,
>
> I believe many people/orgs use it on Windows. We rely on the community to
> test/fix/answer any Windows questions, same as with Linux or MacOS.
> However, based on what I've observed, perhaps there are more people
> answering Linux-related questions.
>
> Eno
>
> > On 14 Jul 2017, at 13:24, harish jadhav <harishjadhav1979@yahoo.com.
> INVALID> wrote:
> >
> > Hello Team,
> >
> > I am exploring Apache Kafka and found that one of the best MQ I have
> encountered. I was exploring option to use it in Windows machine and
> started some kind of proof of concept work referring installation section
> on windows and it work perfectly. Later realized that Kafka documentation
> says under Hardware and OS section " We have seen a few issues running on
> Windows and Windows is not currently a well supported platform though we
> would be happy to change that. "
> >
> > I am curios to know is there any actual issue running Kafka in windows
> OS as whichever feature I am using with single instance (
> Producer-Consumer) works perfectly in windows test bed. Can I use it
> production windows machines?
> >
> > Please advice.
> >
> > Thanks
> > Harish
>
>
>
>
Re: Windows OS platform support
Posted by harish jadhav <ha...@yahoo.com.INVALID>.
Hello M.Manna,
Do you mean we should not use Kafka in windows production until the disk I/O issues(For which you have given workaround) are resolved officilally?
ThanksHarish
On Monday, July 17, 2017, 5:31:11 PM GMT+5:30, M. Manna <ma...@gmail.com> wrote:
The workaround is to download the sources and create a wrapper around method forceUnmap in LogSegment.scala file. Please do not take it a solution as no one has approved this yet on KAFKA-1194 ticket. but it works.
I tried to update the scala sources to run the unit tests - but it fails all the time. Howerver, the fix works. Also, the log files are actually okay, it's the index and timeIndex files which throws RTE.
Do the following (Prerequisite - you have to follow the src download and setup instruction from Kafka website - Also use 0.10.2.1 or higher):
1) Create a new wrapper in AbstractIndex.scala directly under def forceUnmap
// Forces unmapping of multibyte map files on Windowsdef forceUnmapOnWindows() {if (OperatingSystem.IS_WINDOWS) forceUnmap(mmap)}
2. Find the method changeFileSuffixes and invoke the above method (i.e. forceUnmapOnWindows() ) for index and timeIndex files - just before:
i) The try catch block with index file ii) The try catch block with timeIndex file.
So it will become the following:
def changeFileSuffixes(oldSuffix: String, newSuffix: String) {
def kafkaStorageException(fileType: String, e: IOException) =
new KafkaStorageException(s"Failed to change the $fileType file suffix from $oldSuffix to $newSuffix for log segment $baseOffset", e)
try log.renameTo(new File(CoreUtils.replaceSuffix(log.file.getPath, oldSuffix, newSuffix)))
catch {
case e: IOException => throw kafkaStorageException("log", e)
}
index.forceUnmapOnWindows()
try index.renameTo(new File(CoreUtils.replaceSuffix(index.file.getPath, oldSuffix, newSuffix)))
catch {
case e: IOException => throw kafkaStorageException("index", e)
}
timeIndex.forceUnmapOnWindows()
try timeIndex.renameTo(new File(CoreUtils.replaceSuffix(timeIndex.file.getPath, oldSuffix, newSuffix)))
catch {
case e: IOException => throw kafkaStorageException("timeindex", e)
}
}
3. Execute gradlew -jar --debug (REMEMBER - you need to setup everything with the project beforehand, otherwise it won't work).
Once the build is complete - get the core jar and dump it in your kafka broker lib folder (it will replace the old one).
4. Redeploy and check if the cleanup is working.
KR,
On 17 July 2017 at 12:21, harish jadhav <ha...@yahoo.com> wrote:
Hello M.Manna,
Thanks for your quick and valuable response.
Actually I am getting following error -
Caused by: java.nio.file. FileSystemException: \tmp\kafka-logs\TopicName-2\ 00000000000000007262.log-> \tmp\kafka-logs\TopicName -2\00000000000000007262.log. deleted: Theprocess cannot access the file because it is being used by another process.
Uncaught exception in scheduled task 'kafka-log-retention'(kafka.utils.KafkaScheduler)
kafka.common. KafkaStorageException: Failed to change the log file suffixfrom to .deleted for log segment 7262
There must be some workaround available for above issues. However I don't know what other potential issues can come in future. Hence need your suggestion on whether to continue to use Kafka in windows production system.
Please guide us on this.
ThanksHarish
On Monday, July 17, 2017, 2:15:15 PM GMT+5:30, M. Manna <ma...@gmail.com> wrote:
Hi,
First I apologise and should just correct myself. I should use "Disk/File level operations" instead of using "I/O" since that refers to different things.
What I mean to say that file deletion/renaming doesn't work even when you provide full permissions in Windows. But Unix/Linux based system it's okay. This can be called as "Deletion/Renaming of shared files" i.e. which are already opened for appending/editing.
As an example try to run the uni tests for Kafka on Windows where file renaming/deletion occurs as part of the tests.They will fail. But same process works all good on Linux.
From the official documentation, Kafka has already made a disclaimer that Windows Implementation is not fully vetted (in progress). I believe all the top examples from LinkedIn are based on Unix/Linux. You are actually okay to do PoC activities on windows - log retention/rename/deletion will work fine.
I hope this helps you understand.
KR,
On 17 July 2017 at 09:22, harish jadhav <ha...@yahoo.com> wrote:
Hi M.Manna,
Thanks for the update.
If possible, can you please share which admin I/O level functionality will impact? Do you mean something related to log files renaming?
In documentation, there is no pointer with respect to I/O.
Kindly advice.
ThanksHarish
On Saturday, July 15, 2017, 1:59:58 PM GMT+5:30, M. Manna <ma...@gmail.com> wrote:
Kafka has some issues wuth IO level admin functionalities e.g. file deletion/renaming doesn't work out of the box like Linux. However, user base is growing and there's a momentum to support Windows OS seamlessly like Linux.
I have been running some stress tests on Windows and the performance is quite consistent. If you have any findings you would like to share, please publish them.
Please read Kafka documentation where there are some disclaimers regarding OS support.
On 15 Jul 2017 1:27 am, "Eno Thereska" <en...@gmail.com> wrote:
Hi Harish,
I believe many people/orgs use it on Windows. We rely on the community to test/fix/answer any Windows questions, same as with Linux or MacOS. However, based on what I've observed, perhaps there are more people answering Linux-related questions.
Eno
> On 14 Jul 2017, at 13:24, harish jadhav <harishjadhav1979@yahoo.com. INVALID> wrote:
>
> Hello Team,
>
> I am exploring Apache Kafka and found that one of the best MQ I have encountered. I was exploring option to use it in Windows machine and started some kind of proof of concept work referring installation section on windows and it work perfectly. Later realized that Kafka documentation says under Hardware and OS section " We have seen a few issues running on Windows and Windows is not currently a well supported platform though we would be happy to change that. "
>
> I am curios to know is there any actual issue running Kafka in windows OS as whichever feature I am using with single instance ( Producer-Consumer) works perfectly in windows test bed. Can I use it production windows machines?
>
> Please advice.
>
> Thanks
> Harish
Re: Windows OS platform support
Posted by harish jadhav <ha...@yahoo.com.INVALID>.
Hello M.Manna,
Thanks for your quick and valuable response.
Actually I am getting following error -
Caused by: java.nio.file.FileSystemException: \tmp\kafka-logs\TopicName-2\00000000000000007262.log-> \tmp\kafka-logs\TopicName -2\00000000000000007262.log.deleted: Theprocess cannot access the file because it is being used by another process.
Uncaught exception in scheduled task 'kafka-log-retention'(kafka.utils.KafkaScheduler)
kafka.common.KafkaStorageException: Failed to change the log file suffixfrom to .deleted for log segment 7262
There must be some workaround available for above issues. However I don't know what other potential issues can come in future. Hence need your suggestion on whether to continue to use Kafka in windows production system.
Please guide us on this.
ThanksHarish
On Monday, July 17, 2017, 2:15:15 PM GMT+5:30, M. Manna <ma...@gmail.com> wrote:
Hi,
First I apologise and should just correct myself. I should use "Disk/File level operations" instead of using "I/O" since that refers to different things.
What I mean to say that file deletion/renaming doesn't work even when you provide full permissions in Windows. But Unix/Linux based system it's okay. This can be called as "Deletion/Renaming of shared files" i.e. which are already opened for appending/editing.
As an example try to run the uni tests for Kafka on Windows where file renaming/deletion occurs as part of the tests.They will fail. But same process works all good on Linux.
From the official documentation, Kafka has already made a disclaimer that Windows Implementation is not fully vetted (in progress). I believe all the top examples from LinkedIn are based on Unix/Linux. You are actually okay to do PoC activities on windows - log retention/rename/deletion will work fine.
I hope this helps you understand.
KR,
On 17 July 2017 at 09:22, harish jadhav <ha...@yahoo.com> wrote:
Hi M.Manna,
Thanks for the update.
If possible, can you please share which admin I/O level functionality will impact? Do you mean something related to log files renaming?
In documentation, there is no pointer with respect to I/O.
Kindly advice.
ThanksHarish
On Saturday, July 15, 2017, 1:59:58 PM GMT+5:30, M. Manna <ma...@gmail.com> wrote:
Kafka has some issues wuth IO level admin functionalities e.g. file deletion/renaming doesn't work out of the box like Linux. However, user base is growing and there's a momentum to support Windows OS seamlessly like Linux.
I have been running some stress tests on Windows and the performance is quite consistent. If you have any findings you would like to share, please publish them.
Please read Kafka documentation where there are some disclaimers regarding OS support.
On 15 Jul 2017 1:27 am, "Eno Thereska" <en...@gmail.com> wrote:
Hi Harish,
I believe many people/orgs use it on Windows. We rely on the community to test/fix/answer any Windows questions, same as with Linux or MacOS. However, based on what I've observed, perhaps there are more people answering Linux-related questions.
Eno
> On 14 Jul 2017, at 13:24, harish jadhav <harishjadhav1979@yahoo.com. INVALID> wrote:
>
> Hello Team,
>
> I am exploring Apache Kafka and found that one of the best MQ I have encountered. I was exploring option to use it in Windows machine and started some kind of proof of concept work referring installation section on windows and it work perfectly. Later realized that Kafka documentation says under Hardware and OS section " We have seen a few issues running on Windows and Windows is not currently a well supported platform though we would be happy to change that. "
>
> I am curios to know is there any actual issue running Kafka in windows OS as whichever feature I am using with single instance ( Producer-Consumer) works perfectly in windows test bed. Can I use it production windows machines?
>
> Please advice.
>
> Thanks
> Harish
Re: Windows OS platform support
Posted by "M. Manna" <ma...@gmail.com>.
Hi,
First I apologise and should just correct myself. I should use "Disk/File
level operations" instead of using "I/O" since that refers to different
things.
What I mean to say that file deletion/renaming doesn't work even when you
provide full permissions in Windows. But Unix/Linux based system it's okay.
This can be called as "Deletion/Renaming of shared files" i.e. which are
already opened for appending/editing.
As an example try to run the uni tests for Kafka on Windows where file
renaming/deletion occurs as part of the tests.They will fail. But same
process works all good on Linux.
From the official documentation, Kafka has already made a disclaimer that
Windows Implementation is not fully vetted (in progress). I believe all the
top examples from LinkedIn are based on Unix/Linux. You are actually okay
to do PoC activities on windows - log retention/rename/deletion will work
fine.
I hope this helps you understand.
KR,
On 17 July 2017 at 09:22, harish jadhav <ha...@yahoo.com> wrote:
> Hi M.Manna,
>
>
> Thanks for the update.
>
> If possible, can you please share which admin I/O level functionality will
> impact? Do you mean something related to log files renaming?
>
> In documentation, there is no pointer with respect to I/O.
>
> Kindly advice.
>
> Thanks
> Harish
>
>
>
> On Saturday, July 15, 2017, 1:59:58 PM GMT+5:30, M. Manna <
> manmedia@gmail.com> wrote:
>
>
> Kafka has some issues wuth IO level admin functionalities e.g. file
> deletion/renaming doesn't work out of the box like Linux. However, user
> base is growing and there's a momentum to support Windows OS seamlessly
> like Linux.
>
> I have been running some stress tests on Windows and the performance is
> quite consistent. If you have any findings you would like to share, please
> publish them.
>
> Please read Kafka documentation where there are some disclaimers regarding
> OS support.
>
> On 15 Jul 2017 1:27 am, "Eno Thereska" <en...@gmail.com> wrote:
>
> Hi Harish,
>
> I believe many people/orgs use it on Windows. We rely on the community to
> test/fix/answer any Windows questions, same as with Linux or MacOS.
> However, based on what I've observed, perhaps there are more people
> answering Linux-related questions.
>
> Eno
>
> > On 14 Jul 2017, at 13:24, harish jadhav <harishjadhav1979@yahoo.com.
> INVALID> wrote:
> >
> > Hello Team,
> >
> > I am exploring Apache Kafka and found that one of the best MQ I have
> encountered. I was exploring option to use it in Windows machine and
> started some kind of proof of concept work referring installation section
> on windows and it work perfectly. Later realized that Kafka documentation
> says under Hardware and OS section " We have seen a few issues running on
> Windows and Windows is not currently a well supported platform though we
> would be happy to change that. "
> >
> > I am curios to know is there any actual issue running Kafka in windows
> OS as whichever feature I am using with single instance (
> Producer-Consumer) works perfectly in windows test bed. Can I use it
> production windows machines?
> >
> > Please advice.
> >
> > Thanks
> > Harish
>
>
Re: Windows OS platform support
Posted by harish jadhav <ha...@yahoo.com.INVALID>.
Hi M.Manna,
Thanks for the update.
If possible, can you please share which admin I/O level functionality will impact? Do you mean something related to log files renaming?
In documentation, there is no pointer with respect to I/O.
Kindly advice.
ThanksHarish
On Saturday, July 15, 2017, 1:59:58 PM GMT+5:30, M. Manna <ma...@gmail.com> wrote:
Kafka has some issues wuth IO level admin functionalities e.g. file deletion/renaming doesn't work out of the box like Linux. However, user base is growing and there's a momentum to support Windows OS seamlessly like Linux.
I have been running some stress tests on Windows and the performance is quite consistent. If you have any findings you would like to share, please publish them.
Please read Kafka documentation where there are some disclaimers regarding OS support.
On 15 Jul 2017 1:27 am, "Eno Thereska" <en...@gmail.com> wrote:
Hi Harish,
I believe many people/orgs use it on Windows. We rely on the community to test/fix/answer any Windows questions, same as with Linux or MacOS. However, based on what I've observed, perhaps there are more people answering Linux-related questions.
Eno
> On 14 Jul 2017, at 13:24, harish jadhav <harishjadhav1979@yahoo.com. INVALID> wrote:
>
> Hello Team,
>
> I am exploring Apache Kafka and found that one of the best MQ I have encountered. I was exploring option to use it in Windows machine and started some kind of proof of concept work referring installation section on windows and it work perfectly. Later realized that Kafka documentation says under Hardware and OS section " We have seen a few issues running on Windows and Windows is not currently a well supported platform though we would be happy to change that. "
>
> I am curios to know is there any actual issue running Kafka in windows OS as whichever feature I am using with single instance ( Producer-Consumer) works perfectly in windows test bed. Can I use it production windows machines?
>
> Please advice.
>
> Thanks
> Harish
Re: Windows OS platform support
Posted by "M. Manna" <ma...@gmail.com>.
Kafka has some issues wuth IO level admin functionalities e.g. file
deletion/renaming doesn't work out of the box like Linux. However, user
base is growing and there's a momentum to support Windows OS seamlessly
like Linux.
I have been running some stress tests on Windows and the performance is
quite consistent. If you have any findings you would like to share, please
publish them.
Please read Kafka documentation where there are some disclaimers regarding
OS support.
On 15 Jul 2017 1:27 am, "Eno Thereska" <en...@gmail.com> wrote:
> Hi Harish,
>
> I believe many people/orgs use it on Windows. We rely on the community to
> test/fix/answer any Windows questions, same as with Linux or MacOS.
> However, based on what I've observed, perhaps there are more people
> answering Linux-related questions.
>
> Eno
>
> > On 14 Jul 2017, at 13:24, harish jadhav <ha...@yahoo.com.INVALID>
> wrote:
> >
> > Hello Team,
> >
> > I am exploring Apache Kafka and found that one of the best MQ I have
> encountered. I was exploring option to use it in Windows machine and
> started some kind of proof of concept work referring installation section
> on windows and it work perfectly. Later realized that Kafka documentation
> says under Hardware and OS section " We have seen a few issues running on
> Windows and Windows is not currently a well supported platform though we
> would be happy to change that. "
> >
> > I am curios to know is there any actual issue running Kafka in windows
> OS as whichever feature I am using with single instance (
> Producer-Consumer) works perfectly in windows test bed. Can I use it
> production windows machines?
> >
> > Please advice.
> >
> > Thanks
> > Harish
>
>
Re: Windows OS platform support
Posted by Eno Thereska <en...@gmail.com>.
Hi Harish,
I believe many people/orgs use it on Windows. We rely on the community to test/fix/answer any Windows questions, same as with Linux or MacOS. However, based on what I've observed, perhaps there are more people answering Linux-related questions.
Eno
> On 14 Jul 2017, at 13:24, harish jadhav <ha...@yahoo.com.INVALID> wrote:
>
> Hello Team,
>
> I am exploring Apache Kafka and found that one of the best MQ I have encountered. I was exploring option to use it in Windows machine and started some kind of proof of concept work referring installation section on windows and it work perfectly. Later realized that Kafka documentation says under Hardware and OS section " We have seen a few issues running on Windows and Windows is not currently a well supported platform though we would be happy to change that. "
>
> I am curios to know is there any actual issue running Kafka in windows OS as whichever feature I am using with single instance ( Producer-Consumer) works perfectly in windows test bed. Can I use it production windows machines?
>
> Please advice.
>
> Thanks
> Harish