You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Mathieu Fenniak <ma...@replicon.com> on 2016/08/04 15:14:21 UTC
Kafka Streams on Windows?
Hey all,
Is it anyone developing Kafka Streams applications on Windows?
It seems like the RocksDB Java library doesn't include a native JNI library
for Windows, which prevents a Kafka Streams app from running on Windows. I
was just wondering if others have run into this, and if so, what approach
you took to resolve it.
I'm favouring the idea of running my applications in a Vagrant VM to avoid
the issue. It makes the Windows development environment a little less
pleasant, but, seems plausible.
Other ideas that occurred to me:
- RocksDB does support Windows, but, there don't seem to be any binaries
available or packaged for it or the jni library. I could probably build
these myself, but it sounds a little painful.
- Not developing on Windows. Works for me, but, won't work as well for
my colleagues.
- Is it possible to make Kafka Streams not use any local state storage
in some kind of development mode? Not sure...
Thanks,
Mathieu
Re: Kafka Streams on Windows?
Posted by Michael Noll <mi...@confluent.io>.
Thanks a lot for investing and also for sharing back your findings, Mathieu!
-Michael
On Fri, Aug 5, 2016 at 3:10 PM, Mathieu Fenniak <
mathieu.fenniak@replicon.com> wrote:
> I took that approach. It was painful, but, ultimately did get me a working
> Windows development environment.
>
> To any who follow in my footsteps, here is my trail:
>
> 1. Upgrade to at least Kafka Streams 0.10.0.1 (currently only in RC).
> - This is necessary because .1 bumps the rocksdb dependency to 4.8.0,
> where the previous 4.4 dependency did not yet support loading a
> Windows JNI
> library.
> 2. Follow the instructions here (
> https://github.com/facebook/rocksdb/blob/v4.8/CMakeLists.txt) to build
> rocksdb.
> - That link is for the v4.8 tag; be sure to match this with the
> version of rocksdb that Kafka Streams depends upon in the
> future. 0.10.0.1
> -> v4.8, but future releases of Kafka Streams will likely depend on
> newer
> versions.
> - Be sure to include the "-DJNI=1" compile option to build the JNI
> wrapper.
> - None of the third-party dependencies (eg. snappy, jemalloc, etc.)
> seem to be required to get something functional, but, it
> probably isn't the
> most efficient build.
> - Ensure that "javac" and "javah" are on your path when running
> cmake; if there are any errors in the cmake output your JNI wrapper
> probably won't build.
> - You may or may not need to make minor patches to make the project
> build in Windows. It appears that the Windows build is often
> broken; for
> example, I had to apply this patch:
> https://github.com/facebook/rocksdb/pull/1223/files
> - Phew, that should give you a build\java\Debug\rocksdbjni.dll. So
> close to the summit... just a horrible hack left...
> 3. Copy rocksdbjni.dll to librocksdbjni-win64.dll.
> 4. Insert librocksdbjni-win64.dll into your rocksdbjni-4.8.0.jar.
> - Ugh, this is the horrible hack. rocksdbjni seems to only look
> inside its own jar for its native libraries, so, this is where
> it needs to
> be.
> - If you're a gradle user on Windows, you'd find this jar file
> in C:\Users\...your-windows-user...\.gradle\caches\modules-2\
> files-2.1\org.rocksdb\rocksdbjni\4.8.0\b543fc4ea5b52ad790730dee376ba0
> df06d9f5f7.
>
> And there you go, almost a working Kafka Streams app in Windows. One other
> detail is that the default state storage directory doesn't seem to be
> created on demand, so I had to mkdir C:\tmp\kafka-streams myself before my
> app would work.
>
> Mathieu
>
>
> On Fri, Aug 5, 2016 at 3:13 AM, Eno Thereska <en...@gmail.com>
> wrote:
>
> > Hi Mathieu,
> >
> > It is true that the DSL currently does not support configuration of the
> > stores.
> >
> > Sounds like it might be worth trying to build RocksDb and dropping into
> > classpath for now.
> >
> > Eno
> >
> > > On 4 Aug 2016, at 17:42, Mathieu Fenniak <mathieu.fenniak@replicon.com
> >
> > wrote:
> > >
> > > Hi Eno,
> > >
> > > Yes, I've looked at that. RocksDB can be built and run in Windows,
> but,
> > > the JNI wrapper does not include Windows binarie (
> > > https://github.com/facebook/rocksdb/issues/703). rocksdbjni-4.4.1.jar
> > > includes librocksdbjni-linux32.so, librocksdbjni-linux64.so, and
> > > librocksdbjni-osx.jnilib, so only supports Linux x86 & x64 and OS X.
> It
> > is
> > > probably possible for me to build it myself and drop it in my
> classpath,
> > > but, I'm looking for a lower friction approach if one exists. :-)
> > >
> > > It looks like this was discussed recently on the Confluent Platform
> > mailing
> > > list (https://groups.google.com/forum/#!topic/confluent-
> > platform/Z1rsfSNrVJk)
> > > and the conclusion there was that high-level streams DSL doesn't
> support
> > > configuration of the stores.
> > >
> > > Mathieu
> > >
> > >
> > > On Thu, Aug 4, 2016 at 10:28 AM, Eno Thereska <en...@gmail.com>
> > > wrote:
> > >
> > >> Hi Mathieu,
> > >>
> > >> Have you had a chance to look at http://rocksdb.org/blog/2033/
> > >> rocksdb-is-now-available-in-windows-platform/? <
> > >> http://rocksdb.org/blog/2033/rocksdb-is-now-available-in-
> > windows-platform/?>
> > >> Curious to hear your and other's comments on whether that worked.
> > >>
> > >> It is possible to have Kafka Streams use an in-memory store (included
> > with
> > >> Kafka Streams) for development purposes. In that scenario RocksDb
> would
> > not
> > >> be needed.
> > >>
> > >> Eno
> > >>
> > >>
> > >>> On 4 Aug 2016, at 16:14, Mathieu Fenniak <
> mathieu.fenniak@replicon.com
> > >
> > >> wrote:
> > >>>
> > >>> Hey all,
> > >>>
> > >>> Is it anyone developing Kafka Streams applications on Windows?
> > >>>
> > >>> It seems like the RocksDB Java library doesn't include a native JNI
> > >> library
> > >>> for Windows, which prevents a Kafka Streams app from running on
> > >> Windows. I
> > >>> was just wondering if others have run into this, and if so, what
> > approach
> > >>> you took to resolve it.
> > >>>
> > >>> I'm favouring the idea of running my applications in a Vagrant VM to
> > >> avoid
> > >>> the issue. It makes the Windows development environment a little
> less
> > >>> pleasant, but, seems plausible.
> > >>>
> > >>> Other ideas that occurred to me:
> > >>>
> > >>> - RocksDB does support Windows, but, there don't seem to be any
> > >> binaries
> > >>> available or packaged for it or the jni library. I could probably
> > >> build
> > >>> these myself, but it sounds a little painful.
> > >>> - Not developing on Windows. Works for me, but, won't work as well
> > for
> > >>> my colleagues.
> > >>> - Is it possible to make Kafka Streams not use any local state
> storage
> > >>> in some kind of development mode? Not sure...
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Mathieu
> > >>
> > >>
> >
> >
>
Re: Kafka Streams on Windows?
Posted by Damian Guy <da...@gmail.com>.
Thanks for providing the instructions - appreciated.
On Fri, 5 Aug 2016 at 14:10 Mathieu Fenniak <ma...@replicon.com>
wrote:
> I took that approach. It was painful, but, ultimately did get me a working
> Windows development environment.
>
> To any who follow in my footsteps, here is my trail:
>
> 1. Upgrade to at least Kafka Streams 0.10.0.1 (currently only in RC).
> - This is necessary because .1 bumps the rocksdb dependency to 4.8.0,
> where the previous 4.4 dependency did not yet support loading a
> Windows JNI
> library.
> 2. Follow the instructions here (
> https://github.com/facebook/rocksdb/blob/v4.8/CMakeLists.txt) to build
> rocksdb.
> - That link is for the v4.8 tag; be sure to match this with the
> version of rocksdb that Kafka Streams depends upon in the
> future. 0.10.0.1
> -> v4.8, but future releases of Kafka Streams will likely depend on
> newer
> versions.
> - Be sure to include the "-DJNI=1" compile option to build the JNI
> wrapper.
> - None of the third-party dependencies (eg. snappy, jemalloc, etc.)
> seem to be required to get something functional, but, it
> probably isn't the
> most efficient build.
> - Ensure that "javac" and "javah" are on your path when running
> cmake; if there are any errors in the cmake output your JNI wrapper
> probably won't build.
> - You may or may not need to make minor patches to make the project
> build in Windows. It appears that the Windows build is often
> broken; for
> example, I had to apply this patch:
> https://github.com/facebook/rocksdb/pull/1223/files
> - Phew, that should give you a build\java\Debug\rocksdbjni.dll. So
> close to the summit... just a horrible hack left...
> 3. Copy rocksdbjni.dll to librocksdbjni-win64.dll.
> 4. Insert librocksdbjni-win64.dll into your rocksdbjni-4.8.0.jar.
> - Ugh, this is the horrible hack. rocksdbjni seems to only look
> inside its own jar for its native libraries, so, this is where
> it needs to
> be.
> - If you're a gradle user on Windows, you'd find this jar file
> in
> C:\Users\...your-windows-user...\.gradle\caches\modules-2\files-2.1\org.rocksdb\rocksdbjni\4.8.0\b543fc4ea5b52ad790730dee376ba0df06d9f5f7.
>
> And there you go, almost a working Kafka Streams app in Windows. One other
> detail is that the default state storage directory doesn't seem to be
> created on demand, so I had to mkdir C:\tmp\kafka-streams myself before my
> app would work.
>
> Mathieu
>
>
> On Fri, Aug 5, 2016 at 3:13 AM, Eno Thereska <en...@gmail.com>
> wrote:
>
> > Hi Mathieu,
> >
> > It is true that the DSL currently does not support configuration of the
> > stores.
> >
> > Sounds like it might be worth trying to build RocksDb and dropping into
> > classpath for now.
> >
> > Eno
> >
> > > On 4 Aug 2016, at 17:42, Mathieu Fenniak <mathieu.fenniak@replicon.com
> >
> > wrote:
> > >
> > > Hi Eno,
> > >
> > > Yes, I've looked at that. RocksDB can be built and run in Windows,
> but,
> > > the JNI wrapper does not include Windows binarie (
> > > https://github.com/facebook/rocksdb/issues/703). rocksdbjni-4.4.1.jar
> > > includes librocksdbjni-linux32.so, librocksdbjni-linux64.so, and
> > > librocksdbjni-osx.jnilib, so only supports Linux x86 & x64 and OS X.
> It
> > is
> > > probably possible for me to build it myself and drop it in my
> classpath,
> > > but, I'm looking for a lower friction approach if one exists. :-)
> > >
> > > It looks like this was discussed recently on the Confluent Platform
> > mailing
> > > list (https://groups.google.com/forum/#!topic/confluent-
> > platform/Z1rsfSNrVJk)
> > > and the conclusion there was that high-level streams DSL doesn't
> support
> > > configuration of the stores.
> > >
> > > Mathieu
> > >
> > >
> > > On Thu, Aug 4, 2016 at 10:28 AM, Eno Thereska <en...@gmail.com>
> > > wrote:
> > >
> > >> Hi Mathieu,
> > >>
> > >> Have you had a chance to look at http://rocksdb.org/blog/2033/
> > >> rocksdb-is-now-available-in-windows-platform/? <
> > >> http://rocksdb.org/blog/2033/rocksdb-is-now-available-in-
> > windows-platform/?>
> > >> Curious to hear your and other's comments on whether that worked.
> > >>
> > >> It is possible to have Kafka Streams use an in-memory store (included
> > with
> > >> Kafka Streams) for development purposes. In that scenario RocksDb
> would
> > not
> > >> be needed.
> > >>
> > >> Eno
> > >>
> > >>
> > >>> On 4 Aug 2016, at 16:14, Mathieu Fenniak <
> mathieu.fenniak@replicon.com
> > >
> > >> wrote:
> > >>>
> > >>> Hey all,
> > >>>
> > >>> Is it anyone developing Kafka Streams applications on Windows?
> > >>>
> > >>> It seems like the RocksDB Java library doesn't include a native JNI
> > >> library
> > >>> for Windows, which prevents a Kafka Streams app from running on
> > >> Windows. I
> > >>> was just wondering if others have run into this, and if so, what
> > approach
> > >>> you took to resolve it.
> > >>>
> > >>> I'm favouring the idea of running my applications in a Vagrant VM to
> > >> avoid
> > >>> the issue. It makes the Windows development environment a little
> less
> > >>> pleasant, but, seems plausible.
> > >>>
> > >>> Other ideas that occurred to me:
> > >>>
> > >>> - RocksDB does support Windows, but, there don't seem to be any
> > >> binaries
> > >>> available or packaged for it or the jni library. I could probably
> > >> build
> > >>> these myself, but it sounds a little painful.
> > >>> - Not developing on Windows. Works for me, but, won't work as well
> > for
> > >>> my colleagues.
> > >>> - Is it possible to make Kafka Streams not use any local state
> storage
> > >>> in some kind of development mode? Not sure...
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Mathieu
> > >>
> > >>
> >
> >
>
Re: Kafka Streams on Windows?
Posted by Mathieu Fenniak <ma...@replicon.com>.
I took that approach. It was painful, but, ultimately did get me a working
Windows development environment.
To any who follow in my footsteps, here is my trail:
1. Upgrade to at least Kafka Streams 0.10.0.1 (currently only in RC).
- This is necessary because .1 bumps the rocksdb dependency to 4.8.0,
where the previous 4.4 dependency did not yet support loading a
Windows JNI
library.
2. Follow the instructions here (
https://github.com/facebook/rocksdb/blob/v4.8/CMakeLists.txt) to build
rocksdb.
- That link is for the v4.8 tag; be sure to match this with the
version of rocksdb that Kafka Streams depends upon in the
future. 0.10.0.1
-> v4.8, but future releases of Kafka Streams will likely depend on newer
versions.
- Be sure to include the "-DJNI=1" compile option to build the JNI
wrapper.
- None of the third-party dependencies (eg. snappy, jemalloc, etc.)
seem to be required to get something functional, but, it
probably isn't the
most efficient build.
- Ensure that "javac" and "javah" are on your path when running
cmake; if there are any errors in the cmake output your JNI wrapper
probably won't build.
- You may or may not need to make minor patches to make the project
build in Windows. It appears that the Windows build is often broken; for
example, I had to apply this patch:
https://github.com/facebook/rocksdb/pull/1223/files
- Phew, that should give you a build\java\Debug\rocksdbjni.dll. So
close to the summit... just a horrible hack left...
3. Copy rocksdbjni.dll to librocksdbjni-win64.dll.
4. Insert librocksdbjni-win64.dll into your rocksdbjni-4.8.0.jar.
- Ugh, this is the horrible hack. rocksdbjni seems to only look
inside its own jar for its native libraries, so, this is where
it needs to
be.
- If you're a gradle user on Windows, you'd find this jar file
in C:\Users\...your-windows-user...\.gradle\caches\modules-2\files-2.1\org.rocksdb\rocksdbjni\4.8.0\b543fc4ea5b52ad790730dee376ba0df06d9f5f7.
And there you go, almost a working Kafka Streams app in Windows. One other
detail is that the default state storage directory doesn't seem to be
created on demand, so I had to mkdir C:\tmp\kafka-streams myself before my
app would work.
Mathieu
On Fri, Aug 5, 2016 at 3:13 AM, Eno Thereska <en...@gmail.com> wrote:
> Hi Mathieu,
>
> It is true that the DSL currently does not support configuration of the
> stores.
>
> Sounds like it might be worth trying to build RocksDb and dropping into
> classpath for now.
>
> Eno
>
> > On 4 Aug 2016, at 17:42, Mathieu Fenniak <ma...@replicon.com>
> wrote:
> >
> > Hi Eno,
> >
> > Yes, I've looked at that. RocksDB can be built and run in Windows, but,
> > the JNI wrapper does not include Windows binarie (
> > https://github.com/facebook/rocksdb/issues/703). rocksdbjni-4.4.1.jar
> > includes librocksdbjni-linux32.so, librocksdbjni-linux64.so, and
> > librocksdbjni-osx.jnilib, so only supports Linux x86 & x64 and OS X. It
> is
> > probably possible for me to build it myself and drop it in my classpath,
> > but, I'm looking for a lower friction approach if one exists. :-)
> >
> > It looks like this was discussed recently on the Confluent Platform
> mailing
> > list (https://groups.google.com/forum/#!topic/confluent-
> platform/Z1rsfSNrVJk)
> > and the conclusion there was that high-level streams DSL doesn't support
> > configuration of the stores.
> >
> > Mathieu
> >
> >
> > On Thu, Aug 4, 2016 at 10:28 AM, Eno Thereska <en...@gmail.com>
> > wrote:
> >
> >> Hi Mathieu,
> >>
> >> Have you had a chance to look at http://rocksdb.org/blog/2033/
> >> rocksdb-is-now-available-in-windows-platform/? <
> >> http://rocksdb.org/blog/2033/rocksdb-is-now-available-in-
> windows-platform/?>
> >> Curious to hear your and other's comments on whether that worked.
> >>
> >> It is possible to have Kafka Streams use an in-memory store (included
> with
> >> Kafka Streams) for development purposes. In that scenario RocksDb would
> not
> >> be needed.
> >>
> >> Eno
> >>
> >>
> >>> On 4 Aug 2016, at 16:14, Mathieu Fenniak <mathieu.fenniak@replicon.com
> >
> >> wrote:
> >>>
> >>> Hey all,
> >>>
> >>> Is it anyone developing Kafka Streams applications on Windows?
> >>>
> >>> It seems like the RocksDB Java library doesn't include a native JNI
> >> library
> >>> for Windows, which prevents a Kafka Streams app from running on
> >> Windows. I
> >>> was just wondering if others have run into this, and if so, what
> approach
> >>> you took to resolve it.
> >>>
> >>> I'm favouring the idea of running my applications in a Vagrant VM to
> >> avoid
> >>> the issue. It makes the Windows development environment a little less
> >>> pleasant, but, seems plausible.
> >>>
> >>> Other ideas that occurred to me:
> >>>
> >>> - RocksDB does support Windows, but, there don't seem to be any
> >> binaries
> >>> available or packaged for it or the jni library. I could probably
> >> build
> >>> these myself, but it sounds a little painful.
> >>> - Not developing on Windows. Works for me, but, won't work as well
> for
> >>> my colleagues.
> >>> - Is it possible to make Kafka Streams not use any local state storage
> >>> in some kind of development mode? Not sure...
> >>>
> >>> Thanks,
> >>>
> >>> Mathieu
> >>
> >>
>
>
Re: Kafka Streams on Windows?
Posted by Eno Thereska <en...@gmail.com>.
Hi Mathieu,
It is true that the DSL currently does not support configuration of the stores.
Sounds like it might be worth trying to build RocksDb and dropping into classpath for now.
Eno
> On 4 Aug 2016, at 17:42, Mathieu Fenniak <ma...@replicon.com> wrote:
>
> Hi Eno,
>
> Yes, I've looked at that. RocksDB can be built and run in Windows, but,
> the JNI wrapper does not include Windows binarie (
> https://github.com/facebook/rocksdb/issues/703). rocksdbjni-4.4.1.jar
> includes librocksdbjni-linux32.so, librocksdbjni-linux64.so, and
> librocksdbjni-osx.jnilib, so only supports Linux x86 & x64 and OS X. It is
> probably possible for me to build it myself and drop it in my classpath,
> but, I'm looking for a lower friction approach if one exists. :-)
>
> It looks like this was discussed recently on the Confluent Platform mailing
> list (https://groups.google.com/forum/#!topic/confluent-platform/Z1rsfSNrVJk)
> and the conclusion there was that high-level streams DSL doesn't support
> configuration of the stores.
>
> Mathieu
>
>
> On Thu, Aug 4, 2016 at 10:28 AM, Eno Thereska <en...@gmail.com>
> wrote:
>
>> Hi Mathieu,
>>
>> Have you had a chance to look at http://rocksdb.org/blog/2033/
>> rocksdb-is-now-available-in-windows-platform/? <
>> http://rocksdb.org/blog/2033/rocksdb-is-now-available-in-windows-platform/?>
>> Curious to hear your and other's comments on whether that worked.
>>
>> It is possible to have Kafka Streams use an in-memory store (included with
>> Kafka Streams) for development purposes. In that scenario RocksDb would not
>> be needed.
>>
>> Eno
>>
>>
>>> On 4 Aug 2016, at 16:14, Mathieu Fenniak <ma...@replicon.com>
>> wrote:
>>>
>>> Hey all,
>>>
>>> Is it anyone developing Kafka Streams applications on Windows?
>>>
>>> It seems like the RocksDB Java library doesn't include a native JNI
>> library
>>> for Windows, which prevents a Kafka Streams app from running on
>> Windows. I
>>> was just wondering if others have run into this, and if so, what approach
>>> you took to resolve it.
>>>
>>> I'm favouring the idea of running my applications in a Vagrant VM to
>> avoid
>>> the issue. It makes the Windows development environment a little less
>>> pleasant, but, seems plausible.
>>>
>>> Other ideas that occurred to me:
>>>
>>> - RocksDB does support Windows, but, there don't seem to be any
>> binaries
>>> available or packaged for it or the jni library. I could probably
>> build
>>> these myself, but it sounds a little painful.
>>> - Not developing on Windows. Works for me, but, won't work as well for
>>> my colleagues.
>>> - Is it possible to make Kafka Streams not use any local state storage
>>> in some kind of development mode? Not sure...
>>>
>>> Thanks,
>>>
>>> Mathieu
>>
>>
Re: Kafka Streams on Windows?
Posted by Mathieu Fenniak <ma...@replicon.com>.
Hi Eno,
Yes, I've looked at that. RocksDB can be built and run in Windows, but,
the JNI wrapper does not include Windows binarie (
https://github.com/facebook/rocksdb/issues/703). rocksdbjni-4.4.1.jar
includes librocksdbjni-linux32.so, librocksdbjni-linux64.so, and
librocksdbjni-osx.jnilib, so only supports Linux x86 & x64 and OS X. It is
probably possible for me to build it myself and drop it in my classpath,
but, I'm looking for a lower friction approach if one exists. :-)
It looks like this was discussed recently on the Confluent Platform mailing
list (https://groups.google.com/forum/#!topic/confluent-platform/Z1rsfSNrVJk)
and the conclusion there was that high-level streams DSL doesn't support
configuration of the stores.
Mathieu
On Thu, Aug 4, 2016 at 10:28 AM, Eno Thereska <en...@gmail.com>
wrote:
> Hi Mathieu,
>
> Have you had a chance to look at http://rocksdb.org/blog/2033/
> rocksdb-is-now-available-in-windows-platform/? <
> http://rocksdb.org/blog/2033/rocksdb-is-now-available-in-windows-platform/?>
> Curious to hear your and other's comments on whether that worked.
>
> It is possible to have Kafka Streams use an in-memory store (included with
> Kafka Streams) for development purposes. In that scenario RocksDb would not
> be needed.
>
> Eno
>
>
> > On 4 Aug 2016, at 16:14, Mathieu Fenniak <ma...@replicon.com>
> wrote:
> >
> > Hey all,
> >
> > Is it anyone developing Kafka Streams applications on Windows?
> >
> > It seems like the RocksDB Java library doesn't include a native JNI
> library
> > for Windows, which prevents a Kafka Streams app from running on
> Windows. I
> > was just wondering if others have run into this, and if so, what approach
> > you took to resolve it.
> >
> > I'm favouring the idea of running my applications in a Vagrant VM to
> avoid
> > the issue. It makes the Windows development environment a little less
> > pleasant, but, seems plausible.
> >
> > Other ideas that occurred to me:
> >
> > - RocksDB does support Windows, but, there don't seem to be any
> binaries
> > available or packaged for it or the jni library. I could probably
> build
> > these myself, but it sounds a little painful.
> > - Not developing on Windows. Works for me, but, won't work as well for
> > my colleagues.
> > - Is it possible to make Kafka Streams not use any local state storage
> > in some kind of development mode? Not sure...
> >
> > Thanks,
> >
> > Mathieu
>
>
Re: Kafka Streams on Windows?
Posted by Eno Thereska <en...@gmail.com>.
Hi Mathieu,
Have you had a chance to look at http://rocksdb.org/blog/2033/rocksdb-is-now-available-in-windows-platform/? <http://rocksdb.org/blog/2033/rocksdb-is-now-available-in-windows-platform/?> Curious to hear your and other's comments on whether that worked.
It is possible to have Kafka Streams use an in-memory store (included with Kafka Streams) for development purposes. In that scenario RocksDb would not be needed.
Eno
> On 4 Aug 2016, at 16:14, Mathieu Fenniak <ma...@replicon.com> wrote:
>
> Hey all,
>
> Is it anyone developing Kafka Streams applications on Windows?
>
> It seems like the RocksDB Java library doesn't include a native JNI library
> for Windows, which prevents a Kafka Streams app from running on Windows. I
> was just wondering if others have run into this, and if so, what approach
> you took to resolve it.
>
> I'm favouring the idea of running my applications in a Vagrant VM to avoid
> the issue. It makes the Windows development environment a little less
> pleasant, but, seems plausible.
>
> Other ideas that occurred to me:
>
> - RocksDB does support Windows, but, there don't seem to be any binaries
> available or packaged for it or the jni library. I could probably build
> these myself, but it sounds a little painful.
> - Not developing on Windows. Works for me, but, won't work as well for
> my colleagues.
> - Is it possible to make Kafka Streams not use any local state storage
> in some kind of development mode? Not sure...
>
> Thanks,
>
> Mathieu