You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Vahid Hashemian <va...@gmail.com> on 2019/05/08 20:25:52 UTC
[VOTE] 2.2.1 RC0
Hello Kafka users, developers and client-developers,
This is the first candidate for release of Apache Kafka 2.2.1, which
includes many bug fixes for Apache Kafka 2.2.
Release notes for the 2.2.1 release:
https://home.apache.org/~vahid/kafka-2.2.1-rc0/RELEASE_NOTES.html
*** Please download, test and vote by Monday, May 13, 6:00 pm PT.
Kafka's KEYS file containing PGP keys we use to sign the release:
https://kafka.apache.org/KEYS
* Release artifacts to be voted upon (source and binary):
https://home.apache.org/~vahid/kafka-2.2.1-rc0/
* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/org/apache/kafka/
* Javadoc:
https://home.apache.org/~vahid/kafka-2.2.1-rc0/javadoc/
* Tag to be voted upon (off 2.2 branch) is the 2.2.1 tag:
https://github.com/apache/kafka/releases/tag/2.2.1-rc0
* Documentation:
https://kafka.apache.org/22/documentation.html
* Protocol:
https://kafka.apache.org/22/protocol.html
* Successful Jenkins builds for the 2.2 branch:
Unit/integration tests: https://builds.apache.org/job/kafka-2.2-jdk8/106/
Thanks,
--Vahid
Re: [VOTE] 2.2.1 RC0
Posted by Vahid Hashemian <va...@gmail.com>.
Hi Jason,
Thanks for quick resolution on this ticket.
I'll work on generating RC1.
Thanks!
--Vahid
On Mon, May 13, 2019 at 9:02 AM Jason Gustafson <ja...@confluent.io> wrote:
> Hi Vahid,
>
> I merged the patch for KAFKA-8335 into 2.2.
>
> Thanks,
> Jason
>
> On Mon, May 13, 2019 at 7:17 AM Vahid Hashemian <vahid.hashemian@gmail.com
> >
> wrote:
>
> > Thanks Patrik for the reference.
> >
> > That JIRA seems to be covering the exact same issue with multiple
> releases,
> > and is not marked as a blocker at this point.
> >
> > --Vahid
> >
> > On Mon, May 13, 2019 at 1:22 AM Patrik Kleindl <pk...@gmail.com>
> wrote:
> >
> > > Hi
> > > This might be related to
> > https://issues.apache.org/jira/browse/KAFKA-5998
> > > at least the behaviour is described there.
> > > Regards
> > > Patrik
> > >
> > > > Am 13.05.2019 um 00:23 schrieb Vahid Hashemian <
> > > vahid.hashemian@gmail.com>:
> > > >
> > > > Hi Jonathan,
> > > >
> > > > Thanks for reporting the issue.
> > > > Do you know if it is something that's introduced since 2.2? Did you
> > run a
> > > > similar test with 2.2?
> > > >
> > > > Thanks,
> > > > --Vahid
> > > >
> > > > On Sun, May 12, 2019 at 2:22 PM Jonathan Santilli <
> > > > jonathansantilli@gmail.com> wrote:
> > > >
> > > >> Hello Vahid,
> > > >>
> > > >>
> > > >> am testing one of our Kafka Stream Apps with the 2.2.1-rc, after few
> > > >> minutes, I see this WARN:
> > > >>
> > > >>
> > > >> 2019-05-09 13:14:37,025 WARN [test-app-id-dc27624a-8e02-
> > > >> 4031-963b-7596a8a77097-StreamThread-1]
> > internals.ProcessorStateManager (
> > > >> ProcessorStateManager.java:349) - task [0_0] Failed to write offset
> > > >> checkpoint file to
> [/tmp/kafka-stream-app/test-app-id/0_0/.checkpoint]
> > > >>
> > > >> java.io.FileNotFoundException:
> > > >> /tmp/kafka-stream-app/test-app-id/0_0/.checkpoint.tmp (No such file
> or
> > > >> directory)
> > > >>
> > > >> at java.io.FileOutputStream.open0(Native Method) ~[?:1.8.0_191]
> > > >>
> > > >> at java.io.FileOutputStream.open(FileOutputStream.java:270)
> > > ~[?:1.8.0_191]
> > > >>
> > > >> at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
> > > >> ~[?:1.8.0_191]
> > > >>
> > > >> at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
> > > >> ~[?:1.8.0_191]
> > > >>
> > > >> at org.apache.kafka.streams.state.internals.OffsetCheckpoint.write(
> > > >> OffsetCheckpoint.java:79) ~[kafka-streams-2.2.1.jar:?]
> > > >>
> > > >> at
> > > >>
> > > >>
> > >
> >
> org.apache.kafka.streams.processor.internals.ProcessorStateManager.checkpoint(
> > > >> ProcessorStateManager.java:347) [kafka-streams-2.2.1.jar:?]
> > > >>
> > > >> at org.apache.kafka.streams.processor.internals.StreamTask.commit(
> > > >> StreamTask.java:476) [kafka-streams-2.2.1.jar:?]
> > > >>
> > > >> at org.apache.kafka.streams.processor.internals.StreamTask.suspend(
> > > >> StreamTask.java:598) [kafka-streams-2.2.1.jar:?]
> > > >>
> > > >> at org.apache.kafka.streams.processor.internals.StreamTask.close(
> > > >> StreamTask.java:724) [kafka-streams-2.2.1.jar:?]
> > > >>
> > > >> at org.apache.kafka.streams.processor.internals.AssignedTasks.close(
> > > >> AssignedTasks.java:337) [kafka-streams-2.2.1.jar:?]
> > > >>
> > > >> at
> org.apache.kafka.streams.processor.internals.TaskManager.shutdown(
> > > >> TaskManager.java:267) [kafka-streams-2.2.1.jar:?]
> > > >>
> > > >> at
> > > >>
> > >
> >
> org.apache.kafka.streams.processor.internals.StreamThread.completeShutdown(
> > > >> StreamThread.java:1208) [kafka-streams-2.2.1.jar:?]
> > > >>
> > > >> at org.apache.kafka.streams.processor.internals.StreamThread.run(
> > > >> StreamThread.java:785) [kafka-streams-2.2.1.jar:?]
> > > >>
> > > >> Checking the system, in fact, the folder does not exist, but, others
> > > were
> > > >> created:
> > > >>
> > > >> # ls /tmp/kafka-stream-app/test-app-id/
> > > >> # 1_0 1_1 1_2
> > > >>
> > > >> After restarting the App, the same WARN shows-up but in this case,
> the
> > > >> folders were created but not the .checkpoint.tmp file:
> > > >>
> > > >> # ls /tmp/kafka-stream-app/test-app-id/
> > > >> # 0_0 0_1 0_2 1_0 1_1 1_2
> > > >>
> > > >> Am just reporting this because I found it strange/suspicious.
> > > >>
> > > >>
> > > >> Cheers!
> > > >> --
> > > >> Jonathan
> > > >>
> > > >>
> > > >>
> > > >> On Wed, May 8, 2019 at 9:26 PM Vahid Hashemian <
> > > vahid.hashemian@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> Hello Kafka users, developers and client-developers,
> > > >>>
> > > >>> This is the first candidate for release of Apache Kafka 2.2.1,
> which
> > > >>> includes many bug fixes for Apache Kafka 2.2.
> > > >>>
> > > >>> Release notes for the 2.2.1 release:
> > > >>> https://home.apache.org/~vahid/kafka-2.2.1-rc0/RELEASE_NOTES.html
> > > >>>
> > > >>> *** Please download, test and vote by Monday, May 13, 6:00 pm PT.
> > > >>>
> > > >>> Kafka's KEYS file containing PGP keys we use to sign the release:
> > > >>> https://kafka.apache.org/KEYS
> > > >>>
> > > >>> * Release artifacts to be voted upon (source and binary):
> > > >>> https://home.apache.org/~vahid/kafka-2.2.1-rc0/
> > > >>>
> > > >>> * Maven artifacts to be voted upon:
> > > >>>
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > >>>
> > > >>> * Javadoc:
> > > >>> https://home.apache.org/~vahid/kafka-2.2.1-rc0/javadoc/
> > > >>>
> > > >>> * Tag to be voted upon (off 2.2 branch) is the 2.2.1 tag:
> > > >>> https://github.com/apache/kafka/releases/tag/2.2.1-rc0
> > > >>>
> > > >>> * Documentation:
> > > >>> https://kafka.apache.org/22/documentation.html
> > > >>>
> > > >>> * Protocol:
> > > >>> https://kafka.apache.org/22/protocol.html
> > > >>>
> > > >>> * Successful Jenkins builds for the 2.2 branch:
> > > >>> Unit/integration tests:
> > > >> https://builds.apache.org/job/kafka-2.2-jdk8/106/
> > > >>>
> > > >>> Thanks,
> > > >>> --Vahid
> > > >>>
> > > >>
> > > >>
> > > >> --
> > > >> Santilli Jonathan
> > > >>
> > > >
> > > >
> > > > --
> > > >
> > > > Thanks!
> > > > --Vahid
> > >
> >
> >
> > --
> >
> > Thanks!
> > --Vahid
> >
>
--
Thanks!
--Vahid
Re: [VOTE] 2.2.1 RC0
Posted by Jason Gustafson <ja...@confluent.io>.
Hi Vahid,
I merged the patch for KAFKA-8335 into 2.2.
Thanks,
Jason
On Mon, May 13, 2019 at 7:17 AM Vahid Hashemian <va...@gmail.com>
wrote:
> Thanks Patrik for the reference.
>
> That JIRA seems to be covering the exact same issue with multiple releases,
> and is not marked as a blocker at this point.
>
> --Vahid
>
> On Mon, May 13, 2019 at 1:22 AM Patrik Kleindl <pk...@gmail.com> wrote:
>
> > Hi
> > This might be related to
> https://issues.apache.org/jira/browse/KAFKA-5998
> > at least the behaviour is described there.
> > Regards
> > Patrik
> >
> > > Am 13.05.2019 um 00:23 schrieb Vahid Hashemian <
> > vahid.hashemian@gmail.com>:
> > >
> > > Hi Jonathan,
> > >
> > > Thanks for reporting the issue.
> > > Do you know if it is something that's introduced since 2.2? Did you
> run a
> > > similar test with 2.2?
> > >
> > > Thanks,
> > > --Vahid
> > >
> > > On Sun, May 12, 2019 at 2:22 PM Jonathan Santilli <
> > > jonathansantilli@gmail.com> wrote:
> > >
> > >> Hello Vahid,
> > >>
> > >>
> > >> am testing one of our Kafka Stream Apps with the 2.2.1-rc, after few
> > >> minutes, I see this WARN:
> > >>
> > >>
> > >> 2019-05-09 13:14:37,025 WARN [test-app-id-dc27624a-8e02-
> > >> 4031-963b-7596a8a77097-StreamThread-1]
> internals.ProcessorStateManager (
> > >> ProcessorStateManager.java:349) - task [0_0] Failed to write offset
> > >> checkpoint file to [/tmp/kafka-stream-app/test-app-id/0_0/.checkpoint]
> > >>
> > >> java.io.FileNotFoundException:
> > >> /tmp/kafka-stream-app/test-app-id/0_0/.checkpoint.tmp (No such file or
> > >> directory)
> > >>
> > >> at java.io.FileOutputStream.open0(Native Method) ~[?:1.8.0_191]
> > >>
> > >> at java.io.FileOutputStream.open(FileOutputStream.java:270)
> > ~[?:1.8.0_191]
> > >>
> > >> at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
> > >> ~[?:1.8.0_191]
> > >>
> > >> at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
> > >> ~[?:1.8.0_191]
> > >>
> > >> at org.apache.kafka.streams.state.internals.OffsetCheckpoint.write(
> > >> OffsetCheckpoint.java:79) ~[kafka-streams-2.2.1.jar:?]
> > >>
> > >> at
> > >>
> > >>
> >
> org.apache.kafka.streams.processor.internals.ProcessorStateManager.checkpoint(
> > >> ProcessorStateManager.java:347) [kafka-streams-2.2.1.jar:?]
> > >>
> > >> at org.apache.kafka.streams.processor.internals.StreamTask.commit(
> > >> StreamTask.java:476) [kafka-streams-2.2.1.jar:?]
> > >>
> > >> at org.apache.kafka.streams.processor.internals.StreamTask.suspend(
> > >> StreamTask.java:598) [kafka-streams-2.2.1.jar:?]
> > >>
> > >> at org.apache.kafka.streams.processor.internals.StreamTask.close(
> > >> StreamTask.java:724) [kafka-streams-2.2.1.jar:?]
> > >>
> > >> at org.apache.kafka.streams.processor.internals.AssignedTasks.close(
> > >> AssignedTasks.java:337) [kafka-streams-2.2.1.jar:?]
> > >>
> > >> at org.apache.kafka.streams.processor.internals.TaskManager.shutdown(
> > >> TaskManager.java:267) [kafka-streams-2.2.1.jar:?]
> > >>
> > >> at
> > >>
> >
> org.apache.kafka.streams.processor.internals.StreamThread.completeShutdown(
> > >> StreamThread.java:1208) [kafka-streams-2.2.1.jar:?]
> > >>
> > >> at org.apache.kafka.streams.processor.internals.StreamThread.run(
> > >> StreamThread.java:785) [kafka-streams-2.2.1.jar:?]
> > >>
> > >> Checking the system, in fact, the folder does not exist, but, others
> > were
> > >> created:
> > >>
> > >> # ls /tmp/kafka-stream-app/test-app-id/
> > >> # 1_0 1_1 1_2
> > >>
> > >> After restarting the App, the same WARN shows-up but in this case, the
> > >> folders were created but not the .checkpoint.tmp file:
> > >>
> > >> # ls /tmp/kafka-stream-app/test-app-id/
> > >> # 0_0 0_1 0_2 1_0 1_1 1_2
> > >>
> > >> Am just reporting this because I found it strange/suspicious.
> > >>
> > >>
> > >> Cheers!
> > >> --
> > >> Jonathan
> > >>
> > >>
> > >>
> > >> On Wed, May 8, 2019 at 9:26 PM Vahid Hashemian <
> > vahid.hashemian@gmail.com>
> > >> wrote:
> > >>
> > >>> Hello Kafka users, developers and client-developers,
> > >>>
> > >>> This is the first candidate for release of Apache Kafka 2.2.1, which
> > >>> includes many bug fixes for Apache Kafka 2.2.
> > >>>
> > >>> Release notes for the 2.2.1 release:
> > >>> https://home.apache.org/~vahid/kafka-2.2.1-rc0/RELEASE_NOTES.html
> > >>>
> > >>> *** Please download, test and vote by Monday, May 13, 6:00 pm PT.
> > >>>
> > >>> Kafka's KEYS file containing PGP keys we use to sign the release:
> > >>> https://kafka.apache.org/KEYS
> > >>>
> > >>> * Release artifacts to be voted upon (source and binary):
> > >>> https://home.apache.org/~vahid/kafka-2.2.1-rc0/
> > >>>
> > >>> * Maven artifacts to be voted upon:
> > >>>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > >>>
> > >>> * Javadoc:
> > >>> https://home.apache.org/~vahid/kafka-2.2.1-rc0/javadoc/
> > >>>
> > >>> * Tag to be voted upon (off 2.2 branch) is the 2.2.1 tag:
> > >>> https://github.com/apache/kafka/releases/tag/2.2.1-rc0
> > >>>
> > >>> * Documentation:
> > >>> https://kafka.apache.org/22/documentation.html
> > >>>
> > >>> * Protocol:
> > >>> https://kafka.apache.org/22/protocol.html
> > >>>
> > >>> * Successful Jenkins builds for the 2.2 branch:
> > >>> Unit/integration tests:
> > >> https://builds.apache.org/job/kafka-2.2-jdk8/106/
> > >>>
> > >>> Thanks,
> > >>> --Vahid
> > >>>
> > >>
> > >>
> > >> --
> > >> Santilli Jonathan
> > >>
> > >
> > >
> > > --
> > >
> > > Thanks!
> > > --Vahid
> >
>
>
> --
>
> Thanks!
> --Vahid
>
Re: [VOTE] 2.2.1 RC0
Posted by Vahid Hashemian <va...@gmail.com>.
Thanks Patrik for the reference.
That JIRA seems to be covering the exact same issue with multiple releases,
and is not marked as a blocker at this point.
--Vahid
On Mon, May 13, 2019 at 1:22 AM Patrik Kleindl <pk...@gmail.com> wrote:
> Hi
> This might be related to https://issues.apache.org/jira/browse/KAFKA-5998
> at least the behaviour is described there.
> Regards
> Patrik
>
> > Am 13.05.2019 um 00:23 schrieb Vahid Hashemian <
> vahid.hashemian@gmail.com>:
> >
> > Hi Jonathan,
> >
> > Thanks for reporting the issue.
> > Do you know if it is something that's introduced since 2.2? Did you run a
> > similar test with 2.2?
> >
> > Thanks,
> > --Vahid
> >
> > On Sun, May 12, 2019 at 2:22 PM Jonathan Santilli <
> > jonathansantilli@gmail.com> wrote:
> >
> >> Hello Vahid,
> >>
> >>
> >> am testing one of our Kafka Stream Apps with the 2.2.1-rc, after few
> >> minutes, I see this WARN:
> >>
> >>
> >> 2019-05-09 13:14:37,025 WARN [test-app-id-dc27624a-8e02-
> >> 4031-963b-7596a8a77097-StreamThread-1] internals.ProcessorStateManager (
> >> ProcessorStateManager.java:349) - task [0_0] Failed to write offset
> >> checkpoint file to [/tmp/kafka-stream-app/test-app-id/0_0/.checkpoint]
> >>
> >> java.io.FileNotFoundException:
> >> /tmp/kafka-stream-app/test-app-id/0_0/.checkpoint.tmp (No such file or
> >> directory)
> >>
> >> at java.io.FileOutputStream.open0(Native Method) ~[?:1.8.0_191]
> >>
> >> at java.io.FileOutputStream.open(FileOutputStream.java:270)
> ~[?:1.8.0_191]
> >>
> >> at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
> >> ~[?:1.8.0_191]
> >>
> >> at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
> >> ~[?:1.8.0_191]
> >>
> >> at org.apache.kafka.streams.state.internals.OffsetCheckpoint.write(
> >> OffsetCheckpoint.java:79) ~[kafka-streams-2.2.1.jar:?]
> >>
> >> at
> >>
> >>
> org.apache.kafka.streams.processor.internals.ProcessorStateManager.checkpoint(
> >> ProcessorStateManager.java:347) [kafka-streams-2.2.1.jar:?]
> >>
> >> at org.apache.kafka.streams.processor.internals.StreamTask.commit(
> >> StreamTask.java:476) [kafka-streams-2.2.1.jar:?]
> >>
> >> at org.apache.kafka.streams.processor.internals.StreamTask.suspend(
> >> StreamTask.java:598) [kafka-streams-2.2.1.jar:?]
> >>
> >> at org.apache.kafka.streams.processor.internals.StreamTask.close(
> >> StreamTask.java:724) [kafka-streams-2.2.1.jar:?]
> >>
> >> at org.apache.kafka.streams.processor.internals.AssignedTasks.close(
> >> AssignedTasks.java:337) [kafka-streams-2.2.1.jar:?]
> >>
> >> at org.apache.kafka.streams.processor.internals.TaskManager.shutdown(
> >> TaskManager.java:267) [kafka-streams-2.2.1.jar:?]
> >>
> >> at
> >>
> org.apache.kafka.streams.processor.internals.StreamThread.completeShutdown(
> >> StreamThread.java:1208) [kafka-streams-2.2.1.jar:?]
> >>
> >> at org.apache.kafka.streams.processor.internals.StreamThread.run(
> >> StreamThread.java:785) [kafka-streams-2.2.1.jar:?]
> >>
> >> Checking the system, in fact, the folder does not exist, but, others
> were
> >> created:
> >>
> >> # ls /tmp/kafka-stream-app/test-app-id/
> >> # 1_0 1_1 1_2
> >>
> >> After restarting the App, the same WARN shows-up but in this case, the
> >> folders were created but not the .checkpoint.tmp file:
> >>
> >> # ls /tmp/kafka-stream-app/test-app-id/
> >> # 0_0 0_1 0_2 1_0 1_1 1_2
> >>
> >> Am just reporting this because I found it strange/suspicious.
> >>
> >>
> >> Cheers!
> >> --
> >> Jonathan
> >>
> >>
> >>
> >> On Wed, May 8, 2019 at 9:26 PM Vahid Hashemian <
> vahid.hashemian@gmail.com>
> >> wrote:
> >>
> >>> Hello Kafka users, developers and client-developers,
> >>>
> >>> This is the first candidate for release of Apache Kafka 2.2.1, which
> >>> includes many bug fixes for Apache Kafka 2.2.
> >>>
> >>> Release notes for the 2.2.1 release:
> >>> https://home.apache.org/~vahid/kafka-2.2.1-rc0/RELEASE_NOTES.html
> >>>
> >>> *** Please download, test and vote by Monday, May 13, 6:00 pm PT.
> >>>
> >>> Kafka's KEYS file containing PGP keys we use to sign the release:
> >>> https://kafka.apache.org/KEYS
> >>>
> >>> * Release artifacts to be voted upon (source and binary):
> >>> https://home.apache.org/~vahid/kafka-2.2.1-rc0/
> >>>
> >>> * Maven artifacts to be voted upon:
> >>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >>>
> >>> * Javadoc:
> >>> https://home.apache.org/~vahid/kafka-2.2.1-rc0/javadoc/
> >>>
> >>> * Tag to be voted upon (off 2.2 branch) is the 2.2.1 tag:
> >>> https://github.com/apache/kafka/releases/tag/2.2.1-rc0
> >>>
> >>> * Documentation:
> >>> https://kafka.apache.org/22/documentation.html
> >>>
> >>> * Protocol:
> >>> https://kafka.apache.org/22/protocol.html
> >>>
> >>> * Successful Jenkins builds for the 2.2 branch:
> >>> Unit/integration tests:
> >> https://builds.apache.org/job/kafka-2.2-jdk8/106/
> >>>
> >>> Thanks,
> >>> --Vahid
> >>>
> >>
> >>
> >> --
> >> Santilli Jonathan
> >>
> >
> >
> > --
> >
> > Thanks!
> > --Vahid
>
--
Thanks!
--Vahid
Re: [VOTE] 2.2.1 RC0
Posted by Patrik Kleindl <pk...@gmail.com>.
Hi
This might be related to https://issues.apache.org/jira/browse/KAFKA-5998
at least the behaviour is described there.
Regards
Patrik
> Am 13.05.2019 um 00:23 schrieb Vahid Hashemian <va...@gmail.com>:
>
> Hi Jonathan,
>
> Thanks for reporting the issue.
> Do you know if it is something that's introduced since 2.2? Did you run a
> similar test with 2.2?
>
> Thanks,
> --Vahid
>
> On Sun, May 12, 2019 at 2:22 PM Jonathan Santilli <
> jonathansantilli@gmail.com> wrote:
>
>> Hello Vahid,
>>
>>
>> am testing one of our Kafka Stream Apps with the 2.2.1-rc, after few
>> minutes, I see this WARN:
>>
>>
>> 2019-05-09 13:14:37,025 WARN [test-app-id-dc27624a-8e02-
>> 4031-963b-7596a8a77097-StreamThread-1] internals.ProcessorStateManager (
>> ProcessorStateManager.java:349) - task [0_0] Failed to write offset
>> checkpoint file to [/tmp/kafka-stream-app/test-app-id/0_0/.checkpoint]
>>
>> java.io.FileNotFoundException:
>> /tmp/kafka-stream-app/test-app-id/0_0/.checkpoint.tmp (No such file or
>> directory)
>>
>> at java.io.FileOutputStream.open0(Native Method) ~[?:1.8.0_191]
>>
>> at java.io.FileOutputStream.open(FileOutputStream.java:270) ~[?:1.8.0_191]
>>
>> at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
>> ~[?:1.8.0_191]
>>
>> at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
>> ~[?:1.8.0_191]
>>
>> at org.apache.kafka.streams.state.internals.OffsetCheckpoint.write(
>> OffsetCheckpoint.java:79) ~[kafka-streams-2.2.1.jar:?]
>>
>> at
>>
>> org.apache.kafka.streams.processor.internals.ProcessorStateManager.checkpoint(
>> ProcessorStateManager.java:347) [kafka-streams-2.2.1.jar:?]
>>
>> at org.apache.kafka.streams.processor.internals.StreamTask.commit(
>> StreamTask.java:476) [kafka-streams-2.2.1.jar:?]
>>
>> at org.apache.kafka.streams.processor.internals.StreamTask.suspend(
>> StreamTask.java:598) [kafka-streams-2.2.1.jar:?]
>>
>> at org.apache.kafka.streams.processor.internals.StreamTask.close(
>> StreamTask.java:724) [kafka-streams-2.2.1.jar:?]
>>
>> at org.apache.kafka.streams.processor.internals.AssignedTasks.close(
>> AssignedTasks.java:337) [kafka-streams-2.2.1.jar:?]
>>
>> at org.apache.kafka.streams.processor.internals.TaskManager.shutdown(
>> TaskManager.java:267) [kafka-streams-2.2.1.jar:?]
>>
>> at
>> org.apache.kafka.streams.processor.internals.StreamThread.completeShutdown(
>> StreamThread.java:1208) [kafka-streams-2.2.1.jar:?]
>>
>> at org.apache.kafka.streams.processor.internals.StreamThread.run(
>> StreamThread.java:785) [kafka-streams-2.2.1.jar:?]
>>
>> Checking the system, in fact, the folder does not exist, but, others were
>> created:
>>
>> # ls /tmp/kafka-stream-app/test-app-id/
>> # 1_0 1_1 1_2
>>
>> After restarting the App, the same WARN shows-up but in this case, the
>> folders were created but not the .checkpoint.tmp file:
>>
>> # ls /tmp/kafka-stream-app/test-app-id/
>> # 0_0 0_1 0_2 1_0 1_1 1_2
>>
>> Am just reporting this because I found it strange/suspicious.
>>
>>
>> Cheers!
>> --
>> Jonathan
>>
>>
>>
>> On Wed, May 8, 2019 at 9:26 PM Vahid Hashemian <va...@gmail.com>
>> wrote:
>>
>>> Hello Kafka users, developers and client-developers,
>>>
>>> This is the first candidate for release of Apache Kafka 2.2.1, which
>>> includes many bug fixes for Apache Kafka 2.2.
>>>
>>> Release notes for the 2.2.1 release:
>>> https://home.apache.org/~vahid/kafka-2.2.1-rc0/RELEASE_NOTES.html
>>>
>>> *** Please download, test and vote by Monday, May 13, 6:00 pm PT.
>>>
>>> Kafka's KEYS file containing PGP keys we use to sign the release:
>>> https://kafka.apache.org/KEYS
>>>
>>> * Release artifacts to be voted upon (source and binary):
>>> https://home.apache.org/~vahid/kafka-2.2.1-rc0/
>>>
>>> * Maven artifacts to be voted upon:
>>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>>
>>> * Javadoc:
>>> https://home.apache.org/~vahid/kafka-2.2.1-rc0/javadoc/
>>>
>>> * Tag to be voted upon (off 2.2 branch) is the 2.2.1 tag:
>>> https://github.com/apache/kafka/releases/tag/2.2.1-rc0
>>>
>>> * Documentation:
>>> https://kafka.apache.org/22/documentation.html
>>>
>>> * Protocol:
>>> https://kafka.apache.org/22/protocol.html
>>>
>>> * Successful Jenkins builds for the 2.2 branch:
>>> Unit/integration tests:
>> https://builds.apache.org/job/kafka-2.2-jdk8/106/
>>>
>>> Thanks,
>>> --Vahid
>>>
>>
>>
>> --
>> Santilli Jonathan
>>
>
>
> --
>
> Thanks!
> --Vahid
Re: [VOTE] 2.2.1 RC0
Posted by Vahid Hashemian <va...@gmail.com>.
Hi Jonathan,
Thanks for reporting the issue.
Do you know if it is something that's introduced since 2.2? Did you run a
similar test with 2.2?
Thanks,
--Vahid
On Sun, May 12, 2019 at 2:22 PM Jonathan Santilli <
jonathansantilli@gmail.com> wrote:
> Hello Vahid,
>
>
> am testing one of our Kafka Stream Apps with the 2.2.1-rc, after few
> minutes, I see this WARN:
>
>
> 2019-05-09 13:14:37,025 WARN [test-app-id-dc27624a-8e02-
> 4031-963b-7596a8a77097-StreamThread-1] internals.ProcessorStateManager (
> ProcessorStateManager.java:349) - task [0_0] Failed to write offset
> checkpoint file to [/tmp/kafka-stream-app/test-app-id/0_0/.checkpoint]
>
> java.io.FileNotFoundException:
> /tmp/kafka-stream-app/test-app-id/0_0/.checkpoint.tmp (No such file or
> directory)
>
> at java.io.FileOutputStream.open0(Native Method) ~[?:1.8.0_191]
>
> at java.io.FileOutputStream.open(FileOutputStream.java:270) ~[?:1.8.0_191]
>
> at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
> ~[?:1.8.0_191]
>
> at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
> ~[?:1.8.0_191]
>
> at org.apache.kafka.streams.state.internals.OffsetCheckpoint.write(
> OffsetCheckpoint.java:79) ~[kafka-streams-2.2.1.jar:?]
>
> at
>
> org.apache.kafka.streams.processor.internals.ProcessorStateManager.checkpoint(
> ProcessorStateManager.java:347) [kafka-streams-2.2.1.jar:?]
>
> at org.apache.kafka.streams.processor.internals.StreamTask.commit(
> StreamTask.java:476) [kafka-streams-2.2.1.jar:?]
>
> at org.apache.kafka.streams.processor.internals.StreamTask.suspend(
> StreamTask.java:598) [kafka-streams-2.2.1.jar:?]
>
> at org.apache.kafka.streams.processor.internals.StreamTask.close(
> StreamTask.java:724) [kafka-streams-2.2.1.jar:?]
>
> at org.apache.kafka.streams.processor.internals.AssignedTasks.close(
> AssignedTasks.java:337) [kafka-streams-2.2.1.jar:?]
>
> at org.apache.kafka.streams.processor.internals.TaskManager.shutdown(
> TaskManager.java:267) [kafka-streams-2.2.1.jar:?]
>
> at
> org.apache.kafka.streams.processor.internals.StreamThread.completeShutdown(
> StreamThread.java:1208) [kafka-streams-2.2.1.jar:?]
>
> at org.apache.kafka.streams.processor.internals.StreamThread.run(
> StreamThread.java:785) [kafka-streams-2.2.1.jar:?]
>
> Checking the system, in fact, the folder does not exist, but, others were
> created:
>
> # ls /tmp/kafka-stream-app/test-app-id/
> # 1_0 1_1 1_2
>
> After restarting the App, the same WARN shows-up but in this case, the
> folders were created but not the .checkpoint.tmp file:
>
> # ls /tmp/kafka-stream-app/test-app-id/
> # 0_0 0_1 0_2 1_0 1_1 1_2
>
> Am just reporting this because I found it strange/suspicious.
>
>
> Cheers!
> --
> Jonathan
>
>
>
> On Wed, May 8, 2019 at 9:26 PM Vahid Hashemian <va...@gmail.com>
> wrote:
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the first candidate for release of Apache Kafka 2.2.1, which
> > includes many bug fixes for Apache Kafka 2.2.
> >
> > Release notes for the 2.2.1 release:
> > https://home.apache.org/~vahid/kafka-2.2.1-rc0/RELEASE_NOTES.html
> >
> > *** Please download, test and vote by Monday, May 13, 6:00 pm PT.
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > https://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > https://home.apache.org/~vahid/kafka-2.2.1-rc0/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >
> > * Javadoc:
> > https://home.apache.org/~vahid/kafka-2.2.1-rc0/javadoc/
> >
> > * Tag to be voted upon (off 2.2 branch) is the 2.2.1 tag:
> > https://github.com/apache/kafka/releases/tag/2.2.1-rc0
> >
> > * Documentation:
> > https://kafka.apache.org/22/documentation.html
> >
> > * Protocol:
> > https://kafka.apache.org/22/protocol.html
> >
> > * Successful Jenkins builds for the 2.2 branch:
> > Unit/integration tests:
> https://builds.apache.org/job/kafka-2.2-jdk8/106/
> >
> > Thanks,
> > --Vahid
> >
>
>
> --
> Santilli Jonathan
>
--
Thanks!
--Vahid
Re: [VOTE] 2.2.1 RC0
Posted by Jonathan Santilli <jo...@gmail.com>.
Hello Vahid,
am testing one of our Kafka Stream Apps with the 2.2.1-rc, after few
minutes, I see this WARN:
2019-05-09 13:14:37,025 WARN [test-app-id-dc27624a-8e02-
4031-963b-7596a8a77097-StreamThread-1] internals.ProcessorStateManager (
ProcessorStateManager.java:349) - task [0_0] Failed to write offset
checkpoint file to [/tmp/kafka-stream-app/test-app-id/0_0/.checkpoint]
java.io.FileNotFoundException:
/tmp/kafka-stream-app/test-app-id/0_0/.checkpoint.tmp (No such file or
directory)
at java.io.FileOutputStream.open0(Native Method) ~[?:1.8.0_191]
at java.io.FileOutputStream.open(FileOutputStream.java:270) ~[?:1.8.0_191]
at java.io.FileOutputStream.<init>(FileOutputStream.java:213) ~[?:1.8.0_191]
at java.io.FileOutputStream.<init>(FileOutputStream.java:162) ~[?:1.8.0_191]
at org.apache.kafka.streams.state.internals.OffsetCheckpoint.write(
OffsetCheckpoint.java:79) ~[kafka-streams-2.2.1.jar:?]
at
org.apache.kafka.streams.processor.internals.ProcessorStateManager.checkpoint(
ProcessorStateManager.java:347) [kafka-streams-2.2.1.jar:?]
at org.apache.kafka.streams.processor.internals.StreamTask.commit(
StreamTask.java:476) [kafka-streams-2.2.1.jar:?]
at org.apache.kafka.streams.processor.internals.StreamTask.suspend(
StreamTask.java:598) [kafka-streams-2.2.1.jar:?]
at org.apache.kafka.streams.processor.internals.StreamTask.close(
StreamTask.java:724) [kafka-streams-2.2.1.jar:?]
at org.apache.kafka.streams.processor.internals.AssignedTasks.close(
AssignedTasks.java:337) [kafka-streams-2.2.1.jar:?]
at org.apache.kafka.streams.processor.internals.TaskManager.shutdown(
TaskManager.java:267) [kafka-streams-2.2.1.jar:?]
at
org.apache.kafka.streams.processor.internals.StreamThread.completeShutdown(
StreamThread.java:1208) [kafka-streams-2.2.1.jar:?]
at org.apache.kafka.streams.processor.internals.StreamThread.run(
StreamThread.java:785) [kafka-streams-2.2.1.jar:?]
Checking the system, in fact, the folder does not exist, but, others were
created:
# ls /tmp/kafka-stream-app/test-app-id/
# 1_0 1_1 1_2
After restarting the App, the same WARN shows-up but in this case, the
folders were created but not the .checkpoint.tmp file:
# ls /tmp/kafka-stream-app/test-app-id/
# 0_0 0_1 0_2 1_0 1_1 1_2
Am just reporting this because I found it strange/suspicious.
Cheers!
--
Jonathan
On Wed, May 8, 2019 at 9:26 PM Vahid Hashemian <va...@gmail.com>
wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for release of Apache Kafka 2.2.1, which
> includes many bug fixes for Apache Kafka 2.2.
>
> Release notes for the 2.2.1 release:
> https://home.apache.org/~vahid/kafka-2.2.1-rc0/RELEASE_NOTES.html
>
> *** Please download, test and vote by Monday, May 13, 6:00 pm PT.
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://home.apache.org/~vahid/kafka-2.2.1-rc0/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://home.apache.org/~vahid/kafka-2.2.1-rc0/javadoc/
>
> * Tag to be voted upon (off 2.2 branch) is the 2.2.1 tag:
> https://github.com/apache/kafka/releases/tag/2.2.1-rc0
>
> * Documentation:
> https://kafka.apache.org/22/documentation.html
>
> * Protocol:
> https://kafka.apache.org/22/protocol.html
>
> * Successful Jenkins builds for the 2.2 branch:
> Unit/integration tests: https://builds.apache.org/job/kafka-2.2-jdk8/106/
>
> Thanks,
> --Vahid
>
--
Santilli Jonathan
Re: [VOTE] 2.2.1 RC0
Posted by Jonathan Santilli <jo...@gmail.com>.
Hello Vahid,
am testing one of our Kafka Stream Apps with the 2.2.1-rc, after few
minutes, I see this WARN:
2019-05-09 13:14:37,025 WARN [test-app-id-dc27624a-8e02-
4031-963b-7596a8a77097-StreamThread-1] internals.ProcessorStateManager (
ProcessorStateManager.java:349) - task [0_0] Failed to write offset
checkpoint file to [/tmp/kafka-stream-app/test-app-id/0_0/.checkpoint]
java.io.FileNotFoundException:
/tmp/kafka-stream-app/test-app-id/0_0/.checkpoint.tmp (No such file or
directory)
at java.io.FileOutputStream.open0(Native Method) ~[?:1.8.0_191]
at java.io.FileOutputStream.open(FileOutputStream.java:270) ~[?:1.8.0_191]
at java.io.FileOutputStream.<init>(FileOutputStream.java:213) ~[?:1.8.0_191]
at java.io.FileOutputStream.<init>(FileOutputStream.java:162) ~[?:1.8.0_191]
at org.apache.kafka.streams.state.internals.OffsetCheckpoint.write(
OffsetCheckpoint.java:79) ~[kafka-streams-2.2.1.jar:?]
at
org.apache.kafka.streams.processor.internals.ProcessorStateManager.checkpoint(
ProcessorStateManager.java:347) [kafka-streams-2.2.1.jar:?]
at org.apache.kafka.streams.processor.internals.StreamTask.commit(
StreamTask.java:476) [kafka-streams-2.2.1.jar:?]
at org.apache.kafka.streams.processor.internals.StreamTask.suspend(
StreamTask.java:598) [kafka-streams-2.2.1.jar:?]
at org.apache.kafka.streams.processor.internals.StreamTask.close(
StreamTask.java:724) [kafka-streams-2.2.1.jar:?]
at org.apache.kafka.streams.processor.internals.AssignedTasks.close(
AssignedTasks.java:337) [kafka-streams-2.2.1.jar:?]
at org.apache.kafka.streams.processor.internals.TaskManager.shutdown(
TaskManager.java:267) [kafka-streams-2.2.1.jar:?]
at
org.apache.kafka.streams.processor.internals.StreamThread.completeShutdown(
StreamThread.java:1208) [kafka-streams-2.2.1.jar:?]
at org.apache.kafka.streams.processor.internals.StreamThread.run(
StreamThread.java:785) [kafka-streams-2.2.1.jar:?]
Checking the system, in fact, the folder does not exist, but, others were
created:
# ls /tmp/kafka-stream-app/test-app-id/
# 1_0 1_1 1_2
After restarting the App, the same WARN shows-up but in this case, the
folders were created but not the .checkpoint.tmp file:
# ls /tmp/kafka-stream-app/test-app-id/
# 0_0 0_1 0_2 1_0 1_1 1_2
Am just reporting this because I found it strange/suspicious.
Cheers!
--
Jonathan
On Wed, May 8, 2019 at 9:26 PM Vahid Hashemian <va...@gmail.com>
wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for release of Apache Kafka 2.2.1, which
> includes many bug fixes for Apache Kafka 2.2.
>
> Release notes for the 2.2.1 release:
> https://home.apache.org/~vahid/kafka-2.2.1-rc0/RELEASE_NOTES.html
>
> *** Please download, test and vote by Monday, May 13, 6:00 pm PT.
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://home.apache.org/~vahid/kafka-2.2.1-rc0/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://home.apache.org/~vahid/kafka-2.2.1-rc0/javadoc/
>
> * Tag to be voted upon (off 2.2 branch) is the 2.2.1 tag:
> https://github.com/apache/kafka/releases/tag/2.2.1-rc0
>
> * Documentation:
> https://kafka.apache.org/22/documentation.html
>
> * Protocol:
> https://kafka.apache.org/22/protocol.html
>
> * Successful Jenkins builds for the 2.2 branch:
> Unit/integration tests: https://builds.apache.org/job/kafka-2.2-jdk8/106/
>
> Thanks,
> --Vahid
>
--
Santilli Jonathan
Re: [VOTE] 2.2.1 RC0
Posted by Vahid Hashemian <va...@gmail.com>.
Hi Jason,
Sure. I'll wait for your PR to merge before cutting another RC.
Thanks!
--Vahid
On Fri, May 10, 2019 at 9:34 AM Jason Gustafson <ja...@confluent.io> wrote:
> Hi Vahid,
>
> I'd like to make the case for
> https://issues.apache.org/jira/browse/KAFKA-8335. This issue can cause
> unbounded growth in the __consumer_offsets topic when using transactions. I
> will have a patch ready today. Can we do another RC?
>
> Thanks,
> Jason
>
> On Wed, May 8, 2019 at 1:26 PM Vahid Hashemian <va...@gmail.com>
> wrote:
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the first candidate for release of Apache Kafka 2.2.1, which
> > includes many bug fixes for Apache Kafka 2.2.
> >
> > Release notes for the 2.2.1 release:
> > https://home.apache.org/~vahid/kafka-2.2.1-rc0/RELEASE_NOTES.html
> >
> > *** Please download, test and vote by Monday, May 13, 6:00 pm PT.
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > https://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > https://home.apache.org/~vahid/kafka-2.2.1-rc0/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >
> > * Javadoc:
> > https://home.apache.org/~vahid/kafka-2.2.1-rc0/javadoc/
> >
> > * Tag to be voted upon (off 2.2 branch) is the 2.2.1 tag:
> > https://github.com/apache/kafka/releases/tag/2.2.1-rc0
> >
> > * Documentation:
> > https://kafka.apache.org/22/documentation.html
> >
> > * Protocol:
> > https://kafka.apache.org/22/protocol.html
> >
> > * Successful Jenkins builds for the 2.2 branch:
> > Unit/integration tests:
> https://builds.apache.org/job/kafka-2.2-jdk8/106/
> >
> > Thanks,
> > --Vahid
> >
>
--
Thanks!
--Vahid
Re: [VOTE] 2.2.1 RC0
Posted by Jason Gustafson <ja...@confluent.io>.
Hi Vahid,
I'd like to make the case for
https://issues.apache.org/jira/browse/KAFKA-8335. This issue can cause
unbounded growth in the __consumer_offsets topic when using transactions. I
will have a patch ready today. Can we do another RC?
Thanks,
Jason
On Wed, May 8, 2019 at 1:26 PM Vahid Hashemian <va...@gmail.com>
wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for release of Apache Kafka 2.2.1, which
> includes many bug fixes for Apache Kafka 2.2.
>
> Release notes for the 2.2.1 release:
> https://home.apache.org/~vahid/kafka-2.2.1-rc0/RELEASE_NOTES.html
>
> *** Please download, test and vote by Monday, May 13, 6:00 pm PT.
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://home.apache.org/~vahid/kafka-2.2.1-rc0/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://home.apache.org/~vahid/kafka-2.2.1-rc0/javadoc/
>
> * Tag to be voted upon (off 2.2 branch) is the 2.2.1 tag:
> https://github.com/apache/kafka/releases/tag/2.2.1-rc0
>
> * Documentation:
> https://kafka.apache.org/22/documentation.html
>
> * Protocol:
> https://kafka.apache.org/22/protocol.html
>
> * Successful Jenkins builds for the 2.2 branch:
> Unit/integration tests: https://builds.apache.org/job/kafka-2.2-jdk8/106/
>
> Thanks,
> --Vahid
>