You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Yi Pan <ni...@gmail.com> on 2015/07/29 07:08:45 UTC
[DISCUSS] Release 0.10.0
Hi, all,
I want to start the discussion on the release schedule for 0.10.0. There
are a few important features that we plan to release in 0.10.0 and I want
to start this thread s.t. we can agree on what to include in 0.10.0 release.
There are the following main features added in 0.10.0:
- RocksDB TTL support
- Add CoordinatorStream and disable CheckpointManager
- Elasticsearch Producer
- Host affinity
And other 0.10.0 tickets:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
I propose to cut a 0.10.0 release after we get the following issues
resolved:
- SAMZA-615: Migrate checkpoint from checkpoint topic to Coordinator stream
- SAMZA-617: YARN host affinity in Samza
Thoughts?
Thanks!
-Yi
Re: [DISCUSS] Release 0.10.0
Posted by Yi Pan <ni...@gmail.com>.
Hi, Guozhang,
Good call. I moved SAMZA-697 out of 0.10.0. Boris is still working on that
and it is not clear yet whether this separation of class loaders will be
ready in 0.10.0.
Thanks!
-Yi
On Fri, Jul 31, 2015 at 10:12 AM, Guozhang Wang <wa...@gmail.com> wrote:
> Would SAMZA-697 worth adding to 0.10.0 as well?
>
> Guozhang
>
> On Thu, Jul 30, 2015 at 6:45 PM, Yan Fang <ya...@gmail.com> wrote:
>
> > For SAMZA-747, may ping Naveen or Chris. :)
> >
> > They have the permission to publish to maven. From the discuss
> > <https://github.com/facebook/rocksdb/issues/606>, they seem ready for
> the
> > release.
> >
> > Thanks,
> >
> > Fang, Yan
> > yanfang724@gmail.com
> >
> > On Thu, Jul 30, 2015 at 5:35 PM, Navina Ramesh
> > <nramesh@linkedin.com.invalid
> > > wrote:
> >
> > > Ok. I think it got confusing because you are talking about tickets NOT
> to
> > > be included in the release :)
> > >
> > > Got it, now. +1 for this list of exclusions!
> > > SAMZA-723 (StreamAppender bug) and SAMZA-747 (rocksdb) should be in
> > 0.10.0.
> > > I think we don't have an ETA on the fix for SAMZA-747?
> > >
> > > Thanks!
> > > Navina
> > >
> > > On Thu, Jul 30, 2015 at 5:26 PM, Yi Pan <ni...@gmail.com> wrote:
> > >
> > > > Uh... wrong math all the day today... :(
> > > >
> > > > Let me re-try:
> > > >
> > > > 29/32 tickets are to be moved to later (i.e. excluded from 0.10.0).
> > > >
> > > > 2/32 tickets (including SAMZA-723) will be included
> > > >
> > > > 1/32 ticket (SAMZA-689) is up for discussion and I am leaning toward
> > mark
> > > > it as won't fix.
> > > >
> > > > -Yi
> > > >
> > > > On Thu, Jul 30, 2015 at 5:24 PM, Yi Pan <ni...@gmail.com> wrote:
> > > >
> > > > > Hi, Navina,
> > > > >
> > > > > The 29/30 tickets are to be excluded from 0.10.0 (i.e. moved to
> > > 0.11.0),
> > > > > the three tickets are either to be included in 0.10.0, or won't
> fix.
> > > > >
> > > > > -Yi
> > > > >
> > > > > On Thu, Jul 30, 2015 at 5:10 PM, Navina Ramesh <
> > > > > nramesh@linkedin.com.invalid> wrote:
> > > > >
> > > > >> Hi Yi,
> > > > >> Thanks for summarizing. But why are we excluding SAMZA-723 from
> the
> > > > >> current
> > > > >> release ? Doesn't this break the existing StreamAppender
> > functionality
> > > > in
> > > > >> 0.9?
> > > > >>
> > > > >> Thanks!
> > > > >> Navina
> > > > >>
> > > > >> On Thu, Jul 30, 2015 at 4:55 PM, Yi Pan <ni...@gmail.com>
> > wrote:
> > > > >>
> > > > >> > Sorry, hit the send button too fast. Let me correct the summary
> > > > section:
> > > > >> >
> > > > >> > 29/32 tickets Status=open/reopen :
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
> > > > >> >
> > > > >> > Three tickets excluded:
> > > > >> > - SAMZA-689, @Chinmay, were you able to re-produce it? I am
> > thinking
> > > > of
> > > > >> > mark it as won't fix since 0.10.0 will remove the checkpoint
> topic
> > > > >> > altogether.
> > > > >> > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to
> include
> > > the
> > > > >> fix
> > > > >> > that breaks Samza 0.10.0 build on Linux boxes
> > > > >> > - SAMZA-723, stream appender deadlock issue
> > > > >> >
> > > > >> > Thanks!
> > > > >> >
> > > > >> > -Yi
> > > > >> >
> > > > >> >
> > > > >> > On Thu, Jul 30, 2015 at 4:52 PM, Yi Pan <ni...@gmail.com>
> > > wrote:
> > > > >> >
> > > > >> > > Hi, all,
> > > > >> > >
> > > > >> > > Thanks a lot for helping out to select the features planned in
> > > > 0.10.0.
> > > > >> > >
> > > > >> > > Based on the above discussion, I am proposing to move the
> > > following
> > > > >> > > tickets later (i.e. 0.11.0).
> > > > >> > >
> > > > >> > > 30/32 tickets Status=open/reopen (exception for SAMZA-723:
> > stream
> > > > >> > appender
> > > > >> > > deadlock issue) :
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
> > > > >> > >
> > > > >> > > Two tickets excluded:
> > > > >> > > - SAMZA-689, @Chinmay, were you able to re-produce it? I am
> > > thinking
> > > > >> of
> > > > >> > > mark it as won't fix since 0.10.0 will remove the checkpoint
> > topic
> > > > >> > > altogether.
> > > > >> > > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to
> > include
> > > > the
> > > > >> fix
> > > > >> > > that breaks Samza 0.10.0 build on Linux boxes
> > > > >> > >
> > > > >> > > Anything that I missed?
> > > > >> > >
> > > > >> > > Thanks!
> > > > >> > >
> > > > >> > > -Yi
> > > > >> > >
> > > > >> > >
> > > > >> > > On Wed, Jul 29, 2015 at 9:57 AM, Navina Ramesh <
> > > > >> > > nramesh@linkedin.com.invalid> wrote:
> > > > >> > >
> > > > >> > >> +1 for StreamAppender bug fix as a mandatory in 0.10.
> > > > >> > >>
> > > > >> > >> @Yan: will review SAMZA-676 today and test it out locally.
> > > > >> > >>
> > > > >> > >> Thanks!
> > > > >> > >> Navina
> > > > >> > >>
> > > > >> > >> On Wed, Jul 29, 2015 at 8:38 AM, Chinmay Soman <
> > > > >> > chinmay.cerebro@gmail.com
> > > > >> > >> >
> > > > >> > >> wrote:
> > > > >> > >>
> > > > >> > >> > I can take care of SAMZA-340, SAMZA-683 and will follow up
> > > with
> > > > >> Luis
> > > > >> > >> for
> > > > >> > >> > SAMZA-401,2,3,4
> > > > >> > >> >
> > > > >> > >> > On Wed, Jul 29, 2015 at 12:10 AM, Dan <
> danharvey42@gmail.com
> > >
> > > > >> wrote:
> > > > >> > >> >
> > > > >> > >> > > I agree SAMZA-741 for the ElasticSearch producer should
> be
> > in
> > > > >> too so
> > > > >> > >> > we've
> > > > >> > >> > > got a better API as part of that release.
> > > > >> > >> > >
> > > > >> > >> > > - Dan
> > > > >> > >> > >
> > > > >> > >> > >
> > > > >> > >> > > On 29 July 2015 at 07:51, Yan Fang <yanfang724@gmail.com
> >
> > > > wrote:
> > > > >> > >> > >
> > > > >> > >> > > > Actually, I also want to include a few patch-available
> > > > >> features,
> > > > >> > >> > > > especially:
> > > > >> > >> > > >
> > > > >> > >> > > > 1. broadcast stream (SAMZA-676)
> > > > >> > >> > > > - waiting for review
> > > > >> > >> > > >
> > > > >> > >> > > > 2. graphite support (SAMZA-340)
> > > > >> > >> > > > 3. meter and histogram (SAMZA-683)
> > > > >> > >> > > > 4. utility (SAMZA-401)
> > > > >> > >> > > > - 2,3,4 belong to Luis, if he does not have time to
> > > > update,
> > > > >> > >> since
> > > > >> > >> > > they
> > > > >> > >> > > > only need some small changes, we can edit it and get +1
> > > from
> > > > >> > another
> > > > >> > >> > > > committer.
> > > > >> > >> > > >
> > > > >> > >> > > > 5. hdfs producer (SAMZA-693)
> > > > >> > >> > > > - I am reviewing.
> > > > >> > >> > > >
> > > > >> > >> > > > 6. upgrade yarn to 2.7.1 (SAMZA-563)
> > > > >> > >> > > > - though I am reviewing, this ticket is negotiable
> if
> > we
> > > > >> want
> > > > >> > to
> > > > >> > >> put
> > > > >> > >> > > > into the 0.10.0 release. If we do not, I think, when
> > users
> > > > >> enable
> > > > >> > >> the
> > > > >> > >> > > > worker-persisting and container-persisting features,
> > Samza
> > > > will
> > > > >> > not
> > > > >> > >> be
> > > > >> > >> > > able
> > > > >> > >> > > > to handle it. (Some classes are only available after
> yarn
> > > > 2.5.0
> > > > >> > >> while
> > > > >> > >> > > Samza
> > > > >> > >> > > > currently only support yarn 2.4.0)
> > > > >> > >> > > >
> > > > >> > >> > > > 7. others: scrooge, class loader isolation, etc.
> > > > >> > >> > > > - those are waiting for reviewing too.
> > > > >> > >> > > >
> > > > >> > >> > > > My opinion is that, if we can clean up all the
> > > > patch-available
> > > > >> > >> tickets,
> > > > >> > >> > > it
> > > > >> > >> > > > will be great. Most of them have been already reviewed
> > more
> > > > >> than
> > > > >> > >> once.
> > > > >> > >> > > So I
> > > > >> > >> > > > think it should not be very time-consuming to have them
> > in
> > > > the
> > > > >> > >> 0.10.0
> > > > >> > >> > > > release.
> > > > >> > >> > > >
> > > > >> > >> > > > What do you think?
> > > > >> > >> > > >
> > > > >> > >> > > > Of course, another must-have is the bug-fix of the
> Stream
> > > > >> > Appender.
> > > > >> > >> :)
> > > > >> > >> > > >
> > > > >> > >> > > > Thanks,
> > > > >> > >> > > >
> > > > >> > >> > > > Fang, Yan
> > > > >> > >> > > > yanfang724@gmail.com
> > > > >> > >> > > >
> > > > >> > >> > > > On Tue, Jul 28, 2015 at 10:27 PM, Roger Hoover <
> > > > >> > >> roger.hoover@gmail.com
> > > > >> > >> > >
> > > > >> > >> > > > wrote:
> > > > >> > >> > > >
> > > > >> > >> > > > > Thanks, Yi.
> > > > >> > >> > > > >
> > > > >> > >> > > > > I propose that we also include SAMZA-741 for
> > > Elasticsearch
> > > > >> > >> versioning
> > > > >> > >> > > > > support with the new ES producer. I think it's very
> > > close
> > > > to
> > > > >> > >> being
> > > > >> > >> > > > merged.
> > > > >> > >> > > > >
> > > > >> > >> > > > > Roger
> > > > >> > >> > > > >
> > > > >> > >> > > > >
> > > > >> > >> > > > > On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan <
> > > > >> nickpan47@gmail.com>
> > > > >> > >> > wrote:
> > > > >> > >> > > > >
> > > > >> > >> > > > > > Hi, all,
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > I want to start the discussion on the release
> > schedule
> > > > for
> > > > >> > >> 0.10.0.
> > > > >> > >> > > > There
> > > > >> > >> > > > > > are a few important features that we plan to
> release
> > in
> > > > >> 0.10.0
> > > > >> > >> and
> > > > >> > >> > I
> > > > >> > >> > > > want
> > > > >> > >> > > > > > to start this thread s.t. we can agree on what to
> > > include
> > > > >> in
> > > > >> > >> 0.10.0
> > > > >> > >> > > > > > release.
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > There are the following main features added in
> > 0.10.0:
> > > > >> > >> > > > > > - RocksDB TTL support
> > > > >> > >> > > > > > - Add CoordinatorStream and disable
> CheckpointManager
> > > > >> > >> > > > > > - Elasticsearch Producer
> > > > >> > >> > > > > > - Host affinity
> > > > >> > >> > > > > > And other 0.10.0 tickets:
> > > > >> > >> > > > > >
> > > > >> > >> > > > > >
> > > > >> > >> > > > >
> > > > >> > >> > > >
> > > > >> > >> > >
> > > > >> > >> >
> > > > >> > >>
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > I propose to cut a 0.10.0 release after we get the
> > > > >> following
> > > > >> > >> issues
> > > > >> > >> > > > > > resolved:
> > > > >> > >> > > > > > - SAMZA-615: Migrate checkpoint from checkpoint
> topic
> > > to
> > > > >> > >> > Coordinator
> > > > >> > >> > > > > stream
> > > > >> > >> > > > > > - SAMZA-617: YARN host affinity in Samza
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > Thoughts?
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > Thanks!
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > -Yi
> > > > >> > >> > > > > >
> > > > >> > >> > > > >
> > > > >> > >> > > >
> > > > >> > >> > >
> > > > >> > >> >
> > > > >> > >> >
> > > > >> > >> >
> > > > >> > >> > --
> > > > >> > >> > Thanks and regards
> > > > >> > >> >
> > > > >> > >> > Chinmay Soman
> > > > >> > >> >
> > > > >> > >>
> > > > >> > >>
> > > > >> > >>
> > > > >> > >> --
> > > > >> > >> Navina R.
> > > > >> > >>
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Navina R.
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Navina R.
> > >
> >
>
>
>
> --
> -- Guozhang
>
Re: [DISCUSS] Release 0.10.0
Posted by Guozhang Wang <wa...@gmail.com>.
Would SAMZA-697 worth adding to 0.10.0 as well?
Guozhang
On Thu, Jul 30, 2015 at 6:45 PM, Yan Fang <ya...@gmail.com> wrote:
> For SAMZA-747, may ping Naveen or Chris. :)
>
> They have the permission to publish to maven. From the discuss
> <https://github.com/facebook/rocksdb/issues/606>, they seem ready for the
> release.
>
> Thanks,
>
> Fang, Yan
> yanfang724@gmail.com
>
> On Thu, Jul 30, 2015 at 5:35 PM, Navina Ramesh
> <nramesh@linkedin.com.invalid
> > wrote:
>
> > Ok. I think it got confusing because you are talking about tickets NOT to
> > be included in the release :)
> >
> > Got it, now. +1 for this list of exclusions!
> > SAMZA-723 (StreamAppender bug) and SAMZA-747 (rocksdb) should be in
> 0.10.0.
> > I think we don't have an ETA on the fix for SAMZA-747?
> >
> > Thanks!
> > Navina
> >
> > On Thu, Jul 30, 2015 at 5:26 PM, Yi Pan <ni...@gmail.com> wrote:
> >
> > > Uh... wrong math all the day today... :(
> > >
> > > Let me re-try:
> > >
> > > 29/32 tickets are to be moved to later (i.e. excluded from 0.10.0).
> > >
> > > 2/32 tickets (including SAMZA-723) will be included
> > >
> > > 1/32 ticket (SAMZA-689) is up for discussion and I am leaning toward
> mark
> > > it as won't fix.
> > >
> > > -Yi
> > >
> > > On Thu, Jul 30, 2015 at 5:24 PM, Yi Pan <ni...@gmail.com> wrote:
> > >
> > > > Hi, Navina,
> > > >
> > > > The 29/30 tickets are to be excluded from 0.10.0 (i.e. moved to
> > 0.11.0),
> > > > the three tickets are either to be included in 0.10.0, or won't fix.
> > > >
> > > > -Yi
> > > >
> > > > On Thu, Jul 30, 2015 at 5:10 PM, Navina Ramesh <
> > > > nramesh@linkedin.com.invalid> wrote:
> > > >
> > > >> Hi Yi,
> > > >> Thanks for summarizing. But why are we excluding SAMZA-723 from the
> > > >> current
> > > >> release ? Doesn't this break the existing StreamAppender
> functionality
> > > in
> > > >> 0.9?
> > > >>
> > > >> Thanks!
> > > >> Navina
> > > >>
> > > >> On Thu, Jul 30, 2015 at 4:55 PM, Yi Pan <ni...@gmail.com>
> wrote:
> > > >>
> > > >> > Sorry, hit the send button too fast. Let me correct the summary
> > > section:
> > > >> >
> > > >> > 29/32 tickets Status=open/reopen :
> > > >> >
> > > >> >
> > > >>
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
> > > >> >
> > > >> > Three tickets excluded:
> > > >> > - SAMZA-689, @Chinmay, were you able to re-produce it? I am
> thinking
> > > of
> > > >> > mark it as won't fix since 0.10.0 will remove the checkpoint topic
> > > >> > altogether.
> > > >> > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include
> > the
> > > >> fix
> > > >> > that breaks Samza 0.10.0 build on Linux boxes
> > > >> > - SAMZA-723, stream appender deadlock issue
> > > >> >
> > > >> > Thanks!
> > > >> >
> > > >> > -Yi
> > > >> >
> > > >> >
> > > >> > On Thu, Jul 30, 2015 at 4:52 PM, Yi Pan <ni...@gmail.com>
> > wrote:
> > > >> >
> > > >> > > Hi, all,
> > > >> > >
> > > >> > > Thanks a lot for helping out to select the features planned in
> > > 0.10.0.
> > > >> > >
> > > >> > > Based on the above discussion, I am proposing to move the
> > following
> > > >> > > tickets later (i.e. 0.11.0).
> > > >> > >
> > > >> > > 30/32 tickets Status=open/reopen (exception for SAMZA-723:
> stream
> > > >> > appender
> > > >> > > deadlock issue) :
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
> > > >> > >
> > > >> > > Two tickets excluded:
> > > >> > > - SAMZA-689, @Chinmay, were you able to re-produce it? I am
> > thinking
> > > >> of
> > > >> > > mark it as won't fix since 0.10.0 will remove the checkpoint
> topic
> > > >> > > altogether.
> > > >> > > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to
> include
> > > the
> > > >> fix
> > > >> > > that breaks Samza 0.10.0 build on Linux boxes
> > > >> > >
> > > >> > > Anything that I missed?
> > > >> > >
> > > >> > > Thanks!
> > > >> > >
> > > >> > > -Yi
> > > >> > >
> > > >> > >
> > > >> > > On Wed, Jul 29, 2015 at 9:57 AM, Navina Ramesh <
> > > >> > > nramesh@linkedin.com.invalid> wrote:
> > > >> > >
> > > >> > >> +1 for StreamAppender bug fix as a mandatory in 0.10.
> > > >> > >>
> > > >> > >> @Yan: will review SAMZA-676 today and test it out locally.
> > > >> > >>
> > > >> > >> Thanks!
> > > >> > >> Navina
> > > >> > >>
> > > >> > >> On Wed, Jul 29, 2015 at 8:38 AM, Chinmay Soman <
> > > >> > chinmay.cerebro@gmail.com
> > > >> > >> >
> > > >> > >> wrote:
> > > >> > >>
> > > >> > >> > I can take care of SAMZA-340, SAMZA-683 and will follow up
> > with
> > > >> Luis
> > > >> > >> for
> > > >> > >> > SAMZA-401,2,3,4
> > > >> > >> >
> > > >> > >> > On Wed, Jul 29, 2015 at 12:10 AM, Dan <danharvey42@gmail.com
> >
> > > >> wrote:
> > > >> > >> >
> > > >> > >> > > I agree SAMZA-741 for the ElasticSearch producer should be
> in
> > > >> too so
> > > >> > >> > we've
> > > >> > >> > > got a better API as part of that release.
> > > >> > >> > >
> > > >> > >> > > - Dan
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> > > On 29 July 2015 at 07:51, Yan Fang <ya...@gmail.com>
> > > wrote:
> > > >> > >> > >
> > > >> > >> > > > Actually, I also want to include a few patch-available
> > > >> features,
> > > >> > >> > > > especially:
> > > >> > >> > > >
> > > >> > >> > > > 1. broadcast stream (SAMZA-676)
> > > >> > >> > > > - waiting for review
> > > >> > >> > > >
> > > >> > >> > > > 2. graphite support (SAMZA-340)
> > > >> > >> > > > 3. meter and histogram (SAMZA-683)
> > > >> > >> > > > 4. utility (SAMZA-401)
> > > >> > >> > > > - 2,3,4 belong to Luis, if he does not have time to
> > > update,
> > > >> > >> since
> > > >> > >> > > they
> > > >> > >> > > > only need some small changes, we can edit it and get +1
> > from
> > > >> > another
> > > >> > >> > > > committer.
> > > >> > >> > > >
> > > >> > >> > > > 5. hdfs producer (SAMZA-693)
> > > >> > >> > > > - I am reviewing.
> > > >> > >> > > >
> > > >> > >> > > > 6. upgrade yarn to 2.7.1 (SAMZA-563)
> > > >> > >> > > > - though I am reviewing, this ticket is negotiable if
> we
> > > >> want
> > > >> > to
> > > >> > >> put
> > > >> > >> > > > into the 0.10.0 release. If we do not, I think, when
> users
> > > >> enable
> > > >> > >> the
> > > >> > >> > > > worker-persisting and container-persisting features,
> Samza
> > > will
> > > >> > not
> > > >> > >> be
> > > >> > >> > > able
> > > >> > >> > > > to handle it. (Some classes are only available after yarn
> > > 2.5.0
> > > >> > >> while
> > > >> > >> > > Samza
> > > >> > >> > > > currently only support yarn 2.4.0)
> > > >> > >> > > >
> > > >> > >> > > > 7. others: scrooge, class loader isolation, etc.
> > > >> > >> > > > - those are waiting for reviewing too.
> > > >> > >> > > >
> > > >> > >> > > > My opinion is that, if we can clean up all the
> > > patch-available
> > > >> > >> tickets,
> > > >> > >> > > it
> > > >> > >> > > > will be great. Most of them have been already reviewed
> more
> > > >> than
> > > >> > >> once.
> > > >> > >> > > So I
> > > >> > >> > > > think it should not be very time-consuming to have them
> in
> > > the
> > > >> > >> 0.10.0
> > > >> > >> > > > release.
> > > >> > >> > > >
> > > >> > >> > > > What do you think?
> > > >> > >> > > >
> > > >> > >> > > > Of course, another must-have is the bug-fix of the Stream
> > > >> > Appender.
> > > >> > >> :)
> > > >> > >> > > >
> > > >> > >> > > > Thanks,
> > > >> > >> > > >
> > > >> > >> > > > Fang, Yan
> > > >> > >> > > > yanfang724@gmail.com
> > > >> > >> > > >
> > > >> > >> > > > On Tue, Jul 28, 2015 at 10:27 PM, Roger Hoover <
> > > >> > >> roger.hoover@gmail.com
> > > >> > >> > >
> > > >> > >> > > > wrote:
> > > >> > >> > > >
> > > >> > >> > > > > Thanks, Yi.
> > > >> > >> > > > >
> > > >> > >> > > > > I propose that we also include SAMZA-741 for
> > Elasticsearch
> > > >> > >> versioning
> > > >> > >> > > > > support with the new ES producer. I think it's very
> > close
> > > to
> > > >> > >> being
> > > >> > >> > > > merged.
> > > >> > >> > > > >
> > > >> > >> > > > > Roger
> > > >> > >> > > > >
> > > >> > >> > > > >
> > > >> > >> > > > > On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan <
> > > >> nickpan47@gmail.com>
> > > >> > >> > wrote:
> > > >> > >> > > > >
> > > >> > >> > > > > > Hi, all,
> > > >> > >> > > > > >
> > > >> > >> > > > > > I want to start the discussion on the release
> schedule
> > > for
> > > >> > >> 0.10.0.
> > > >> > >> > > > There
> > > >> > >> > > > > > are a few important features that we plan to release
> in
> > > >> 0.10.0
> > > >> > >> and
> > > >> > >> > I
> > > >> > >> > > > want
> > > >> > >> > > > > > to start this thread s.t. we can agree on what to
> > include
> > > >> in
> > > >> > >> 0.10.0
> > > >> > >> > > > > > release.
> > > >> > >> > > > > >
> > > >> > >> > > > > > There are the following main features added in
> 0.10.0:
> > > >> > >> > > > > > - RocksDB TTL support
> > > >> > >> > > > > > - Add CoordinatorStream and disable CheckpointManager
> > > >> > >> > > > > > - Elasticsearch Producer
> > > >> > >> > > > > > - Host affinity
> > > >> > >> > > > > > And other 0.10.0 tickets:
> > > >> > >> > > > > >
> > > >> > >> > > > > >
> > > >> > >> > > > >
> > > >> > >> > > >
> > > >> > >> > >
> > > >> > >> >
> > > >> > >>
> > > >> >
> > > >>
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
> > > >> > >> > > > > >
> > > >> > >> > > > > > I propose to cut a 0.10.0 release after we get the
> > > >> following
> > > >> > >> issues
> > > >> > >> > > > > > resolved:
> > > >> > >> > > > > > - SAMZA-615: Migrate checkpoint from checkpoint topic
> > to
> > > >> > >> > Coordinator
> > > >> > >> > > > > stream
> > > >> > >> > > > > > - SAMZA-617: YARN host affinity in Samza
> > > >> > >> > > > > >
> > > >> > >> > > > > > Thoughts?
> > > >> > >> > > > > >
> > > >> > >> > > > > > Thanks!
> > > >> > >> > > > > >
> > > >> > >> > > > > > -Yi
> > > >> > >> > > > > >
> > > >> > >> > > > >
> > > >> > >> > > >
> > > >> > >> > >
> > > >> > >> >
> > > >> > >> >
> > > >> > >> >
> > > >> > >> > --
> > > >> > >> > Thanks and regards
> > > >> > >> >
> > > >> > >> > Chinmay Soman
> > > >> > >> >
> > > >> > >>
> > > >> > >>
> > > >> > >>
> > > >> > >> --
> > > >> > >> Navina R.
> > > >> > >>
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Navina R.
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Navina R.
> >
>
--
-- Guozhang
Re: [DISCUSS] Release 0.10.0
Posted by Yan Fang <ya...@gmail.com>.
For SAMZA-747, may ping Naveen or Chris. :)
They have the permission to publish to maven. From the discuss
<https://github.com/facebook/rocksdb/issues/606>, they seem ready for the
release.
Thanks,
Fang, Yan
yanfang724@gmail.com
On Thu, Jul 30, 2015 at 5:35 PM, Navina Ramesh <nramesh@linkedin.com.invalid
> wrote:
> Ok. I think it got confusing because you are talking about tickets NOT to
> be included in the release :)
>
> Got it, now. +1 for this list of exclusions!
> SAMZA-723 (StreamAppender bug) and SAMZA-747 (rocksdb) should be in 0.10.0.
> I think we don't have an ETA on the fix for SAMZA-747?
>
> Thanks!
> Navina
>
> On Thu, Jul 30, 2015 at 5:26 PM, Yi Pan <ni...@gmail.com> wrote:
>
> > Uh... wrong math all the day today... :(
> >
> > Let me re-try:
> >
> > 29/32 tickets are to be moved to later (i.e. excluded from 0.10.0).
> >
> > 2/32 tickets (including SAMZA-723) will be included
> >
> > 1/32 ticket (SAMZA-689) is up for discussion and I am leaning toward mark
> > it as won't fix.
> >
> > -Yi
> >
> > On Thu, Jul 30, 2015 at 5:24 PM, Yi Pan <ni...@gmail.com> wrote:
> >
> > > Hi, Navina,
> > >
> > > The 29/30 tickets are to be excluded from 0.10.0 (i.e. moved to
> 0.11.0),
> > > the three tickets are either to be included in 0.10.0, or won't fix.
> > >
> > > -Yi
> > >
> > > On Thu, Jul 30, 2015 at 5:10 PM, Navina Ramesh <
> > > nramesh@linkedin.com.invalid> wrote:
> > >
> > >> Hi Yi,
> > >> Thanks for summarizing. But why are we excluding SAMZA-723 from the
> > >> current
> > >> release ? Doesn't this break the existing StreamAppender functionality
> > in
> > >> 0.9?
> > >>
> > >> Thanks!
> > >> Navina
> > >>
> > >> On Thu, Jul 30, 2015 at 4:55 PM, Yi Pan <ni...@gmail.com> wrote:
> > >>
> > >> > Sorry, hit the send button too fast. Let me correct the summary
> > section:
> > >> >
> > >> > 29/32 tickets Status=open/reopen :
> > >> >
> > >> >
> > >>
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
> > >> >
> > >> > Three tickets excluded:
> > >> > - SAMZA-689, @Chinmay, were you able to re-produce it? I am thinking
> > of
> > >> > mark it as won't fix since 0.10.0 will remove the checkpoint topic
> > >> > altogether.
> > >> > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include
> the
> > >> fix
> > >> > that breaks Samza 0.10.0 build on Linux boxes
> > >> > - SAMZA-723, stream appender deadlock issue
> > >> >
> > >> > Thanks!
> > >> >
> > >> > -Yi
> > >> >
> > >> >
> > >> > On Thu, Jul 30, 2015 at 4:52 PM, Yi Pan <ni...@gmail.com>
> wrote:
> > >> >
> > >> > > Hi, all,
> > >> > >
> > >> > > Thanks a lot for helping out to select the features planned in
> > 0.10.0.
> > >> > >
> > >> > > Based on the above discussion, I am proposing to move the
> following
> > >> > > tickets later (i.e. 0.11.0).
> > >> > >
> > >> > > 30/32 tickets Status=open/reopen (exception for SAMZA-723: stream
> > >> > appender
> > >> > > deadlock issue) :
> > >> > >
> > >> >
> > >>
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
> > >> > >
> > >> > > Two tickets excluded:
> > >> > > - SAMZA-689, @Chinmay, were you able to re-produce it? I am
> thinking
> > >> of
> > >> > > mark it as won't fix since 0.10.0 will remove the checkpoint topic
> > >> > > altogether.
> > >> > > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include
> > the
> > >> fix
> > >> > > that breaks Samza 0.10.0 build on Linux boxes
> > >> > >
> > >> > > Anything that I missed?
> > >> > >
> > >> > > Thanks!
> > >> > >
> > >> > > -Yi
> > >> > >
> > >> > >
> > >> > > On Wed, Jul 29, 2015 at 9:57 AM, Navina Ramesh <
> > >> > > nramesh@linkedin.com.invalid> wrote:
> > >> > >
> > >> > >> +1 for StreamAppender bug fix as a mandatory in 0.10.
> > >> > >>
> > >> > >> @Yan: will review SAMZA-676 today and test it out locally.
> > >> > >>
> > >> > >> Thanks!
> > >> > >> Navina
> > >> > >>
> > >> > >> On Wed, Jul 29, 2015 at 8:38 AM, Chinmay Soman <
> > >> > chinmay.cerebro@gmail.com
> > >> > >> >
> > >> > >> wrote:
> > >> > >>
> > >> > >> > I can take care of SAMZA-340, SAMZA-683 and will follow up
> with
> > >> Luis
> > >> > >> for
> > >> > >> > SAMZA-401,2,3,4
> > >> > >> >
> > >> > >> > On Wed, Jul 29, 2015 at 12:10 AM, Dan <da...@gmail.com>
> > >> wrote:
> > >> > >> >
> > >> > >> > > I agree SAMZA-741 for the ElasticSearch producer should be in
> > >> too so
> > >> > >> > we've
> > >> > >> > > got a better API as part of that release.
> > >> > >> > >
> > >> > >> > > - Dan
> > >> > >> > >
> > >> > >> > >
> > >> > >> > > On 29 July 2015 at 07:51, Yan Fang <ya...@gmail.com>
> > wrote:
> > >> > >> > >
> > >> > >> > > > Actually, I also want to include a few patch-available
> > >> features,
> > >> > >> > > > especially:
> > >> > >> > > >
> > >> > >> > > > 1. broadcast stream (SAMZA-676)
> > >> > >> > > > - waiting for review
> > >> > >> > > >
> > >> > >> > > > 2. graphite support (SAMZA-340)
> > >> > >> > > > 3. meter and histogram (SAMZA-683)
> > >> > >> > > > 4. utility (SAMZA-401)
> > >> > >> > > > - 2,3,4 belong to Luis, if he does not have time to
> > update,
> > >> > >> since
> > >> > >> > > they
> > >> > >> > > > only need some small changes, we can edit it and get +1
> from
> > >> > another
> > >> > >> > > > committer.
> > >> > >> > > >
> > >> > >> > > > 5. hdfs producer (SAMZA-693)
> > >> > >> > > > - I am reviewing.
> > >> > >> > > >
> > >> > >> > > > 6. upgrade yarn to 2.7.1 (SAMZA-563)
> > >> > >> > > > - though I am reviewing, this ticket is negotiable if we
> > >> want
> > >> > to
> > >> > >> put
> > >> > >> > > > into the 0.10.0 release. If we do not, I think, when users
> > >> enable
> > >> > >> the
> > >> > >> > > > worker-persisting and container-persisting features, Samza
> > will
> > >> > not
> > >> > >> be
> > >> > >> > > able
> > >> > >> > > > to handle it. (Some classes are only available after yarn
> > 2.5.0
> > >> > >> while
> > >> > >> > > Samza
> > >> > >> > > > currently only support yarn 2.4.0)
> > >> > >> > > >
> > >> > >> > > > 7. others: scrooge, class loader isolation, etc.
> > >> > >> > > > - those are waiting for reviewing too.
> > >> > >> > > >
> > >> > >> > > > My opinion is that, if we can clean up all the
> > patch-available
> > >> > >> tickets,
> > >> > >> > > it
> > >> > >> > > > will be great. Most of them have been already reviewed more
> > >> than
> > >> > >> once.
> > >> > >> > > So I
> > >> > >> > > > think it should not be very time-consuming to have them in
> > the
> > >> > >> 0.10.0
> > >> > >> > > > release.
> > >> > >> > > >
> > >> > >> > > > What do you think?
> > >> > >> > > >
> > >> > >> > > > Of course, another must-have is the bug-fix of the Stream
> > >> > Appender.
> > >> > >> :)
> > >> > >> > > >
> > >> > >> > > > Thanks,
> > >> > >> > > >
> > >> > >> > > > Fang, Yan
> > >> > >> > > > yanfang724@gmail.com
> > >> > >> > > >
> > >> > >> > > > On Tue, Jul 28, 2015 at 10:27 PM, Roger Hoover <
> > >> > >> roger.hoover@gmail.com
> > >> > >> > >
> > >> > >> > > > wrote:
> > >> > >> > > >
> > >> > >> > > > > Thanks, Yi.
> > >> > >> > > > >
> > >> > >> > > > > I propose that we also include SAMZA-741 for
> Elasticsearch
> > >> > >> versioning
> > >> > >> > > > > support with the new ES producer. I think it's very
> close
> > to
> > >> > >> being
> > >> > >> > > > merged.
> > >> > >> > > > >
> > >> > >> > > > > Roger
> > >> > >> > > > >
> > >> > >> > > > >
> > >> > >> > > > > On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan <
> > >> nickpan47@gmail.com>
> > >> > >> > wrote:
> > >> > >> > > > >
> > >> > >> > > > > > Hi, all,
> > >> > >> > > > > >
> > >> > >> > > > > > I want to start the discussion on the release schedule
> > for
> > >> > >> 0.10.0.
> > >> > >> > > > There
> > >> > >> > > > > > are a few important features that we plan to release in
> > >> 0.10.0
> > >> > >> and
> > >> > >> > I
> > >> > >> > > > want
> > >> > >> > > > > > to start this thread s.t. we can agree on what to
> include
> > >> in
> > >> > >> 0.10.0
> > >> > >> > > > > > release.
> > >> > >> > > > > >
> > >> > >> > > > > > There are the following main features added in 0.10.0:
> > >> > >> > > > > > - RocksDB TTL support
> > >> > >> > > > > > - Add CoordinatorStream and disable CheckpointManager
> > >> > >> > > > > > - Elasticsearch Producer
> > >> > >> > > > > > - Host affinity
> > >> > >> > > > > > And other 0.10.0 tickets:
> > >> > >> > > > > >
> > >> > >> > > > > >
> > >> > >> > > > >
> > >> > >> > > >
> > >> > >> > >
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
> > >> > >> > > > > >
> > >> > >> > > > > > I propose to cut a 0.10.0 release after we get the
> > >> following
> > >> > >> issues
> > >> > >> > > > > > resolved:
> > >> > >> > > > > > - SAMZA-615: Migrate checkpoint from checkpoint topic
> to
> > >> > >> > Coordinator
> > >> > >> > > > > stream
> > >> > >> > > > > > - SAMZA-617: YARN host affinity in Samza
> > >> > >> > > > > >
> > >> > >> > > > > > Thoughts?
> > >> > >> > > > > >
> > >> > >> > > > > > Thanks!
> > >> > >> > > > > >
> > >> > >> > > > > > -Yi
> > >> > >> > > > > >
> > >> > >> > > > >
> > >> > >> > > >
> > >> > >> > >
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > --
> > >> > >> > Thanks and regards
> > >> > >> >
> > >> > >> > Chinmay Soman
> > >> > >> >
> > >> > >>
> > >> > >>
> > >> > >>
> > >> > >> --
> > >> > >> Navina R.
> > >> > >>
> > >> > >
> > >> > >
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Navina R.
> > >>
> > >
> > >
> >
>
>
>
> --
> Navina R.
>
Re: [DISCUSS] Release 0.10.0
Posted by Navina Ramesh <nr...@linkedin.com.INVALID>.
Ok. I think it got confusing because you are talking about tickets NOT to
be included in the release :)
Got it, now. +1 for this list of exclusions!
SAMZA-723 (StreamAppender bug) and SAMZA-747 (rocksdb) should be in 0.10.0.
I think we don't have an ETA on the fix for SAMZA-747?
Thanks!
Navina
On Thu, Jul 30, 2015 at 5:26 PM, Yi Pan <ni...@gmail.com> wrote:
> Uh... wrong math all the day today... :(
>
> Let me re-try:
>
> 29/32 tickets are to be moved to later (i.e. excluded from 0.10.0).
>
> 2/32 tickets (including SAMZA-723) will be included
>
> 1/32 ticket (SAMZA-689) is up for discussion and I am leaning toward mark
> it as won't fix.
>
> -Yi
>
> On Thu, Jul 30, 2015 at 5:24 PM, Yi Pan <ni...@gmail.com> wrote:
>
> > Hi, Navina,
> >
> > The 29/30 tickets are to be excluded from 0.10.0 (i.e. moved to 0.11.0),
> > the three tickets are either to be included in 0.10.0, or won't fix.
> >
> > -Yi
> >
> > On Thu, Jul 30, 2015 at 5:10 PM, Navina Ramesh <
> > nramesh@linkedin.com.invalid> wrote:
> >
> >> Hi Yi,
> >> Thanks for summarizing. But why are we excluding SAMZA-723 from the
> >> current
> >> release ? Doesn't this break the existing StreamAppender functionality
> in
> >> 0.9?
> >>
> >> Thanks!
> >> Navina
> >>
> >> On Thu, Jul 30, 2015 at 4:55 PM, Yi Pan <ni...@gmail.com> wrote:
> >>
> >> > Sorry, hit the send button too fast. Let me correct the summary
> section:
> >> >
> >> > 29/32 tickets Status=open/reopen :
> >> >
> >> >
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
> >> >
> >> > Three tickets excluded:
> >> > - SAMZA-689, @Chinmay, were you able to re-produce it? I am thinking
> of
> >> > mark it as won't fix since 0.10.0 will remove the checkpoint topic
> >> > altogether.
> >> > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include the
> >> fix
> >> > that breaks Samza 0.10.0 build on Linux boxes
> >> > - SAMZA-723, stream appender deadlock issue
> >> >
> >> > Thanks!
> >> >
> >> > -Yi
> >> >
> >> >
> >> > On Thu, Jul 30, 2015 at 4:52 PM, Yi Pan <ni...@gmail.com> wrote:
> >> >
> >> > > Hi, all,
> >> > >
> >> > > Thanks a lot for helping out to select the features planned in
> 0.10.0.
> >> > >
> >> > > Based on the above discussion, I am proposing to move the following
> >> > > tickets later (i.e. 0.11.0).
> >> > >
> >> > > 30/32 tickets Status=open/reopen (exception for SAMZA-723: stream
> >> > appender
> >> > > deadlock issue) :
> >> > >
> >> >
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
> >> > >
> >> > > Two tickets excluded:
> >> > > - SAMZA-689, @Chinmay, were you able to re-produce it? I am thinking
> >> of
> >> > > mark it as won't fix since 0.10.0 will remove the checkpoint topic
> >> > > altogether.
> >> > > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include
> the
> >> fix
> >> > > that breaks Samza 0.10.0 build on Linux boxes
> >> > >
> >> > > Anything that I missed?
> >> > >
> >> > > Thanks!
> >> > >
> >> > > -Yi
> >> > >
> >> > >
> >> > > On Wed, Jul 29, 2015 at 9:57 AM, Navina Ramesh <
> >> > > nramesh@linkedin.com.invalid> wrote:
> >> > >
> >> > >> +1 for StreamAppender bug fix as a mandatory in 0.10.
> >> > >>
> >> > >> @Yan: will review SAMZA-676 today and test it out locally.
> >> > >>
> >> > >> Thanks!
> >> > >> Navina
> >> > >>
> >> > >> On Wed, Jul 29, 2015 at 8:38 AM, Chinmay Soman <
> >> > chinmay.cerebro@gmail.com
> >> > >> >
> >> > >> wrote:
> >> > >>
> >> > >> > I can take care of SAMZA-340, SAMZA-683 and will follow up with
> >> Luis
> >> > >> for
> >> > >> > SAMZA-401,2,3,4
> >> > >> >
> >> > >> > On Wed, Jul 29, 2015 at 12:10 AM, Dan <da...@gmail.com>
> >> wrote:
> >> > >> >
> >> > >> > > I agree SAMZA-741 for the ElasticSearch producer should be in
> >> too so
> >> > >> > we've
> >> > >> > > got a better API as part of that release.
> >> > >> > >
> >> > >> > > - Dan
> >> > >> > >
> >> > >> > >
> >> > >> > > On 29 July 2015 at 07:51, Yan Fang <ya...@gmail.com>
> wrote:
> >> > >> > >
> >> > >> > > > Actually, I also want to include a few patch-available
> >> features,
> >> > >> > > > especially:
> >> > >> > > >
> >> > >> > > > 1. broadcast stream (SAMZA-676)
> >> > >> > > > - waiting for review
> >> > >> > > >
> >> > >> > > > 2. graphite support (SAMZA-340)
> >> > >> > > > 3. meter and histogram (SAMZA-683)
> >> > >> > > > 4. utility (SAMZA-401)
> >> > >> > > > - 2,3,4 belong to Luis, if he does not have time to
> update,
> >> > >> since
> >> > >> > > they
> >> > >> > > > only need some small changes, we can edit it and get +1 from
> >> > another
> >> > >> > > > committer.
> >> > >> > > >
> >> > >> > > > 5. hdfs producer (SAMZA-693)
> >> > >> > > > - I am reviewing.
> >> > >> > > >
> >> > >> > > > 6. upgrade yarn to 2.7.1 (SAMZA-563)
> >> > >> > > > - though I am reviewing, this ticket is negotiable if we
> >> want
> >> > to
> >> > >> put
> >> > >> > > > into the 0.10.0 release. If we do not, I think, when users
> >> enable
> >> > >> the
> >> > >> > > > worker-persisting and container-persisting features, Samza
> will
> >> > not
> >> > >> be
> >> > >> > > able
> >> > >> > > > to handle it. (Some classes are only available after yarn
> 2.5.0
> >> > >> while
> >> > >> > > Samza
> >> > >> > > > currently only support yarn 2.4.0)
> >> > >> > > >
> >> > >> > > > 7. others: scrooge, class loader isolation, etc.
> >> > >> > > > - those are waiting for reviewing too.
> >> > >> > > >
> >> > >> > > > My opinion is that, if we can clean up all the
> patch-available
> >> > >> tickets,
> >> > >> > > it
> >> > >> > > > will be great. Most of them have been already reviewed more
> >> than
> >> > >> once.
> >> > >> > > So I
> >> > >> > > > think it should not be very time-consuming to have them in
> the
> >> > >> 0.10.0
> >> > >> > > > release.
> >> > >> > > >
> >> > >> > > > What do you think?
> >> > >> > > >
> >> > >> > > > Of course, another must-have is the bug-fix of the Stream
> >> > Appender.
> >> > >> :)
> >> > >> > > >
> >> > >> > > > Thanks,
> >> > >> > > >
> >> > >> > > > Fang, Yan
> >> > >> > > > yanfang724@gmail.com
> >> > >> > > >
> >> > >> > > > On Tue, Jul 28, 2015 at 10:27 PM, Roger Hoover <
> >> > >> roger.hoover@gmail.com
> >> > >> > >
> >> > >> > > > wrote:
> >> > >> > > >
> >> > >> > > > > Thanks, Yi.
> >> > >> > > > >
> >> > >> > > > > I propose that we also include SAMZA-741 for Elasticsearch
> >> > >> versioning
> >> > >> > > > > support with the new ES producer. I think it's very close
> to
> >> > >> being
> >> > >> > > > merged.
> >> > >> > > > >
> >> > >> > > > > Roger
> >> > >> > > > >
> >> > >> > > > >
> >> > >> > > > > On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan <
> >> nickpan47@gmail.com>
> >> > >> > wrote:
> >> > >> > > > >
> >> > >> > > > > > Hi, all,
> >> > >> > > > > >
> >> > >> > > > > > I want to start the discussion on the release schedule
> for
> >> > >> 0.10.0.
> >> > >> > > > There
> >> > >> > > > > > are a few important features that we plan to release in
> >> 0.10.0
> >> > >> and
> >> > >> > I
> >> > >> > > > want
> >> > >> > > > > > to start this thread s.t. we can agree on what to include
> >> in
> >> > >> 0.10.0
> >> > >> > > > > > release.
> >> > >> > > > > >
> >> > >> > > > > > There are the following main features added in 0.10.0:
> >> > >> > > > > > - RocksDB TTL support
> >> > >> > > > > > - Add CoordinatorStream and disable CheckpointManager
> >> > >> > > > > > - Elasticsearch Producer
> >> > >> > > > > > - Host affinity
> >> > >> > > > > > And other 0.10.0 tickets:
> >> > >> > > > > >
> >> > >> > > > > >
> >> > >> > > > >
> >> > >> > > >
> >> > >> > >
> >> > >> >
> >> > >>
> >> >
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
> >> > >> > > > > >
> >> > >> > > > > > I propose to cut a 0.10.0 release after we get the
> >> following
> >> > >> issues
> >> > >> > > > > > resolved:
> >> > >> > > > > > - SAMZA-615: Migrate checkpoint from checkpoint topic to
> >> > >> > Coordinator
> >> > >> > > > > stream
> >> > >> > > > > > - SAMZA-617: YARN host affinity in Samza
> >> > >> > > > > >
> >> > >> > > > > > Thoughts?
> >> > >> > > > > >
> >> > >> > > > > > Thanks!
> >> > >> > > > > >
> >> > >> > > > > > -Yi
> >> > >> > > > > >
> >> > >> > > > >
> >> > >> > > >
> >> > >> > >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> > --
> >> > >> > Thanks and regards
> >> > >> >
> >> > >> > Chinmay Soman
> >> > >> >
> >> > >>
> >> > >>
> >> > >>
> >> > >> --
> >> > >> Navina R.
> >> > >>
> >> > >
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Navina R.
> >>
> >
> >
>
--
Navina R.
Re: [DISCUSS] Release 0.10.0
Posted by Yi Pan <ni...@gmail.com>.
Uh... wrong math all the day today... :(
Let me re-try:
29/32 tickets are to be moved to later (i.e. excluded from 0.10.0).
2/32 tickets (including SAMZA-723) will be included
1/32 ticket (SAMZA-689) is up for discussion and I am leaning toward mark
it as won't fix.
-Yi
On Thu, Jul 30, 2015 at 5:24 PM, Yi Pan <ni...@gmail.com> wrote:
> Hi, Navina,
>
> The 29/30 tickets are to be excluded from 0.10.0 (i.e. moved to 0.11.0),
> the three tickets are either to be included in 0.10.0, or won't fix.
>
> -Yi
>
> On Thu, Jul 30, 2015 at 5:10 PM, Navina Ramesh <
> nramesh@linkedin.com.invalid> wrote:
>
>> Hi Yi,
>> Thanks for summarizing. But why are we excluding SAMZA-723 from the
>> current
>> release ? Doesn't this break the existing StreamAppender functionality in
>> 0.9?
>>
>> Thanks!
>> Navina
>>
>> On Thu, Jul 30, 2015 at 4:55 PM, Yi Pan <ni...@gmail.com> wrote:
>>
>> > Sorry, hit the send button too fast. Let me correct the summary section:
>> >
>> > 29/32 tickets Status=open/reopen :
>> >
>> >
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>> >
>> > Three tickets excluded:
>> > - SAMZA-689, @Chinmay, were you able to re-produce it? I am thinking of
>> > mark it as won't fix since 0.10.0 will remove the checkpoint topic
>> > altogether.
>> > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include the
>> fix
>> > that breaks Samza 0.10.0 build on Linux boxes
>> > - SAMZA-723, stream appender deadlock issue
>> >
>> > Thanks!
>> >
>> > -Yi
>> >
>> >
>> > On Thu, Jul 30, 2015 at 4:52 PM, Yi Pan <ni...@gmail.com> wrote:
>> >
>> > > Hi, all,
>> > >
>> > > Thanks a lot for helping out to select the features planned in 0.10.0.
>> > >
>> > > Based on the above discussion, I am proposing to move the following
>> > > tickets later (i.e. 0.11.0).
>> > >
>> > > 30/32 tickets Status=open/reopen (exception for SAMZA-723: stream
>> > appender
>> > > deadlock issue) :
>> > >
>> >
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>> > >
>> > > Two tickets excluded:
>> > > - SAMZA-689, @Chinmay, were you able to re-produce it? I am thinking
>> of
>> > > mark it as won't fix since 0.10.0 will remove the checkpoint topic
>> > > altogether.
>> > > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include the
>> fix
>> > > that breaks Samza 0.10.0 build on Linux boxes
>> > >
>> > > Anything that I missed?
>> > >
>> > > Thanks!
>> > >
>> > > -Yi
>> > >
>> > >
>> > > On Wed, Jul 29, 2015 at 9:57 AM, Navina Ramesh <
>> > > nramesh@linkedin.com.invalid> wrote:
>> > >
>> > >> +1 for StreamAppender bug fix as a mandatory in 0.10.
>> > >>
>> > >> @Yan: will review SAMZA-676 today and test it out locally.
>> > >>
>> > >> Thanks!
>> > >> Navina
>> > >>
>> > >> On Wed, Jul 29, 2015 at 8:38 AM, Chinmay Soman <
>> > chinmay.cerebro@gmail.com
>> > >> >
>> > >> wrote:
>> > >>
>> > >> > I can take care of SAMZA-340, SAMZA-683 and will follow up with
>> Luis
>> > >> for
>> > >> > SAMZA-401,2,3,4
>> > >> >
>> > >> > On Wed, Jul 29, 2015 at 12:10 AM, Dan <da...@gmail.com>
>> wrote:
>> > >> >
>> > >> > > I agree SAMZA-741 for the ElasticSearch producer should be in
>> too so
>> > >> > we've
>> > >> > > got a better API as part of that release.
>> > >> > >
>> > >> > > - Dan
>> > >> > >
>> > >> > >
>> > >> > > On 29 July 2015 at 07:51, Yan Fang <ya...@gmail.com> wrote:
>> > >> > >
>> > >> > > > Actually, I also want to include a few patch-available
>> features,
>> > >> > > > especially:
>> > >> > > >
>> > >> > > > 1. broadcast stream (SAMZA-676)
>> > >> > > > - waiting for review
>> > >> > > >
>> > >> > > > 2. graphite support (SAMZA-340)
>> > >> > > > 3. meter and histogram (SAMZA-683)
>> > >> > > > 4. utility (SAMZA-401)
>> > >> > > > - 2,3,4 belong to Luis, if he does not have time to update,
>> > >> since
>> > >> > > they
>> > >> > > > only need some small changes, we can edit it and get +1 from
>> > another
>> > >> > > > committer.
>> > >> > > >
>> > >> > > > 5. hdfs producer (SAMZA-693)
>> > >> > > > - I am reviewing.
>> > >> > > >
>> > >> > > > 6. upgrade yarn to 2.7.1 (SAMZA-563)
>> > >> > > > - though I am reviewing, this ticket is negotiable if we
>> want
>> > to
>> > >> put
>> > >> > > > into the 0.10.0 release. If we do not, I think, when users
>> enable
>> > >> the
>> > >> > > > worker-persisting and container-persisting features, Samza will
>> > not
>> > >> be
>> > >> > > able
>> > >> > > > to handle it. (Some classes are only available after yarn 2.5.0
>> > >> while
>> > >> > > Samza
>> > >> > > > currently only support yarn 2.4.0)
>> > >> > > >
>> > >> > > > 7. others: scrooge, class loader isolation, etc.
>> > >> > > > - those are waiting for reviewing too.
>> > >> > > >
>> > >> > > > My opinion is that, if we can clean up all the patch-available
>> > >> tickets,
>> > >> > > it
>> > >> > > > will be great. Most of them have been already reviewed more
>> than
>> > >> once.
>> > >> > > So I
>> > >> > > > think it should not be very time-consuming to have them in the
>> > >> 0.10.0
>> > >> > > > release.
>> > >> > > >
>> > >> > > > What do you think?
>> > >> > > >
>> > >> > > > Of course, another must-have is the bug-fix of the Stream
>> > Appender.
>> > >> :)
>> > >> > > >
>> > >> > > > Thanks,
>> > >> > > >
>> > >> > > > Fang, Yan
>> > >> > > > yanfang724@gmail.com
>> > >> > > >
>> > >> > > > On Tue, Jul 28, 2015 at 10:27 PM, Roger Hoover <
>> > >> roger.hoover@gmail.com
>> > >> > >
>> > >> > > > wrote:
>> > >> > > >
>> > >> > > > > Thanks, Yi.
>> > >> > > > >
>> > >> > > > > I propose that we also include SAMZA-741 for Elasticsearch
>> > >> versioning
>> > >> > > > > support with the new ES producer. I think it's very close to
>> > >> being
>> > >> > > > merged.
>> > >> > > > >
>> > >> > > > > Roger
>> > >> > > > >
>> > >> > > > >
>> > >> > > > > On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan <
>> nickpan47@gmail.com>
>> > >> > wrote:
>> > >> > > > >
>> > >> > > > > > Hi, all,
>> > >> > > > > >
>> > >> > > > > > I want to start the discussion on the release schedule for
>> > >> 0.10.0.
>> > >> > > > There
>> > >> > > > > > are a few important features that we plan to release in
>> 0.10.0
>> > >> and
>> > >> > I
>> > >> > > > want
>> > >> > > > > > to start this thread s.t. we can agree on what to include
>> in
>> > >> 0.10.0
>> > >> > > > > > release.
>> > >> > > > > >
>> > >> > > > > > There are the following main features added in 0.10.0:
>> > >> > > > > > - RocksDB TTL support
>> > >> > > > > > - Add CoordinatorStream and disable CheckpointManager
>> > >> > > > > > - Elasticsearch Producer
>> > >> > > > > > - Host affinity
>> > >> > > > > > And other 0.10.0 tickets:
>> > >> > > > > >
>> > >> > > > > >
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
>> > >> > > > > >
>> > >> > > > > > I propose to cut a 0.10.0 release after we get the
>> following
>> > >> issues
>> > >> > > > > > resolved:
>> > >> > > > > > - SAMZA-615: Migrate checkpoint from checkpoint topic to
>> > >> > Coordinator
>> > >> > > > > stream
>> > >> > > > > > - SAMZA-617: YARN host affinity in Samza
>> > >> > > > > >
>> > >> > > > > > Thoughts?
>> > >> > > > > >
>> > >> > > > > > Thanks!
>> > >> > > > > >
>> > >> > > > > > -Yi
>> > >> > > > > >
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >> >
>> > >> >
>> > >> > --
>> > >> > Thanks and regards
>> > >> >
>> > >> > Chinmay Soman
>> > >> >
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> Navina R.
>> > >>
>> > >
>> > >
>> >
>>
>>
>>
>> --
>> Navina R.
>>
>
>
Re: [DISCUSS] Release 0.10.0
Posted by Yi Pan <ni...@gmail.com>.
Hi, Navina,
The 29/30 tickets are to be excluded from 0.10.0 (i.e. moved to 0.11.0),
the three tickets are either to be included in 0.10.0, or won't fix.
-Yi
On Thu, Jul 30, 2015 at 5:10 PM, Navina Ramesh <nramesh@linkedin.com.invalid
> wrote:
> Hi Yi,
> Thanks for summarizing. But why are we excluding SAMZA-723 from the current
> release ? Doesn't this break the existing StreamAppender functionality in
> 0.9?
>
> Thanks!
> Navina
>
> On Thu, Jul 30, 2015 at 4:55 PM, Yi Pan <ni...@gmail.com> wrote:
>
> > Sorry, hit the send button too fast. Let me correct the summary section:
> >
> > 29/32 tickets Status=open/reopen :
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
> >
> > Three tickets excluded:
> > - SAMZA-689, @Chinmay, were you able to re-produce it? I am thinking of
> > mark it as won't fix since 0.10.0 will remove the checkpoint topic
> > altogether.
> > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include the fix
> > that breaks Samza 0.10.0 build on Linux boxes
> > - SAMZA-723, stream appender deadlock issue
> >
> > Thanks!
> >
> > -Yi
> >
> >
> > On Thu, Jul 30, 2015 at 4:52 PM, Yi Pan <ni...@gmail.com> wrote:
> >
> > > Hi, all,
> > >
> > > Thanks a lot for helping out to select the features planned in 0.10.0.
> > >
> > > Based on the above discussion, I am proposing to move the following
> > > tickets later (i.e. 0.11.0).
> > >
> > > 30/32 tickets Status=open/reopen (exception for SAMZA-723: stream
> > appender
> > > deadlock issue) :
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
> > >
> > > Two tickets excluded:
> > > - SAMZA-689, @Chinmay, were you able to re-produce it? I am thinking of
> > > mark it as won't fix since 0.10.0 will remove the checkpoint topic
> > > altogether.
> > > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include the
> fix
> > > that breaks Samza 0.10.0 build on Linux boxes
> > >
> > > Anything that I missed?
> > >
> > > Thanks!
> > >
> > > -Yi
> > >
> > >
> > > On Wed, Jul 29, 2015 at 9:57 AM, Navina Ramesh <
> > > nramesh@linkedin.com.invalid> wrote:
> > >
> > >> +1 for StreamAppender bug fix as a mandatory in 0.10.
> > >>
> > >> @Yan: will review SAMZA-676 today and test it out locally.
> > >>
> > >> Thanks!
> > >> Navina
> > >>
> > >> On Wed, Jul 29, 2015 at 8:38 AM, Chinmay Soman <
> > chinmay.cerebro@gmail.com
> > >> >
> > >> wrote:
> > >>
> > >> > I can take care of SAMZA-340, SAMZA-683 and will follow up with
> Luis
> > >> for
> > >> > SAMZA-401,2,3,4
> > >> >
> > >> > On Wed, Jul 29, 2015 at 12:10 AM, Dan <da...@gmail.com>
> wrote:
> > >> >
> > >> > > I agree SAMZA-741 for the ElasticSearch producer should be in too
> so
> > >> > we've
> > >> > > got a better API as part of that release.
> > >> > >
> > >> > > - Dan
> > >> > >
> > >> > >
> > >> > > On 29 July 2015 at 07:51, Yan Fang <ya...@gmail.com> wrote:
> > >> > >
> > >> > > > Actually, I also want to include a few patch-available features,
> > >> > > > especially:
> > >> > > >
> > >> > > > 1. broadcast stream (SAMZA-676)
> > >> > > > - waiting for review
> > >> > > >
> > >> > > > 2. graphite support (SAMZA-340)
> > >> > > > 3. meter and histogram (SAMZA-683)
> > >> > > > 4. utility (SAMZA-401)
> > >> > > > - 2,3,4 belong to Luis, if he does not have time to update,
> > >> since
> > >> > > they
> > >> > > > only need some small changes, we can edit it and get +1 from
> > another
> > >> > > > committer.
> > >> > > >
> > >> > > > 5. hdfs producer (SAMZA-693)
> > >> > > > - I am reviewing.
> > >> > > >
> > >> > > > 6. upgrade yarn to 2.7.1 (SAMZA-563)
> > >> > > > - though I am reviewing, this ticket is negotiable if we want
> > to
> > >> put
> > >> > > > into the 0.10.0 release. If we do not, I think, when users
> enable
> > >> the
> > >> > > > worker-persisting and container-persisting features, Samza will
> > not
> > >> be
> > >> > > able
> > >> > > > to handle it. (Some classes are only available after yarn 2.5.0
> > >> while
> > >> > > Samza
> > >> > > > currently only support yarn 2.4.0)
> > >> > > >
> > >> > > > 7. others: scrooge, class loader isolation, etc.
> > >> > > > - those are waiting for reviewing too.
> > >> > > >
> > >> > > > My opinion is that, if we can clean up all the patch-available
> > >> tickets,
> > >> > > it
> > >> > > > will be great. Most of them have been already reviewed more than
> > >> once.
> > >> > > So I
> > >> > > > think it should not be very time-consuming to have them in the
> > >> 0.10.0
> > >> > > > release.
> > >> > > >
> > >> > > > What do you think?
> > >> > > >
> > >> > > > Of course, another must-have is the bug-fix of the Stream
> > Appender.
> > >> :)
> > >> > > >
> > >> > > > Thanks,
> > >> > > >
> > >> > > > Fang, Yan
> > >> > > > yanfang724@gmail.com
> > >> > > >
> > >> > > > On Tue, Jul 28, 2015 at 10:27 PM, Roger Hoover <
> > >> roger.hoover@gmail.com
> > >> > >
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Thanks, Yi.
> > >> > > > >
> > >> > > > > I propose that we also include SAMZA-741 for Elasticsearch
> > >> versioning
> > >> > > > > support with the new ES producer. I think it's very close to
> > >> being
> > >> > > > merged.
> > >> > > > >
> > >> > > > > Roger
> > >> > > > >
> > >> > > > >
> > >> > > > > On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan <nickpan47@gmail.com
> >
> > >> > wrote:
> > >> > > > >
> > >> > > > > > Hi, all,
> > >> > > > > >
> > >> > > > > > I want to start the discussion on the release schedule for
> > >> 0.10.0.
> > >> > > > There
> > >> > > > > > are a few important features that we plan to release in
> 0.10.0
> > >> and
> > >> > I
> > >> > > > want
> > >> > > > > > to start this thread s.t. we can agree on what to include in
> > >> 0.10.0
> > >> > > > > > release.
> > >> > > > > >
> > >> > > > > > There are the following main features added in 0.10.0:
> > >> > > > > > - RocksDB TTL support
> > >> > > > > > - Add CoordinatorStream and disable CheckpointManager
> > >> > > > > > - Elasticsearch Producer
> > >> > > > > > - Host affinity
> > >> > > > > > And other 0.10.0 tickets:
> > >> > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
> > >> > > > > >
> > >> > > > > > I propose to cut a 0.10.0 release after we get the following
> > >> issues
> > >> > > > > > resolved:
> > >> > > > > > - SAMZA-615: Migrate checkpoint from checkpoint topic to
> > >> > Coordinator
> > >> > > > > stream
> > >> > > > > > - SAMZA-617: YARN host affinity in Samza
> > >> > > > > >
> > >> > > > > > Thoughts?
> > >> > > > > >
> > >> > > > > > Thanks!
> > >> > > > > >
> > >> > > > > > -Yi
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Thanks and regards
> > >> >
> > >> > Chinmay Soman
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Navina R.
> > >>
> > >
> > >
> >
>
>
>
> --
> Navina R.
>
Re: [DISCUSS] Release 0.10.0
Posted by Navina Ramesh <nr...@linkedin.com.INVALID>.
Hi Yi,
Thanks for summarizing. But why are we excluding SAMZA-723 from the current
release ? Doesn't this break the existing StreamAppender functionality in
0.9?
Thanks!
Navina
On Thu, Jul 30, 2015 at 4:55 PM, Yi Pan <ni...@gmail.com> wrote:
> Sorry, hit the send button too fast. Let me correct the summary section:
>
> 29/32 tickets Status=open/reopen :
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>
> Three tickets excluded:
> - SAMZA-689, @Chinmay, were you able to re-produce it? I am thinking of
> mark it as won't fix since 0.10.0 will remove the checkpoint topic
> altogether.
> - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include the fix
> that breaks Samza 0.10.0 build on Linux boxes
> - SAMZA-723, stream appender deadlock issue
>
> Thanks!
>
> -Yi
>
>
> On Thu, Jul 30, 2015 at 4:52 PM, Yi Pan <ni...@gmail.com> wrote:
>
> > Hi, all,
> >
> > Thanks a lot for helping out to select the features planned in 0.10.0.
> >
> > Based on the above discussion, I am proposing to move the following
> > tickets later (i.e. 0.11.0).
> >
> > 30/32 tickets Status=open/reopen (exception for SAMZA-723: stream
> appender
> > deadlock issue) :
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
> >
> > Two tickets excluded:
> > - SAMZA-689, @Chinmay, were you able to re-produce it? I am thinking of
> > mark it as won't fix since 0.10.0 will remove the checkpoint topic
> > altogether.
> > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include the fix
> > that breaks Samza 0.10.0 build on Linux boxes
> >
> > Anything that I missed?
> >
> > Thanks!
> >
> > -Yi
> >
> >
> > On Wed, Jul 29, 2015 at 9:57 AM, Navina Ramesh <
> > nramesh@linkedin.com.invalid> wrote:
> >
> >> +1 for StreamAppender bug fix as a mandatory in 0.10.
> >>
> >> @Yan: will review SAMZA-676 today and test it out locally.
> >>
> >> Thanks!
> >> Navina
> >>
> >> On Wed, Jul 29, 2015 at 8:38 AM, Chinmay Soman <
> chinmay.cerebro@gmail.com
> >> >
> >> wrote:
> >>
> >> > I can take care of SAMZA-340, SAMZA-683 and will follow up with Luis
> >> for
> >> > SAMZA-401,2,3,4
> >> >
> >> > On Wed, Jul 29, 2015 at 12:10 AM, Dan <da...@gmail.com> wrote:
> >> >
> >> > > I agree SAMZA-741 for the ElasticSearch producer should be in too so
> >> > we've
> >> > > got a better API as part of that release.
> >> > >
> >> > > - Dan
> >> > >
> >> > >
> >> > > On 29 July 2015 at 07:51, Yan Fang <ya...@gmail.com> wrote:
> >> > >
> >> > > > Actually, I also want to include a few patch-available features,
> >> > > > especially:
> >> > > >
> >> > > > 1. broadcast stream (SAMZA-676)
> >> > > > - waiting for review
> >> > > >
> >> > > > 2. graphite support (SAMZA-340)
> >> > > > 3. meter and histogram (SAMZA-683)
> >> > > > 4. utility (SAMZA-401)
> >> > > > - 2,3,4 belong to Luis, if he does not have time to update,
> >> since
> >> > > they
> >> > > > only need some small changes, we can edit it and get +1 from
> another
> >> > > > committer.
> >> > > >
> >> > > > 5. hdfs producer (SAMZA-693)
> >> > > > - I am reviewing.
> >> > > >
> >> > > > 6. upgrade yarn to 2.7.1 (SAMZA-563)
> >> > > > - though I am reviewing, this ticket is negotiable if we want
> to
> >> put
> >> > > > into the 0.10.0 release. If we do not, I think, when users enable
> >> the
> >> > > > worker-persisting and container-persisting features, Samza will
> not
> >> be
> >> > > able
> >> > > > to handle it. (Some classes are only available after yarn 2.5.0
> >> while
> >> > > Samza
> >> > > > currently only support yarn 2.4.0)
> >> > > >
> >> > > > 7. others: scrooge, class loader isolation, etc.
> >> > > > - those are waiting for reviewing too.
> >> > > >
> >> > > > My opinion is that, if we can clean up all the patch-available
> >> tickets,
> >> > > it
> >> > > > will be great. Most of them have been already reviewed more than
> >> once.
> >> > > So I
> >> > > > think it should not be very time-consuming to have them in the
> >> 0.10.0
> >> > > > release.
> >> > > >
> >> > > > What do you think?
> >> > > >
> >> > > > Of course, another must-have is the bug-fix of the Stream
> Appender.
> >> :)
> >> > > >
> >> > > > Thanks,
> >> > > >
> >> > > > Fang, Yan
> >> > > > yanfang724@gmail.com
> >> > > >
> >> > > > On Tue, Jul 28, 2015 at 10:27 PM, Roger Hoover <
> >> roger.hoover@gmail.com
> >> > >
> >> > > > wrote:
> >> > > >
> >> > > > > Thanks, Yi.
> >> > > > >
> >> > > > > I propose that we also include SAMZA-741 for Elasticsearch
> >> versioning
> >> > > > > support with the new ES producer. I think it's very close to
> >> being
> >> > > > merged.
> >> > > > >
> >> > > > > Roger
> >> > > > >
> >> > > > >
> >> > > > > On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan <ni...@gmail.com>
> >> > wrote:
> >> > > > >
> >> > > > > > Hi, all,
> >> > > > > >
> >> > > > > > I want to start the discussion on the release schedule for
> >> 0.10.0.
> >> > > > There
> >> > > > > > are a few important features that we plan to release in 0.10.0
> >> and
> >> > I
> >> > > > want
> >> > > > > > to start this thread s.t. we can agree on what to include in
> >> 0.10.0
> >> > > > > > release.
> >> > > > > >
> >> > > > > > There are the following main features added in 0.10.0:
> >> > > > > > - RocksDB TTL support
> >> > > > > > - Add CoordinatorStream and disable CheckpointManager
> >> > > > > > - Elasticsearch Producer
> >> > > > > > - Host affinity
> >> > > > > > And other 0.10.0 tickets:
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
> >> > > > > >
> >> > > > > > I propose to cut a 0.10.0 release after we get the following
> >> issues
> >> > > > > > resolved:
> >> > > > > > - SAMZA-615: Migrate checkpoint from checkpoint topic to
> >> > Coordinator
> >> > > > > stream
> >> > > > > > - SAMZA-617: YARN host affinity in Samza
> >> > > > > >
> >> > > > > > Thoughts?
> >> > > > > >
> >> > > > > > Thanks!
> >> > > > > >
> >> > > > > > -Yi
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Thanks and regards
> >> >
> >> > Chinmay Soman
> >> >
> >>
> >>
> >>
> >> --
> >> Navina R.
> >>
> >
> >
>
--
Navina R.
Re: [DISCUSS] Release 0.10.0
Posted by Yi Pan <ni...@gmail.com>.
Sorry, hit the send button too fast. Let me correct the summary section:
29/32 tickets Status=open/reopen :
https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
Three tickets excluded:
- SAMZA-689, @Chinmay, were you able to re-produce it? I am thinking of
mark it as won't fix since 0.10.0 will remove the checkpoint topic
altogether.
- SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include the fix
that breaks Samza 0.10.0 build on Linux boxes
- SAMZA-723, stream appender deadlock issue
Thanks!
-Yi
On Thu, Jul 30, 2015 at 4:52 PM, Yi Pan <ni...@gmail.com> wrote:
> Hi, all,
>
> Thanks a lot for helping out to select the features planned in 0.10.0.
>
> Based on the above discussion, I am proposing to move the following
> tickets later (i.e. 0.11.0).
>
> 30/32 tickets Status=open/reopen (exception for SAMZA-723: stream appender
> deadlock issue) :
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>
> Two tickets excluded:
> - SAMZA-689, @Chinmay, were you able to re-produce it? I am thinking of
> mark it as won't fix since 0.10.0 will remove the checkpoint topic
> altogether.
> - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include the fix
> that breaks Samza 0.10.0 build on Linux boxes
>
> Anything that I missed?
>
> Thanks!
>
> -Yi
>
>
> On Wed, Jul 29, 2015 at 9:57 AM, Navina Ramesh <
> nramesh@linkedin.com.invalid> wrote:
>
>> +1 for StreamAppender bug fix as a mandatory in 0.10.
>>
>> @Yan: will review SAMZA-676 today and test it out locally.
>>
>> Thanks!
>> Navina
>>
>> On Wed, Jul 29, 2015 at 8:38 AM, Chinmay Soman <chinmay.cerebro@gmail.com
>> >
>> wrote:
>>
>> > I can take care of SAMZA-340, SAMZA-683 and will follow up with Luis
>> for
>> > SAMZA-401,2,3,4
>> >
>> > On Wed, Jul 29, 2015 at 12:10 AM, Dan <da...@gmail.com> wrote:
>> >
>> > > I agree SAMZA-741 for the ElasticSearch producer should be in too so
>> > we've
>> > > got a better API as part of that release.
>> > >
>> > > - Dan
>> > >
>> > >
>> > > On 29 July 2015 at 07:51, Yan Fang <ya...@gmail.com> wrote:
>> > >
>> > > > Actually, I also want to include a few patch-available features,
>> > > > especially:
>> > > >
>> > > > 1. broadcast stream (SAMZA-676)
>> > > > - waiting for review
>> > > >
>> > > > 2. graphite support (SAMZA-340)
>> > > > 3. meter and histogram (SAMZA-683)
>> > > > 4. utility (SAMZA-401)
>> > > > - 2,3,4 belong to Luis, if he does not have time to update,
>> since
>> > > they
>> > > > only need some small changes, we can edit it and get +1 from another
>> > > > committer.
>> > > >
>> > > > 5. hdfs producer (SAMZA-693)
>> > > > - I am reviewing.
>> > > >
>> > > > 6. upgrade yarn to 2.7.1 (SAMZA-563)
>> > > > - though I am reviewing, this ticket is negotiable if we want to
>> put
>> > > > into the 0.10.0 release. If we do not, I think, when users enable
>> the
>> > > > worker-persisting and container-persisting features, Samza will not
>> be
>> > > able
>> > > > to handle it. (Some classes are only available after yarn 2.5.0
>> while
>> > > Samza
>> > > > currently only support yarn 2.4.0)
>> > > >
>> > > > 7. others: scrooge, class loader isolation, etc.
>> > > > - those are waiting for reviewing too.
>> > > >
>> > > > My opinion is that, if we can clean up all the patch-available
>> tickets,
>> > > it
>> > > > will be great. Most of them have been already reviewed more than
>> once.
>> > > So I
>> > > > think it should not be very time-consuming to have them in the
>> 0.10.0
>> > > > release.
>> > > >
>> > > > What do you think?
>> > > >
>> > > > Of course, another must-have is the bug-fix of the Stream Appender.
>> :)
>> > > >
>> > > > Thanks,
>> > > >
>> > > > Fang, Yan
>> > > > yanfang724@gmail.com
>> > > >
>> > > > On Tue, Jul 28, 2015 at 10:27 PM, Roger Hoover <
>> roger.hoover@gmail.com
>> > >
>> > > > wrote:
>> > > >
>> > > > > Thanks, Yi.
>> > > > >
>> > > > > I propose that we also include SAMZA-741 for Elasticsearch
>> versioning
>> > > > > support with the new ES producer. I think it's very close to
>> being
>> > > > merged.
>> > > > >
>> > > > > Roger
>> > > > >
>> > > > >
>> > > > > On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan <ni...@gmail.com>
>> > wrote:
>> > > > >
>> > > > > > Hi, all,
>> > > > > >
>> > > > > > I want to start the discussion on the release schedule for
>> 0.10.0.
>> > > > There
>> > > > > > are a few important features that we plan to release in 0.10.0
>> and
>> > I
>> > > > want
>> > > > > > to start this thread s.t. we can agree on what to include in
>> 0.10.0
>> > > > > > release.
>> > > > > >
>> > > > > > There are the following main features added in 0.10.0:
>> > > > > > - RocksDB TTL support
>> > > > > > - Add CoordinatorStream and disable CheckpointManager
>> > > > > > - Elasticsearch Producer
>> > > > > > - Host affinity
>> > > > > > And other 0.10.0 tickets:
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
>> > > > > >
>> > > > > > I propose to cut a 0.10.0 release after we get the following
>> issues
>> > > > > > resolved:
>> > > > > > - SAMZA-615: Migrate checkpoint from checkpoint topic to
>> > Coordinator
>> > > > > stream
>> > > > > > - SAMZA-617: YARN host affinity in Samza
>> > > > > >
>> > > > > > Thoughts?
>> > > > > >
>> > > > > > Thanks!
>> > > > > >
>> > > > > > -Yi
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Thanks and regards
>> >
>> > Chinmay Soman
>> >
>>
>>
>>
>> --
>> Navina R.
>>
>
>
Re: [DISCUSS] Release 0.10.0
Posted by Yi Pan <ni...@gmail.com>.
Hi, all,
Thanks a lot for helping out to select the features planned in 0.10.0.
Based on the above discussion, I am proposing to move the following tickets
later (i.e. 0.11.0).
30/32 tickets Status=open/reopen (exception for SAMZA-723: stream appender
deadlock issue) :
https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
Two tickets excluded:
- SAMZA-689, @Chinmay, were you able to re-produce it? I am thinking of
mark it as won't fix since 0.10.0 will remove the checkpoint topic
altogether.
- SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include the fix
that breaks Samza 0.10.0 build on Linux boxes
Anything that I missed?
Thanks!
-Yi
On Wed, Jul 29, 2015 at 9:57 AM, Navina Ramesh <nramesh@linkedin.com.invalid
> wrote:
> +1 for StreamAppender bug fix as a mandatory in 0.10.
>
> @Yan: will review SAMZA-676 today and test it out locally.
>
> Thanks!
> Navina
>
> On Wed, Jul 29, 2015 at 8:38 AM, Chinmay Soman <ch...@gmail.com>
> wrote:
>
> > I can take care of SAMZA-340, SAMZA-683 and will follow up with Luis for
> > SAMZA-401,2,3,4
> >
> > On Wed, Jul 29, 2015 at 12:10 AM, Dan <da...@gmail.com> wrote:
> >
> > > I agree SAMZA-741 for the ElasticSearch producer should be in too so
> > we've
> > > got a better API as part of that release.
> > >
> > > - Dan
> > >
> > >
> > > On 29 July 2015 at 07:51, Yan Fang <ya...@gmail.com> wrote:
> > >
> > > > Actually, I also want to include a few patch-available features,
> > > > especially:
> > > >
> > > > 1. broadcast stream (SAMZA-676)
> > > > - waiting for review
> > > >
> > > > 2. graphite support (SAMZA-340)
> > > > 3. meter and histogram (SAMZA-683)
> > > > 4. utility (SAMZA-401)
> > > > - 2,3,4 belong to Luis, if he does not have time to update, since
> > > they
> > > > only need some small changes, we can edit it and get +1 from another
> > > > committer.
> > > >
> > > > 5. hdfs producer (SAMZA-693)
> > > > - I am reviewing.
> > > >
> > > > 6. upgrade yarn to 2.7.1 (SAMZA-563)
> > > > - though I am reviewing, this ticket is negotiable if we want to
> put
> > > > into the 0.10.0 release. If we do not, I think, when users enable the
> > > > worker-persisting and container-persisting features, Samza will not
> be
> > > able
> > > > to handle it. (Some classes are only available after yarn 2.5.0 while
> > > Samza
> > > > currently only support yarn 2.4.0)
> > > >
> > > > 7. others: scrooge, class loader isolation, etc.
> > > > - those are waiting for reviewing too.
> > > >
> > > > My opinion is that, if we can clean up all the patch-available
> tickets,
> > > it
> > > > will be great. Most of them have been already reviewed more than
> once.
> > > So I
> > > > think it should not be very time-consuming to have them in the 0.10.0
> > > > release.
> > > >
> > > > What do you think?
> > > >
> > > > Of course, another must-have is the bug-fix of the Stream Appender.
> :)
> > > >
> > > > Thanks,
> > > >
> > > > Fang, Yan
> > > > yanfang724@gmail.com
> > > >
> > > > On Tue, Jul 28, 2015 at 10:27 PM, Roger Hoover <
> roger.hoover@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Thanks, Yi.
> > > > >
> > > > > I propose that we also include SAMZA-741 for Elasticsearch
> versioning
> > > > > support with the new ES producer. I think it's very close to being
> > > > merged.
> > > > >
> > > > > Roger
> > > > >
> > > > >
> > > > > On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan <ni...@gmail.com>
> > wrote:
> > > > >
> > > > > > Hi, all,
> > > > > >
> > > > > > I want to start the discussion on the release schedule for
> 0.10.0.
> > > > There
> > > > > > are a few important features that we plan to release in 0.10.0
> and
> > I
> > > > want
> > > > > > to start this thread s.t. we can agree on what to include in
> 0.10.0
> > > > > > release.
> > > > > >
> > > > > > There are the following main features added in 0.10.0:
> > > > > > - RocksDB TTL support
> > > > > > - Add CoordinatorStream and disable CheckpointManager
> > > > > > - Elasticsearch Producer
> > > > > > - Host affinity
> > > > > > And other 0.10.0 tickets:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
> > > > > >
> > > > > > I propose to cut a 0.10.0 release after we get the following
> issues
> > > > > > resolved:
> > > > > > - SAMZA-615: Migrate checkpoint from checkpoint topic to
> > Coordinator
> > > > > stream
> > > > > > - SAMZA-617: YARN host affinity in Samza
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > -Yi
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Thanks and regards
> >
> > Chinmay Soman
> >
>
>
>
> --
> Navina R.
>
Re: [DISCUSS] Release 0.10.0
Posted by Navina Ramesh <nr...@linkedin.com.INVALID>.
+1 for StreamAppender bug fix as a mandatory in 0.10.
@Yan: will review SAMZA-676 today and test it out locally.
Thanks!
Navina
On Wed, Jul 29, 2015 at 8:38 AM, Chinmay Soman <ch...@gmail.com>
wrote:
> I can take care of SAMZA-340, SAMZA-683 and will follow up with Luis for
> SAMZA-401,2,3,4
>
> On Wed, Jul 29, 2015 at 12:10 AM, Dan <da...@gmail.com> wrote:
>
> > I agree SAMZA-741 for the ElasticSearch producer should be in too so
> we've
> > got a better API as part of that release.
> >
> > - Dan
> >
> >
> > On 29 July 2015 at 07:51, Yan Fang <ya...@gmail.com> wrote:
> >
> > > Actually, I also want to include a few patch-available features,
> > > especially:
> > >
> > > 1. broadcast stream (SAMZA-676)
> > > - waiting for review
> > >
> > > 2. graphite support (SAMZA-340)
> > > 3. meter and histogram (SAMZA-683)
> > > 4. utility (SAMZA-401)
> > > - 2,3,4 belong to Luis, if he does not have time to update, since
> > they
> > > only need some small changes, we can edit it and get +1 from another
> > > committer.
> > >
> > > 5. hdfs producer (SAMZA-693)
> > > - I am reviewing.
> > >
> > > 6. upgrade yarn to 2.7.1 (SAMZA-563)
> > > - though I am reviewing, this ticket is negotiable if we want to put
> > > into the 0.10.0 release. If we do not, I think, when users enable the
> > > worker-persisting and container-persisting features, Samza will not be
> > able
> > > to handle it. (Some classes are only available after yarn 2.5.0 while
> > Samza
> > > currently only support yarn 2.4.0)
> > >
> > > 7. others: scrooge, class loader isolation, etc.
> > > - those are waiting for reviewing too.
> > >
> > > My opinion is that, if we can clean up all the patch-available tickets,
> > it
> > > will be great. Most of them have been already reviewed more than once.
> > So I
> > > think it should not be very time-consuming to have them in the 0.10.0
> > > release.
> > >
> > > What do you think?
> > >
> > > Of course, another must-have is the bug-fix of the Stream Appender. :)
> > >
> > > Thanks,
> > >
> > > Fang, Yan
> > > yanfang724@gmail.com
> > >
> > > On Tue, Jul 28, 2015 at 10:27 PM, Roger Hoover <roger.hoover@gmail.com
> >
> > > wrote:
> > >
> > > > Thanks, Yi.
> > > >
> > > > I propose that we also include SAMZA-741 for Elasticsearch versioning
> > > > support with the new ES producer. I think it's very close to being
> > > merged.
> > > >
> > > > Roger
> > > >
> > > >
> > > > On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan <ni...@gmail.com>
> wrote:
> > > >
> > > > > Hi, all,
> > > > >
> > > > > I want to start the discussion on the release schedule for 0.10.0.
> > > There
> > > > > are a few important features that we plan to release in 0.10.0 and
> I
> > > want
> > > > > to start this thread s.t. we can agree on what to include in 0.10.0
> > > > > release.
> > > > >
> > > > > There are the following main features added in 0.10.0:
> > > > > - RocksDB TTL support
> > > > > - Add CoordinatorStream and disable CheckpointManager
> > > > > - Elasticsearch Producer
> > > > > - Host affinity
> > > > > And other 0.10.0 tickets:
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
> > > > >
> > > > > I propose to cut a 0.10.0 release after we get the following issues
> > > > > resolved:
> > > > > - SAMZA-615: Migrate checkpoint from checkpoint topic to
> Coordinator
> > > > stream
> > > > > - SAMZA-617: YARN host affinity in Samza
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > Thanks!
> > > > >
> > > > > -Yi
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Thanks and regards
>
> Chinmay Soman
>
--
Navina R.
Re: [DISCUSS] Release 0.10.0
Posted by Chinmay Soman <ch...@gmail.com>.
I can take care of SAMZA-340, SAMZA-683 and will follow up with Luis for
SAMZA-401,2,3,4
On Wed, Jul 29, 2015 at 12:10 AM, Dan <da...@gmail.com> wrote:
> I agree SAMZA-741 for the ElasticSearch producer should be in too so we've
> got a better API as part of that release.
>
> - Dan
>
>
> On 29 July 2015 at 07:51, Yan Fang <ya...@gmail.com> wrote:
>
> > Actually, I also want to include a few patch-available features,
> > especially:
> >
> > 1. broadcast stream (SAMZA-676)
> > - waiting for review
> >
> > 2. graphite support (SAMZA-340)
> > 3. meter and histogram (SAMZA-683)
> > 4. utility (SAMZA-401)
> > - 2,3,4 belong to Luis, if he does not have time to update, since
> they
> > only need some small changes, we can edit it and get +1 from another
> > committer.
> >
> > 5. hdfs producer (SAMZA-693)
> > - I am reviewing.
> >
> > 6. upgrade yarn to 2.7.1 (SAMZA-563)
> > - though I am reviewing, this ticket is negotiable if we want to put
> > into the 0.10.0 release. If we do not, I think, when users enable the
> > worker-persisting and container-persisting features, Samza will not be
> able
> > to handle it. (Some classes are only available after yarn 2.5.0 while
> Samza
> > currently only support yarn 2.4.0)
> >
> > 7. others: scrooge, class loader isolation, etc.
> > - those are waiting for reviewing too.
> >
> > My opinion is that, if we can clean up all the patch-available tickets,
> it
> > will be great. Most of them have been already reviewed more than once.
> So I
> > think it should not be very time-consuming to have them in the 0.10.0
> > release.
> >
> > What do you think?
> >
> > Of course, another must-have is the bug-fix of the Stream Appender. :)
> >
> > Thanks,
> >
> > Fang, Yan
> > yanfang724@gmail.com
> >
> > On Tue, Jul 28, 2015 at 10:27 PM, Roger Hoover <ro...@gmail.com>
> > wrote:
> >
> > > Thanks, Yi.
> > >
> > > I propose that we also include SAMZA-741 for Elasticsearch versioning
> > > support with the new ES producer. I think it's very close to being
> > merged.
> > >
> > > Roger
> > >
> > >
> > > On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan <ni...@gmail.com> wrote:
> > >
> > > > Hi, all,
> > > >
> > > > I want to start the discussion on the release schedule for 0.10.0.
> > There
> > > > are a few important features that we plan to release in 0.10.0 and I
> > want
> > > > to start this thread s.t. we can agree on what to include in 0.10.0
> > > > release.
> > > >
> > > > There are the following main features added in 0.10.0:
> > > > - RocksDB TTL support
> > > > - Add CoordinatorStream and disable CheckpointManager
> > > > - Elasticsearch Producer
> > > > - Host affinity
> > > > And other 0.10.0 tickets:
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
> > > >
> > > > I propose to cut a 0.10.0 release after we get the following issues
> > > > resolved:
> > > > - SAMZA-615: Migrate checkpoint from checkpoint topic to Coordinator
> > > stream
> > > > - SAMZA-617: YARN host affinity in Samza
> > > >
> > > > Thoughts?
> > > >
> > > > Thanks!
> > > >
> > > > -Yi
> > > >
> > >
> >
>
--
Thanks and regards
Chinmay Soman
Re: [DISCUSS] Release 0.10.0
Posted by Dan <da...@gmail.com>.
I agree SAMZA-741 for the ElasticSearch producer should be in too so we've
got a better API as part of that release.
- Dan
On 29 July 2015 at 07:51, Yan Fang <ya...@gmail.com> wrote:
> Actually, I also want to include a few patch-available features,
> especially:
>
> 1. broadcast stream (SAMZA-676)
> - waiting for review
>
> 2. graphite support (SAMZA-340)
> 3. meter and histogram (SAMZA-683)
> 4. utility (SAMZA-401)
> - 2,3,4 belong to Luis, if he does not have time to update, since they
> only need some small changes, we can edit it and get +1 from another
> committer.
>
> 5. hdfs producer (SAMZA-693)
> - I am reviewing.
>
> 6. upgrade yarn to 2.7.1 (SAMZA-563)
> - though I am reviewing, this ticket is negotiable if we want to put
> into the 0.10.0 release. If we do not, I think, when users enable the
> worker-persisting and container-persisting features, Samza will not be able
> to handle it. (Some classes are only available after yarn 2.5.0 while Samza
> currently only support yarn 2.4.0)
>
> 7. others: scrooge, class loader isolation, etc.
> - those are waiting for reviewing too.
>
> My opinion is that, if we can clean up all the patch-available tickets, it
> will be great. Most of them have been already reviewed more than once. So I
> think it should not be very time-consuming to have them in the 0.10.0
> release.
>
> What do you think?
>
> Of course, another must-have is the bug-fix of the Stream Appender. :)
>
> Thanks,
>
> Fang, Yan
> yanfang724@gmail.com
>
> On Tue, Jul 28, 2015 at 10:27 PM, Roger Hoover <ro...@gmail.com>
> wrote:
>
> > Thanks, Yi.
> >
> > I propose that we also include SAMZA-741 for Elasticsearch versioning
> > support with the new ES producer. I think it's very close to being
> merged.
> >
> > Roger
> >
> >
> > On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan <ni...@gmail.com> wrote:
> >
> > > Hi, all,
> > >
> > > I want to start the discussion on the release schedule for 0.10.0.
> There
> > > are a few important features that we plan to release in 0.10.0 and I
> want
> > > to start this thread s.t. we can agree on what to include in 0.10.0
> > > release.
> > >
> > > There are the following main features added in 0.10.0:
> > > - RocksDB TTL support
> > > - Add CoordinatorStream and disable CheckpointManager
> > > - Elasticsearch Producer
> > > - Host affinity
> > > And other 0.10.0 tickets:
> > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
> > >
> > > I propose to cut a 0.10.0 release after we get the following issues
> > > resolved:
> > > - SAMZA-615: Migrate checkpoint from checkpoint topic to Coordinator
> > stream
> > > - SAMZA-617: YARN host affinity in Samza
> > >
> > > Thoughts?
> > >
> > > Thanks!
> > >
> > > -Yi
> > >
> >
>
Re: [DISCUSS] Release 0.10.0
Posted by Yan Fang <ya...@gmail.com>.
Actually, I also want to include a few patch-available features, especially:
1. broadcast stream (SAMZA-676)
- waiting for review
2. graphite support (SAMZA-340)
3. meter and histogram (SAMZA-683)
4. utility (SAMZA-401)
- 2,3,4 belong to Luis, if he does not have time to update, since they
only need some small changes, we can edit it and get +1 from another
committer.
5. hdfs producer (SAMZA-693)
- I am reviewing.
6. upgrade yarn to 2.7.1 (SAMZA-563)
- though I am reviewing, this ticket is negotiable if we want to put
into the 0.10.0 release. If we do not, I think, when users enable the
worker-persisting and container-persisting features, Samza will not be able
to handle it. (Some classes are only available after yarn 2.5.0 while Samza
currently only support yarn 2.4.0)
7. others: scrooge, class loader isolation, etc.
- those are waiting for reviewing too.
My opinion is that, if we can clean up all the patch-available tickets, it
will be great. Most of them have been already reviewed more than once. So I
think it should not be very time-consuming to have them in the 0.10.0
release.
What do you think?
Of course, another must-have is the bug-fix of the Stream Appender. :)
Thanks,
Fang, Yan
yanfang724@gmail.com
On Tue, Jul 28, 2015 at 10:27 PM, Roger Hoover <ro...@gmail.com>
wrote:
> Thanks, Yi.
>
> I propose that we also include SAMZA-741 for Elasticsearch versioning
> support with the new ES producer. I think it's very close to being merged.
>
> Roger
>
>
> On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan <ni...@gmail.com> wrote:
>
> > Hi, all,
> >
> > I want to start the discussion on the release schedule for 0.10.0. There
> > are a few important features that we plan to release in 0.10.0 and I want
> > to start this thread s.t. we can agree on what to include in 0.10.0
> > release.
> >
> > There are the following main features added in 0.10.0:
> > - RocksDB TTL support
> > - Add CoordinatorStream and disable CheckpointManager
> > - Elasticsearch Producer
> > - Host affinity
> > And other 0.10.0 tickets:
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
> >
> > I propose to cut a 0.10.0 release after we get the following issues
> > resolved:
> > - SAMZA-615: Migrate checkpoint from checkpoint topic to Coordinator
> stream
> > - SAMZA-617: YARN host affinity in Samza
> >
> > Thoughts?
> >
> > Thanks!
> >
> > -Yi
> >
>
Re: [DISCUSS] Release 0.10.0
Posted by Roger Hoover <ro...@gmail.com>.
Thanks, Yi.
I propose that we also include SAMZA-741 for Elasticsearch versioning
support with the new ES producer. I think it's very close to being merged.
Roger
On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan <ni...@gmail.com> wrote:
> Hi, all,
>
> I want to start the discussion on the release schedule for 0.10.0. There
> are a few important features that we plan to release in 0.10.0 and I want
> to start this thread s.t. we can agree on what to include in 0.10.0
> release.
>
> There are the following main features added in 0.10.0:
> - RocksDB TTL support
> - Add CoordinatorStream and disable CheckpointManager
> - Elasticsearch Producer
> - Host affinity
> And other 0.10.0 tickets:
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0
>
> I propose to cut a 0.10.0 release after we get the following issues
> resolved:
> - SAMZA-615: Migrate checkpoint from checkpoint topic to Coordinator stream
> - SAMZA-617: YARN host affinity in Samza
>
> Thoughts?
>
> Thanks!
>
> -Yi
>