You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@storm.apache.org by Jungtaek Lim <ka...@gmail.com> on 2017/01/05 04:43:13 UTC

Re: [DISCUSS] Release Storm 1.1.0

Recently we receive some requests regarding release Storm 1.0.3, so would
like to bump this again.

Given that blocker issues for Storm 1.1.0 are also blocker for 1.0.3, I'd
like to ask a favor of taking care of 'open' / 'in progress' issues on
1.1.0 epic.
https://issues.apache.org/jira/browse/STORM-1856

There're one 'open' issue and three 'in progress' issues. Two of three 'in
progress' issues are tiny fix so easy to be handled, so actual blockers are
STORM-2176 <https://issues.apache.org/jira/browse/STORM-2176> and STORM-2228
<https://issues.apache.org/jira/browse/STORM-2228>.

Thanks,
Jungtaek Lim (HeartSaVioR)

2016년 11월 17일 (목) 오후 11:41, Satish Duggana <sa...@gmail.com>님이 작성:

>     STORM-2205: Race condition in getting nimbus summaries while ZK
> connections are reconnected.
>
> This issue seems to occur in our environments and I would like this to be
> part of 1.1.0.
>
> Thanks,
> Satish.
>
> On Thu, Nov 17, 2016 at 9:36 AM, Jungtaek Lim <ka...@gmail.com> wrote:
>
> > I have no idea on storm-kafka-client, but some bugfix issues for
> > storm-kafka-client are waiting for reviewing / merging.
> >
> > STORM-2014 <https://issues.apache.org/jira/browse/STORM-2014>
> > STORM-2087 <https://issues.apache.org/jira/browse/STORM-2087>
> > STORM-2104 <https://issues.apache.org/jira/browse/STORM-2104>
> >
> > If someone can review them in several days it would be great.
> >
> > I hope that we include currently opened pull requests for Storm SQL so
> that
> > we can release 'usable Storm SQL' more usable, but I'm also OK to
> postpone
> > them to be included to next release if they drag the release.
> >
> > STORM-1446 <http://issues.apache.org/jira/browse/STORM-1446>
> > STORM-1443 <https://issues.apache.org/jira/browse/STORM-1443>
> > STORM-2148 <https://issues.apache.org/jira/browse/STORM-2148>
> > STORM-2170 <https://issues.apache.org/jira/browse/STORM-2170>
> >
> > I can see some pull requests which address Trident implementations for
> > storm-kafka-client, storm-mongodb, storm-cassandra.
> >
> > storm-kafka-client: STORM-1694
> > <https://issues.apache.org/jira/browse/STORM-1694> (patch for 2.0 is
> > merged, patch for 1.x is ready for reviewing)
> > storm-cassandra: STORM-1369
> > <https://issues.apache.org/jira/browse/STORM-1369>
> > storm-mongodb: STORM-1607 <https://issues.apache.org/
> > jira/browse/STORM-1607>
> >
> > If we want to cut the release now, we could include only bugfix issues
> and
> > postpone others. Otherwise we could discuss and include some or all of
> the
> > above.
> >
> > What do you think? When we want to start the release process for 1.1.0?
> >
> > - Jungtaek Lim (HeartSaVioR)
> >
> > 2016년 11월 16일 (수) 오전 4:11, P. Taylor Goetz <pt...@gmail.com>님이 작성:
> >
> > Thanks Xin, I added it to the 1.1.0 epic.
> >
> > -Taylor
> >
> > > On Nov 15, 2016, at 9:01 AM, Xin Wang <da...@gmail.com> wrote:
> > >
> > > STORM-2198 ( PR: https://github.com/apache/storm/pull/1773 ) fixes a
> bug
> > of
> > > storm-hdfs. Do we have a consideration to include this?
> > >
> > > Thanks,
> > > Xin Wang (vesense)
> > >
> > > 2016-11-15 10:03 GMT+08:00 Jungtaek Lim <ka...@gmail.com>:
> > >
> > >> Some issues on Storm SQL are resolved but not documented yet. I'll
> file
> > an
> > >> issue and assign to 1.1.0 release epic.
> > >> And also I want to address dropping aggregation and join on Storm SQL
> > >> Trident mode before releasing. I'll assign it too.
> > >>
> > >> - Jungtaek Lim (HeartSaVioR)
> > >>
> > >>
> > >> 2016년 11월 15일 (화) 오전 5:55, P. Taylor Goetz <pt...@gmail.com>님이 작성:
> > >>
> > >>> I think we’re very close. I would like to confirm that the 1.x-branch
> > is
> > >>> not affected by STORM-2176.
> > >>>
> > >>> The worker lifecycle API was added in 1.0, but doesn’t work in any
> > >>> released version due to STORM-2176.
> > >>>
> > >>> If there are any other open JIRAs that anyone is passionate about,
> now
> > >>> would be a good time to assign them to the 1.1.0 release epic
> > >> (STORM-1856).
> > >>>
> > >>> -Taylor
> > >>>
> > >>>
> > >>>
> > >>>> On Oct 27, 2016, at 12:19 PM, Jungtaek Lim <ka...@gmail.com>
> wrote:
> > >>>>
> > >>>> Finally Pacemaker H/A, Supervisor V2, and Storm SQL PRs which were
> > >> opened
> > >>>> at the last mail (4 weeks ago) are all merged to 1.x branch.
> > >>>>
> > >>>> There're some more PRs on Storm SQL opened, but given that we can
> > >> release
> > >>>> new minor at any time when we feel it's enough change, I can wait
> for
> > >> it.
> > >>>> They didn't get reviewed yet indeed.
> > >>>>
> > >>>> Is there something else we would want to include it to 1.1.0?
> > >>>>
> > >>>> Thanks,
> > >>>> Jungtaek Lim (HeartSaVioR)
> > >>>>
> > >>>> 2016년 10월 1일 (토) 오전 9:30, Jungtaek Lim <ka...@gmail.com>님이 작성:
> > >>>>
> > >>>>> Personally, merging and porting back to three branches are painful
> > >>> enough,
> > >>>>> especially we don't have merging script and having verbose process
> (I
> > >>> mean
> > >>>>> CHANGELOG).
> > >>>>> It would be better if merging process is automated (by running
> script
> > >> or
> > >>>>> so), so I'd +1 to revisit Harsha's suggestion (adopting Kafka merge
> > >>> script)
> > >>>>> and modify script to fit to Storm.
> > >>>>> (It will not work if it's the case we need to handle PRs for each
> > >>> version
> > >>>>> line, since 'Close' in commit log doesn't close the PR if its
> target
> > >>> branch
> > >>>>> is not master.)
> > >>>>>
> > >>>>> Anyway, without automation I don't want to maintain more version
> > >> lines.
> > >>>>> I'm looking at the announces from other projects, and others are
> only
> > >>>>> maintaining two version lines.
> > >>>>> Since we maintain 2.0.0 version line we can't reduce version lines
> to
> > >> 2,
> > >>>>> but hopefully at most 3.
> > >>>>>
> > >>>>> Btw, let's check pending pull requests and enumerate which can be
> > >>> included
> > >>>>> in 1.0.0, and start/finish review and merge them soon.
> > >>>>> For me Supervisor V2 and Pacemaker H/A, and pending Storm SQL PRs
> can
> > >> be
> > >>>>> included, since they are small or in reviewing and expected to pass
> > >>> review
> > >>>>> phase soon.
> > >>>>> (And some small PRs. There're other valuable PRs in PR list but I'm
> > >> not
> > >>>>> sure we can review them soon. One example is unified API.)
> > >>>>>
> > >>>>> One issue which is not clear is STORM-2006
> > >>>>> <https://github.com/apache/storm/pull/1595>. This is a candidate
> for
> > >>> me,
> > >>>>> but gets blocked while reviewing. If we plan to put great effort to
> > >>> revise
> > >>>>> Metric we can skip this.
> > >>>>>
> > >>>>> Please enumerate other PRs as well if you want to include in 1.1.0.
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Jungtaek Lim
> > >>>>>
> > >>>>> 2016년 9월 30일 (금) 오후 11:09, Bobby Evans <evans@yahoo-inc.com.invalid
> >
> > >> 님이
> > >>> 작성:
> > >>>>>
> > >>>>> Sounds good to me.  It would be nice to get some of the new
> features
> > >>> out.
> > >>>>> Do we expect to maintain both 1.0.x and 1.1.x lines with bug fixes?
> > >>> And if
> > >>>>> so for how long do we want to do this for? - Bobby
> > >>>>>
> > >>>>>   On Thursday, September 29, 2016 7:35 PM, Jungtaek Lim <
> > >>>>> kabhwan@gmail.com> wrote:
> > >>>>>
> > >>>>>
> > >>>>> Hi devs,
> > >>>>>
> > >>>>> It's been 5 months after releasing Storm 1.0.0, and now 1.x branch
> > has
> > >>> lots
> > >>>>> of CHANGELOG and also pending reviews.
> > >>>>> It's also been a long time after 1.1.0 RC1 is canceled.
> > >>>>>
> > >>>>> I think it may be good to put some efforts to review and merge
> > pending
> > >>> pull
> > >>>>> requests (except things which takes time to review and test), and
> > >>> release
> > >>>>> 1.1.0 soon.
> > >>>>>
> > >>>>> What do you think?
> > >>>>>
> > >>>>> I'm also open to volunteer release manager for 1.1.0 after we
> > document
> > >>> the
> > >>>>> process of official release.
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Jungtaek Lim (HeartSaVioR)
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>
> > >>>
> > >>
> >
>

Re: [DISCUSS] Release Storm 1.1.0

Posted by Jungtaek Lim <ka...@gmail.com>.

Apache Zeppelin provides 700 MB of full set and 272 MB of netinst via
Apache download page so there should be ways to do it, but for me 200MB is
already too huge.

I have been wondering why we package external connectors, which other
projects don't.

When we didn't make uber jar for connectors from previous versions, only
adding jar shouldn't work which makes packaged jars useless.
Since we start to make uber jar for some connectors, and 1.1.0 will
introduce "adding jars at the topology submission step" so it's not
useless, but still inconvenient way to use. Moreover, in this case we can
still use new feature "adding maven artifacts at the topology submission
step".
It's only useful for who has the node for submission which blocks Maven
central or even internet.

If we want to have a discussion about this, I'll initiate separate
discussion thread.

- Jungtaek Lim (HeartSaVioR)

2017년 2월 17일 (금) 오전 6:00, P. Taylor Goetz <pt...@gmail.com>님이 작성:

> I have a release candidate for 1.1.0 built, but I can’t stage the
> convenience binaries because they are now over the ASF subversion size
> limit of 200MB.
>
> I’ll work with infra to figure out a solution.
>
> -Taylor
>
> > On Feb 16, 2017, at 9:46 AM, Jungtaek Lim <ka...@gmail.com> wrote:
> >
> > Now all issues on the Storm 1.1.0 epic got resolved. :)
> >
> > There're still some bugfix pull requests for storm-kafka-client, but
> > releasing Storm 1.1.0 has been tremendously dragged (I initiated this at
> > Sep. 2016) so unless they're critical, I'm +1 on starting release phase
> on
> > Storm 1.1.0 ASAP.
> >
> > - Jungtaek Lim
> >
> > 2017년 2월 14일 (화) 오전 11:41, Harsha Chintalapani <st...@harsha.io>님이 작성:
> >
> > STORM-2340 is more of a feature . Auto-commit mode in storm-kafka used
> > rarely and most users
> > run the kafka spout with ackers and get at-least once guarantee.  If its
> > going to longer to address the PR reviews
> > I am +1 on moving this out of Storm 1.1.0. We already quite a few patches
> > storm-kafka-client and 1.1.0 release brings in lot of improvements
> > and bug-fixes.
> > -Harsha
> >
> > On Wed, Feb 8, 2017 at 6:15 PM Jungtaek Lim <ka...@gmail.com> wrote:
> >
> >> There seems some pull requests for bugfix/improvement on
> >> storm-kafka-client, and some authors in PRs are not availble for now.
> >> (waiting 7 days)
> >>
> >> If we plan to get 1.1.1 out soon (say 1 month later or even closer) we
> can
> >> postpone, but if not, it might be better to coordinate these things ASAP
> >> and include to 1.1.0.
> >>
> >> There seems to be other small PRs, but nothing seems critical so it
> would
> >> be OK to not wait for merging.
> >>
> >> - Jungtaek Lim
> >>
> >> 2017년 2월 9일 (목) 오전 6:48, P. Taylor Goetz <pt...@gmail.com>님이 작성:
> >>
> >> Right now we’re down to 1 open issue on the 1.1.0 release epic:
> STORM-2250
> >> which is under active review/discussion.
> >>
> >> Assuming that is mergeable in the near future, are there any other open
> >> issues that should be considered for this release?
> >>
> >> -Taylor
> >>
> >>
> >>> On Feb 2, 2017, at 4:48 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
> >>>
> >>> Thanks for putting this list together Jungtaek. I added a few to the
> 1.1
> >> release epic that I think are important. Feel free to do the same.
> >>>
> >>> Looks like we have a few to go, but there are pull requests for them.
> >> It’s mostly just a matter of reviews and review responses, so I think we
> >> are close.
> >>>
> >>> -Taylor
> >>>
> >>>> On Feb 2, 2017, at 1:41 AM, Jungtaek Lim <ka...@gmail.com> wrote:
> >>>>
> >>>> Seems like there're not blockers for 1.1.0, but some pull requests are
> >>>> worth to check.
> >>>> There're pending pull requests for storm-kafka-client waited on
> >> STORM-2225.
> >>>> Given that STORM-2225 is now merged, we might need to take a look at.
> >>>>
> >>>> *- reviewing*
> >>>>
> >>>> [storm-core]
> >>>>
> >>>>> STORM-2324 : Fix deployment failure if resources directory is missing
> >> in
> >>>> topology jar
> >>>> (master) https://github.com/apache/storm/pull/1908
> >>>> (1.x) https://github.com/apache/storm/pull/1898
> >>>>
> >>>>> STORM-2321 Handle blobstore zk key deletion in KeySequenceNumber
> >>>> (master) https://github.com/apache/storm/pull/1904
> >>>> (1.x) https://github.com/apache/storm/pull/1905
> >>>>
> >>>> [storm-kafka]
> >>>>
> >>>>> STORM-2270 Kafka spout should consume from latest when ZK partition
> >>>> commit offset bigger than the latest offset
> >>>> (1.x) https://github.com/apache/storm/pull/1851
> >>>>
> >>>> [storm-kafka-client]
> >>>>
> >>>>> STORM-2281: Running Multiple Kafka Spouts (Trident) Throws Illegal
> >> State
> >>>> Exception
> >>>> (1.x) https://github.com/apache/storm/pull/1902
> >>>>
> >>>>> STORM-2315 Storm kafka client does not commit offsets when ack is
> >> disabled
> >>>> (1.x) https://github.com/apache/storm/pull/1891
> >>>>
> >>>>> fix: KafkaSpout is blocked in AutoCommitMode
> >>>> (master) https://github.com/apache/storm/pull/1863
> >>>>
> >>>>> STORM-2250: Kafka Spout Refactoring to Increase Modularity and
> >> Testability
> >>>> (master) https://github.com/apache/storm/pull/1832
> >>>>
> >>>>> STORM-2014: Put logic around dropping messages into RetryService,
> >> remove
> >>>> maxRetry setting from new KafkaSpout
> >>>> (master) https://github.com/apache/storm/pull/1605
> >>>>
> >>>>> fix NullPointException with acked.get(rtp)
> >>>> (master) https://github.com/apache/storm/pull/1807
> >>>>
> >>>> [storm-sql]
> >>>>
> >>>>> STORM-1443 [Storm SQL] Support customizing parallelism in StormSQL
> >>>> https://github.com/apache/storm/pull/1739
> >>>>
> >>>> *- pending*
> >>>>
> >>>> [storm-kafka-client]
> >>>>
> >>>>> STORM-2296 Kafka spout no dup on leader changes
> >>>> (1.0.x) https://github.com/apache/storm/pull/1873
> >>>> (1.x) https://github.com/apache/storm/pull/1888
> >>>>
> >>>> [storm-sql]
> >>>>
> >>>>> STORM-2148 [Storm SQL] Trident mode: back to code generate and
> compile
> >>>> Trident topology
> >>>> https://github.com/apache/storm/pull/1743
> >>>>
> >>>> Thanks,
> >>>> Jungtaek Lim (HeartSaVioR)
> >>>>
> >>>> 2017년 2월 2일 (목) 오전 8:14, Harsha Chintalapani <st...@harsha.io>님이 작성:
> >>>>
> >>>>> Trying to check the status on this release of 1.1.0. Are we going to
> > do
> >>>>> this release anytime soon?
> >>>>>
> >>>>>
> >>>>> On Fri, Jan 13, 2017 at 7:50 PM S G <sg...@gmail.com>
> wrote:
> >>>>>
> >>>>>> Not sure if its a little late to include for the 1.1.0 and 1.0.3
> >> releases
> >>>>>> now, but can we consider using zookeeper 3.4.9 for the future
> > versions
> >> as
> >>>>>> 3.4.9 brings in a lot of stability improvements (
> >>>>>> http://zookeeper.apache.org/releases.html) and storm is still using
> >>>>> 3.4.6
> >>>>>> (
> >>>>>> https://github.com/apache/storm/blob/master/pom.xml)
> >>>>>>
> >>>>>> On Fri, Jan 6, 2017 at 11:54 AM, P. Taylor Goetz <ptgoetz@gmail.com
> >
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Thanks for the update Jungtaek.
> >>>>>>>
> >>>>>>> I’m verifying the patches now. And they should be mergeable
> shortly.
> >>>>>>>
> >>>>>>> I think we’ll likely be ready for 1.1.0 and 1.0.3 releases next
> > week.
> >>>>>>>
> >>>>>>> -Taylor
> >>>>>>>
> >>>>>>>> On Jan 6, 2017, at 3:40 AM, Jungtaek Lim <ka...@gmail.com>
> wrote:
> >>>>>>>>
> >>>>>>>> I just submitted a patch for STORM-2176
> >>>>>>>> <https://issues.apache.org/jira/browse/STORM-2176>. Since it's a
> >>>>> small
> >>>>>>> fix,
> >>>>>>>> we just need to handle STORM-2228
> >>>>>>>> <https://issues.apache.org/jira/browse/STORM-2228> to release.
> >>>>>>>>
> >>>>>>>> - Jungtaek Lim (HeartSaVioR)
> >>>>>>>>
> >>>>>>>> 2017년 1월 5일 (목) 오후 1:43, Jungtaek Lim <ka...@gmail.com>님이 작성:
> >>>>>>>>
> >>>>>>>>> Recently we receive some requests regarding release Storm 1.0.3,
> > so
> >>>>>>> would
> >>>>>>>>> like to bump this again.
> >>>>>>>>>
> >>>>>>>>> Given that blocker issues for Storm 1.1.0 are also blocker for
> >>>>> 1.0.3,
> >>>>>>> I'd
> >>>>>>>>> like to ask a favor of taking care of 'open' / 'in progress'
> > issues
> >>>>> on
> >>>>>>>>> 1.1.0 epic.
> >>>>>>>>> https://issues.apache.org/jira/browse/STORM-1856
> >>>>>>>>>
> >>>>>>>>> There're one 'open' issue and three 'in progress' issues. Two of
> >>>>> three
> >>>>>>> 'in
> >>>>>>>>> progress' issues are tiny fix so easy to be handled, so actual
> >>>>>> blockers
> >>>>>>> are
> >>>>>>>>> STORM-2176 <https://issues.apache.org/jira/browse/STORM-2176>
> and
> >>>>>>>>> STORM-2228 <https://issues.apache.org/jira/browse/STORM-2228>.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Jungtaek Lim (HeartSaVioR)
> >>>>>>>>>
> >>>>>>>>> 2016년 11월 17일 (목) 오후 11:41, Satish Duggana <
> >>>>> satish.duggana@gmail.com
> >>>>>>> 님이
> >>>>>>>>> 작성:
> >>>>>>>>>
> >>>>>>>>> STORM-2205: Race condition in getting nimbus summaries while ZK
> >>>>>>>>> connections are reconnected.
> >>>>>>>>>
> >>>>>>>>> This issue seems to occur in our environments and I would like
> > this
> >>>>> to
> >>>>>>> be
> >>>>>>>>> part of 1.1.0.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Satish.
> >>>>>>>>>
> >>>>>>>>> On Thu, Nov 17, 2016 at 9:36 AM, Jungtaek Lim <kabhwan@gmail.com
> >
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> I have no idea on storm-kafka-client, but some bugfix issues for
> >>>>>>>>>> storm-kafka-client are waiting for reviewing / merging.
> >>>>>>>>>>
> >>>>>>>>>> STORM-2014 <https://issues.apache.org/jira/browse/STORM-2014>
> >>>>>>>>>> STORM-2087 <https://issues.apache.org/jira/browse/STORM-2087>
> >>>>>>>>>> STORM-2104 <https://issues.apache.org/jira/browse/STORM-2104>
> >>>>>>>>>>
> >>>>>>>>>> If someone can review them in several days it would be great.
> >>>>>>>>>>
> >>>>>>>>>> I hope that we include currently opened pull requests for Storm
> >> SQL
> >>>>>> so
> >>>>>>>>> that
> >>>>>>>>>> we can release 'usable Storm SQL' more usable, but I'm also OK
> to
> >>>>>>>>> postpone
> >>>>>>>>>> them to be included to next release if they drag the release.
> >>>>>>>>>>
> >>>>>>>>>> STORM-1446 <http://issues.apache.org/jira/browse/STORM-1446>
> >>>>>>>>>> STORM-1443 <https://issues.apache.org/jira/browse/STORM-1443>
> >>>>>>>>>> STORM-2148 <https://issues.apache.org/jira/browse/STORM-2148>
> >>>>>>>>>> STORM-2170 <https://issues.apache.org/jira/browse/STORM-2170>
> >>>>>>>>>>
> >>>>>>>>>> I can see some pull requests which address Trident
> > implementations
> >>>>>> for
> >>>>>>>>>> storm-kafka-client, storm-mongodb, storm-cassandra.
> >>>>>>>>>>
> >>>>>>>>>> storm-kafka-client: STORM-1694
> >>>>>>>>>> <https://issues.apache.org/jira/browse/STORM-1694> (patch for
> 2.0
> >>>>> is
> >>>>>>>>>> merged, patch for 1.x is ready for reviewing)
> >>>>>>>>>> storm-cassandra: STORM-1369
> >>>>>>>>>> <https://issues.apache.org/jira/browse/STORM-1369>
> >>>>>>>>>> storm-mongodb: STORM-1607 <https://issues.apache.org/
> >>>>>>>>>> jira/browse/STORM-1607>
> >>>>>>>>>>
> >>>>>>>>>> If we want to cut the release now, we could include only bugfix
> >>>>>> issues
> >>>>>>>>> and
> >>>>>>>>>> postpone others. Otherwise we could discuss and include some or
> >> all
> >>>>>> of
> >>>>>>>>> the
> >>>>>>>>>> above.
> >>>>>>>>>>
> >>>>>>>>>> What do you think? When we want to start the release process for
> >>>>>> 1.1.0?
> >>>>>>>>>>
> >>>>>>>>>> - Jungtaek Lim (HeartSaVioR)
> >>>>>>>>>>
> >>>>>>>>>> 2016년 11월 16일 (수) 오전 4:11, P. Taylor Goetz <ptgoetz@gmail.com
> >님이
> >>>>> 작성:
> >>>>>>>>>>
> >>>>>>>>>> Thanks Xin, I added it to the 1.1.0 epic.
> >>>>>>>>>>
> >>>>>>>>>> -Taylor
> >>>>>>>>>>
> >>>>>>>>>>> On Nov 15, 2016, at 9:01 AM, Xin Wang <da...@gmail.com>
> >>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> STORM-2198 ( PR: https://github.com/apache/storm/pull/1773 )
> >>>>> fixes
> >>>>>> a
> >>>>>>>>> bug
> >>>>>>>>>> of
> >>>>>>>>>>> storm-hdfs. Do we have a consideration to include this?
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Xin Wang (vesense)
> >>>>>>>>>>>
> >>>>>>>>>>> 2016-11-15 10:03 GMT+08:00 Jungtaek Lim <ka...@gmail.com>:
> >>>>>>>>>>>
> >>>>>>>>>>>> Some issues on Storm SQL are resolved but not documented yet.
> >>>>> I'll
> >>>>>>>>> file
> >>>>>>>>>> an
> >>>>>>>>>>>> issue and assign to 1.1.0 release epic.
> >>>>>>>>>>>> And also I want to address dropping aggregation and join on
> >> Storm
> >>>>>> SQL
> >>>>>>>>>>>> Trident mode before releasing. I'll assign it too.
> >>>>>>>>>>>>
> >>>>>>>>>>>> - Jungtaek Lim (HeartSaVioR)
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> 2016년 11월 15일 (화) 오전 5:55, P. Taylor Goetz <ptgoetz@gmail.com
> >>> 님이
> >>>>>> 작성:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> I think we’re very close. I would like to confirm that the
> >>>>>>> 1.x-branch
> >>>>>>>>>> is
> >>>>>>>>>>>>> not affected by STORM-2176.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The worker lifecycle API was added in 1.0, but doesn’t work
> in
> >>>>> any
> >>>>>>>>>>>>> released version due to STORM-2176.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> If there are any other open JIRAs that anyone is passionate
> >>>>> about,
> >>>>>>>>> now
> >>>>>>>>>>>>> would be a good time to assign them to the 1.1.0 release epic
> >>>>>>>>>>>> (STORM-1856).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> -Taylor
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Oct 27, 2016, at 12:19 PM, Jungtaek Lim <
> kabhwan@gmail.com
> >>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Finally Pacemaker H/A, Supervisor V2, and Storm SQL PRs
> which
> >>>>>> were
> >>>>>>>>>>>> opened
> >>>>>>>>>>>>>> at the last mail (4 weeks ago) are all merged to 1.x branch.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> There're some more PRs on Storm SQL opened, but given that
> we
> >>>>> can
> >>>>>>>>>>>> release
> >>>>>>>>>>>>>> new minor at any time when we feel it's enough change, I can
> >>>>> wait
> >>>>>>>>> for
> >>>>>>>>>>>> it.
> >>>>>>>>>>>>>> They didn't get reviewed yet indeed.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Is there something else we would want to include it to
> 1.1.0?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 2016년 10월 1일 (토) 오전 9:30, Jungtaek Lim <kabhwan@gmail.com
> >님이
> >>>>> 작성:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Personally, merging and porting back to three branches are
> >>>>>> painful
> >>>>>>>>>>>>> enough,
> >>>>>>>>>>>>>>> especially we don't have merging script and having verbose
> >>>>>> process
> >>>>>>>>> (I
> >>>>>>>>>>>>> mean
> >>>>>>>>>>>>>>> CHANGELOG).
> >>>>>>>>>>>>>>> It would be better if merging process is automated (by
> >> running
> >>>>>>>>> script
> >>>>>>>>>>>> or
> >>>>>>>>>>>>>>> so), so I'd +1 to revisit Harsha's suggestion (adopting
> > Kafka
> >>>>>>> merge
> >>>>>>>>>>>>> script)
> >>>>>>>>>>>>>>> and modify script to fit to Storm.
> >>>>>>>>>>>>>>> (It will not work if it's the case we need to handle PRs
> for
> >>>>>> each
> >>>>>>>>>>>>> version
> >>>>>>>>>>>>>>> line, since 'Close' in commit log doesn't close the PR if
> > its
> >>>>>>>>> target
> >>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>> is not master.)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Anyway, without automation I don't want to maintain more
> >>>>> version
> >>>>>>>>>>>> lines.
> >>>>>>>>>>>>>>> I'm looking at the announces from other projects, and
> others
> >>>>> are
> >>>>>>>>> only
> >>>>>>>>>>>>>>> maintaining two version lines.
> >>>>>>>>>>>>>>> Since we maintain 2.0.0 version line we can't reduce
> version
> >>>>>> lines
> >>>>>>>>> to
> >>>>>>>>>>>> 2,
> >>>>>>>>>>>>>>> but hopefully at most 3.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Btw, let's check pending pull requests and enumerate which
> >> can
> >>>>>> be
> >>>>>>>>>>>>> included
> >>>>>>>>>>>>>>> in 1.0.0, and start/finish review and merge them soon.
> >>>>>>>>>>>>>>> For me Supervisor V2 and Pacemaker H/A, and pending Storm
> > SQL
> >>>>>> PRs
> >>>>>>>>> can
> >>>>>>>>>>>> be
> >>>>>>>>>>>>>>> included, since they are small or in reviewing and expected
> >> to
> >>>>>>> pass
> >>>>>>>>>>>>> review
> >>>>>>>>>>>>>>> phase soon.
> >>>>>>>>>>>>>>> (And some small PRs. There're other valuable PRs in PR list
> >>>>> but
> >>>>>>> I'm
> >>>>>>>>>>>> not
> >>>>>>>>>>>>>>> sure we can review them soon. One example is unified API.)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> One issue which is not clear is STORM-2006
> >>>>>>>>>>>>>>> <https://github.com/apache/storm/pull/1595>. This is a
> >>>>>> candidate
> >>>>>>>>> for
> >>>>>>>>>>>>> me,
> >>>>>>>>>>>>>>> but gets blocked while reviewing. If we plan to put great
> >>>>> effort
> >>>>>>> to
> >>>>>>>>>>>>> revise
> >>>>>>>>>>>>>>> Metric we can skip this.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Please enumerate other PRs as well if you want to include
> in
> >>>>>>> 1.1.0.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> Jungtaek Lim
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 2016년 9월 30일 (금) 오후 11:09, Bobby Evans
> >>>>>>> <evans@yahoo-inc.com.invalid
> >>>>>>>>>>
> >>>>>>>>>>>> 님이
> >>>>>>>>>>>>> 작성:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Sounds good to me.  It would be nice to get some of the new
> >>>>>>>>> features
> >>>>>>>>>>>>> out.
> >>>>>>>>>>>>>>> Do we expect to maintain both 1.0.x and 1.1.x lines with
> bug
> >>>>>>> fixes?
> >>>>>>>>>>>>> And if
> >>>>>>>>>>>>>>> so for how long do we want to do this for? - Bobby
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thursday, September 29, 2016 7:35 PM, Jungtaek Lim <
> >>>>>>>>>>>>>>> kabhwan@gmail.com> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi devs,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> It's been 5 months after releasing Storm 1.0.0, and now 1.x
> >>>>>> branch
> >>>>>>>>>> has
> >>>>>>>>>>>>> lots
> >>>>>>>>>>>>>>> of CHANGELOG and also pending reviews.
> >>>>>>>>>>>>>>> It's also been a long time after 1.1.0 RC1 is canceled.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I think it may be good to put some efforts to review and
> >> merge
> >>>>>>>>>> pending
> >>>>>>>>>>>>> pull
> >>>>>>>>>>>>>>> requests (except things which takes time to review and
> > test),
> >>>>>> and
> >>>>>>>>>>>>> release
> >>>>>>>>>>>>>>> 1.1.0 soon.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I'm also open to volunteer release manager for 1.1.0 after
> > we
> >>>>>>>>>> document
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> process of official release.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>
>
>

Re: [DISCUSS] Release Storm 1.1.0

Posted by "P. Taylor Goetz" <pt...@gmail.com>.

I have a release candidate for 1.1.0 built, but I can’t stage the convenience binaries because they are now over the ASF subversion size limit of 200MB.

I’ll work with infra to figure out a solution.

-Taylor

> On Feb 16, 2017, at 9:46 AM, Jungtaek Lim <ka...@gmail.com> wrote:
> 
> Now all issues on the Storm 1.1.0 epic got resolved. :)
> 
> There're still some bugfix pull requests for storm-kafka-client, but
> releasing Storm 1.1.0 has been tremendously dragged (I initiated this at
> Sep. 2016) so unless they're critical, I'm +1 on starting release phase on
> Storm 1.1.0 ASAP.
> 
> - Jungtaek Lim
> 
> 2017년 2월 14일 (화) 오전 11:41, Harsha Chintalapani <st...@harsha.io>님이 작성:
> 
> STORM-2340 is more of a feature . Auto-commit mode in storm-kafka used
> rarely and most users
> run the kafka spout with ackers and get at-least once guarantee.  If its
> going to longer to address the PR reviews
> I am +1 on moving this out of Storm 1.1.0. We already quite a few patches
> storm-kafka-client and 1.1.0 release brings in lot of improvements
> and bug-fixes.
> -Harsha
> 
> On Wed, Feb 8, 2017 at 6:15 PM Jungtaek Lim <ka...@gmail.com> wrote:
> 
>> There seems some pull requests for bugfix/improvement on
>> storm-kafka-client, and some authors in PRs are not availble for now.
>> (waiting 7 days)
>> 
>> If we plan to get 1.1.1 out soon (say 1 month later or even closer) we can
>> postpone, but if not, it might be better to coordinate these things ASAP
>> and include to 1.1.0.
>> 
>> There seems to be other small PRs, but nothing seems critical so it would
>> be OK to not wait for merging.
>> 
>> - Jungtaek Lim
>> 
>> 2017년 2월 9일 (목) 오전 6:48, P. Taylor Goetz <pt...@gmail.com>님이 작성:
>> 
>> Right now we’re down to 1 open issue on the 1.1.0 release epic: STORM-2250
>> which is under active review/discussion.
>> 
>> Assuming that is mergeable in the near future, are there any other open
>> issues that should be considered for this release?
>> 
>> -Taylor
>> 
>> 
>>> On Feb 2, 2017, at 4:48 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
>>> 
>>> Thanks for putting this list together Jungtaek. I added a few to the 1.1
>> release epic that I think are important. Feel free to do the same.
>>> 
>>> Looks like we have a few to go, but there are pull requests for them.
>> It’s mostly just a matter of reviews and review responses, so I think we
>> are close.
>>> 
>>> -Taylor
>>> 
>>>> On Feb 2, 2017, at 1:41 AM, Jungtaek Lim <ka...@gmail.com> wrote:
>>>> 
>>>> Seems like there're not blockers for 1.1.0, but some pull requests are
>>>> worth to check.
>>>> There're pending pull requests for storm-kafka-client waited on
>> STORM-2225.
>>>> Given that STORM-2225 is now merged, we might need to take a look at.
>>>> 
>>>> *- reviewing*
>>>> 
>>>> [storm-core]
>>>> 
>>>>> STORM-2324 : Fix deployment failure if resources directory is missing
>> in
>>>> topology jar
>>>> (master) https://github.com/apache/storm/pull/1908
>>>> (1.x) https://github.com/apache/storm/pull/1898
>>>> 
>>>>> STORM-2321 Handle blobstore zk key deletion in KeySequenceNumber
>>>> (master) https://github.com/apache/storm/pull/1904
>>>> (1.x) https://github.com/apache/storm/pull/1905
>>>> 
>>>> [storm-kafka]
>>>> 
>>>>> STORM-2270 Kafka spout should consume from latest when ZK partition
>>>> commit offset bigger than the latest offset
>>>> (1.x) https://github.com/apache/storm/pull/1851
>>>> 
>>>> [storm-kafka-client]
>>>> 
>>>>> STORM-2281: Running Multiple Kafka Spouts (Trident) Throws Illegal
>> State
>>>> Exception
>>>> (1.x) https://github.com/apache/storm/pull/1902
>>>> 
>>>>> STORM-2315 Storm kafka client does not commit offsets when ack is
>> disabled
>>>> (1.x) https://github.com/apache/storm/pull/1891
>>>> 
>>>>> fix: KafkaSpout is blocked in AutoCommitMode
>>>> (master) https://github.com/apache/storm/pull/1863
>>>> 
>>>>> STORM-2250: Kafka Spout Refactoring to Increase Modularity and
>> Testability
>>>> (master) https://github.com/apache/storm/pull/1832
>>>> 
>>>>> STORM-2014: Put logic around dropping messages into RetryService,
>> remove
>>>> maxRetry setting from new KafkaSpout
>>>> (master) https://github.com/apache/storm/pull/1605
>>>> 
>>>>> fix NullPointException with acked.get(rtp)
>>>> (master) https://github.com/apache/storm/pull/1807
>>>> 
>>>> [storm-sql]
>>>> 
>>>>> STORM-1443 [Storm SQL] Support customizing parallelism in StormSQL
>>>> https://github.com/apache/storm/pull/1739
>>>> 
>>>> *- pending*
>>>> 
>>>> [storm-kafka-client]
>>>> 
>>>>> STORM-2296 Kafka spout no dup on leader changes
>>>> (1.0.x) https://github.com/apache/storm/pull/1873
>>>> (1.x) https://github.com/apache/storm/pull/1888
>>>> 
>>>> [storm-sql]
>>>> 
>>>>> STORM-2148 [Storm SQL] Trident mode: back to code generate and compile
>>>> Trident topology
>>>> https://github.com/apache/storm/pull/1743
>>>> 
>>>> Thanks,
>>>> Jungtaek Lim (HeartSaVioR)
>>>> 
>>>> 2017년 2월 2일 (목) 오전 8:14, Harsha Chintalapani <st...@harsha.io>님이 작성:
>>>> 
>>>>> Trying to check the status on this release of 1.1.0. Are we going to
> do
>>>>> this release anytime soon?
>>>>> 
>>>>> 
>>>>> On Fri, Jan 13, 2017 at 7:50 PM S G <sg...@gmail.com> wrote:
>>>>> 
>>>>>> Not sure if its a little late to include for the 1.1.0 and 1.0.3
>> releases
>>>>>> now, but can we consider using zookeeper 3.4.9 for the future
> versions
>> as
>>>>>> 3.4.9 brings in a lot of stability improvements (
>>>>>> http://zookeeper.apache.org/releases.html) and storm is still using
>>>>> 3.4.6
>>>>>> (
>>>>>> https://github.com/apache/storm/blob/master/pom.xml)
>>>>>> 
>>>>>> On Fri, Jan 6, 2017 at 11:54 AM, P. Taylor Goetz <pt...@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> Thanks for the update Jungtaek.
>>>>>>> 
>>>>>>> I’m verifying the patches now. And they should be mergeable shortly.
>>>>>>> 
>>>>>>> I think we’ll likely be ready for 1.1.0 and 1.0.3 releases next
> week.
>>>>>>> 
>>>>>>> -Taylor
>>>>>>> 
>>>>>>>> On Jan 6, 2017, at 3:40 AM, Jungtaek Lim <ka...@gmail.com> wrote:
>>>>>>>> 
>>>>>>>> I just submitted a patch for STORM-2176
>>>>>>>> <https://issues.apache.org/jira/browse/STORM-2176>. Since it's a
>>>>> small
>>>>>>> fix,
>>>>>>>> we just need to handle STORM-2228
>>>>>>>> <https://issues.apache.org/jira/browse/STORM-2228> to release.
>>>>>>>> 
>>>>>>>> - Jungtaek Lim (HeartSaVioR)
>>>>>>>> 
>>>>>>>> 2017년 1월 5일 (목) 오후 1:43, Jungtaek Lim <ka...@gmail.com>님이 작성:
>>>>>>>> 
>>>>>>>>> Recently we receive some requests regarding release Storm 1.0.3,
> so
>>>>>>> would
>>>>>>>>> like to bump this again.
>>>>>>>>> 
>>>>>>>>> Given that blocker issues for Storm 1.1.0 are also blocker for
>>>>> 1.0.3,
>>>>>>> I'd
>>>>>>>>> like to ask a favor of taking care of 'open' / 'in progress'
> issues
>>>>> on
>>>>>>>>> 1.1.0 epic.
>>>>>>>>> https://issues.apache.org/jira/browse/STORM-1856
>>>>>>>>> 
>>>>>>>>> There're one 'open' issue and three 'in progress' issues. Two of
>>>>> three
>>>>>>> 'in
>>>>>>>>> progress' issues are tiny fix so easy to be handled, so actual
>>>>>> blockers
>>>>>>> are
>>>>>>>>> STORM-2176 <https://issues.apache.org/jira/browse/STORM-2176> and
>>>>>>>>> STORM-2228 <https://issues.apache.org/jira/browse/STORM-2228>.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>> 
>>>>>>>>> 2016년 11월 17일 (목) 오후 11:41, Satish Duggana <
>>>>> satish.duggana@gmail.com
>>>>>>> 님이
>>>>>>>>> 작성:
>>>>>>>>> 
>>>>>>>>> STORM-2205: Race condition in getting nimbus summaries while ZK
>>>>>>>>> connections are reconnected.
>>>>>>>>> 
>>>>>>>>> This issue seems to occur in our environments and I would like
> this
>>>>> to
>>>>>>> be
>>>>>>>>> part of 1.1.0.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Satish.
>>>>>>>>> 
>>>>>>>>> On Thu, Nov 17, 2016 at 9:36 AM, Jungtaek Lim <ka...@gmail.com>
>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> I have no idea on storm-kafka-client, but some bugfix issues for
>>>>>>>>>> storm-kafka-client are waiting for reviewing / merging.
>>>>>>>>>> 
>>>>>>>>>> STORM-2014 <https://issues.apache.org/jira/browse/STORM-2014>
>>>>>>>>>> STORM-2087 <https://issues.apache.org/jira/browse/STORM-2087>
>>>>>>>>>> STORM-2104 <https://issues.apache.org/jira/browse/STORM-2104>
>>>>>>>>>> 
>>>>>>>>>> If someone can review them in several days it would be great.
>>>>>>>>>> 
>>>>>>>>>> I hope that we include currently opened pull requests for Storm
>> SQL
>>>>>> so
>>>>>>>>> that
>>>>>>>>>> we can release 'usable Storm SQL' more usable, but I'm also OK to
>>>>>>>>> postpone
>>>>>>>>>> them to be included to next release if they drag the release.
>>>>>>>>>> 
>>>>>>>>>> STORM-1446 <http://issues.apache.org/jira/browse/STORM-1446>
>>>>>>>>>> STORM-1443 <https://issues.apache.org/jira/browse/STORM-1443>
>>>>>>>>>> STORM-2148 <https://issues.apache.org/jira/browse/STORM-2148>
>>>>>>>>>> STORM-2170 <https://issues.apache.org/jira/browse/STORM-2170>
>>>>>>>>>> 
>>>>>>>>>> I can see some pull requests which address Trident
> implementations
>>>>>> for
>>>>>>>>>> storm-kafka-client, storm-mongodb, storm-cassandra.
>>>>>>>>>> 
>>>>>>>>>> storm-kafka-client: STORM-1694
>>>>>>>>>> <https://issues.apache.org/jira/browse/STORM-1694> (patch for 2.0
>>>>> is
>>>>>>>>>> merged, patch for 1.x is ready for reviewing)
>>>>>>>>>> storm-cassandra: STORM-1369
>>>>>>>>>> <https://issues.apache.org/jira/browse/STORM-1369>
>>>>>>>>>> storm-mongodb: STORM-1607 <https://issues.apache.org/
>>>>>>>>>> jira/browse/STORM-1607>
>>>>>>>>>> 
>>>>>>>>>> If we want to cut the release now, we could include only bugfix
>>>>>> issues
>>>>>>>>> and
>>>>>>>>>> postpone others. Otherwise we could discuss and include some or
>> all
>>>>>> of
>>>>>>>>> the
>>>>>>>>>> above.
>>>>>>>>>> 
>>>>>>>>>> What do you think? When we want to start the release process for
>>>>>> 1.1.0?
>>>>>>>>>> 
>>>>>>>>>> - Jungtaek Lim (HeartSaVioR)
>>>>>>>>>> 
>>>>>>>>>> 2016년 11월 16일 (수) 오전 4:11, P. Taylor Goetz <pt...@gmail.com>님이
>>>>> 작성:
>>>>>>>>>> 
>>>>>>>>>> Thanks Xin, I added it to the 1.1.0 epic.
>>>>>>>>>> 
>>>>>>>>>> -Taylor
>>>>>>>>>> 
>>>>>>>>>>> On Nov 15, 2016, at 9:01 AM, Xin Wang <da...@gmail.com>
>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> STORM-2198 ( PR: https://github.com/apache/storm/pull/1773 )
>>>>> fixes
>>>>>> a
>>>>>>>>> bug
>>>>>>>>>> of
>>>>>>>>>>> storm-hdfs. Do we have a consideration to include this?
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Xin Wang (vesense)
>>>>>>>>>>> 
>>>>>>>>>>> 2016-11-15 10:03 GMT+08:00 Jungtaek Lim <ka...@gmail.com>:
>>>>>>>>>>> 
>>>>>>>>>>>> Some issues on Storm SQL are resolved but not documented yet.
>>>>> I'll
>>>>>>>>> file
>>>>>>>>>> an
>>>>>>>>>>>> issue and assign to 1.1.0 release epic.
>>>>>>>>>>>> And also I want to address dropping aggregation and join on
>> Storm
>>>>>> SQL
>>>>>>>>>>>> Trident mode before releasing. I'll assign it too.
>>>>>>>>>>>> 
>>>>>>>>>>>> - Jungtaek Lim (HeartSaVioR)
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 2016년 11월 15일 (화) 오전 5:55, P. Taylor Goetz <ptgoetz@gmail.com
>>> 님이
>>>>>> 작성:
>>>>>>>>>>>> 
>>>>>>>>>>>>> I think we’re very close. I would like to confirm that the
>>>>>>> 1.x-branch
>>>>>>>>>> is
>>>>>>>>>>>>> not affected by STORM-2176.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> The worker lifecycle API was added in 1.0, but doesn’t work in
>>>>> any
>>>>>>>>>>>>> released version due to STORM-2176.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> If there are any other open JIRAs that anyone is passionate
>>>>> about,
>>>>>>>>> now
>>>>>>>>>>>>> would be a good time to assign them to the 1.1.0 release epic
>>>>>>>>>>>> (STORM-1856).
>>>>>>>>>>>>> 
>>>>>>>>>>>>> -Taylor
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Oct 27, 2016, at 12:19 PM, Jungtaek Lim <kabhwan@gmail.com
>>> 
>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Finally Pacemaker H/A, Supervisor V2, and Storm SQL PRs which
>>>>>> were
>>>>>>>>>>>> opened
>>>>>>>>>>>>>> at the last mail (4 weeks ago) are all merged to 1.x branch.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> There're some more PRs on Storm SQL opened, but given that we
>>>>> can
>>>>>>>>>>>> release
>>>>>>>>>>>>>> new minor at any time when we feel it's enough change, I can
>>>>> wait
>>>>>>>>> for
>>>>>>>>>>>> it.
>>>>>>>>>>>>>> They didn't get reviewed yet indeed.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Is there something else we would want to include it to 1.1.0?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 2016년 10월 1일 (토) 오전 9:30, Jungtaek Lim <ka...@gmail.com>님이
>>>>> 작성:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Personally, merging and porting back to three branches are
>>>>>> painful
>>>>>>>>>>>>> enough,
>>>>>>>>>>>>>>> especially we don't have merging script and having verbose
>>>>>> process
>>>>>>>>> (I
>>>>>>>>>>>>> mean
>>>>>>>>>>>>>>> CHANGELOG).
>>>>>>>>>>>>>>> It would be better if merging process is automated (by
>> running
>>>>>>>>> script
>>>>>>>>>>>> or
>>>>>>>>>>>>>>> so), so I'd +1 to revisit Harsha's suggestion (adopting
> Kafka
>>>>>>> merge
>>>>>>>>>>>>> script)
>>>>>>>>>>>>>>> and modify script to fit to Storm.
>>>>>>>>>>>>>>> (It will not work if it's the case we need to handle PRs for
>>>>>> each
>>>>>>>>>>>>> version
>>>>>>>>>>>>>>> line, since 'Close' in commit log doesn't close the PR if
> its
>>>>>>>>> target
>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>> is not master.)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Anyway, without automation I don't want to maintain more
>>>>> version
>>>>>>>>>>>> lines.
>>>>>>>>>>>>>>> I'm looking at the announces from other projects, and others
>>>>> are
>>>>>>>>> only
>>>>>>>>>>>>>>> maintaining two version lines.
>>>>>>>>>>>>>>> Since we maintain 2.0.0 version line we can't reduce version
>>>>>> lines
>>>>>>>>> to
>>>>>>>>>>>> 2,
>>>>>>>>>>>>>>> but hopefully at most 3.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Btw, let's check pending pull requests and enumerate which
>> can
>>>>>> be
>>>>>>>>>>>>> included
>>>>>>>>>>>>>>> in 1.0.0, and start/finish review and merge them soon.
>>>>>>>>>>>>>>> For me Supervisor V2 and Pacemaker H/A, and pending Storm
> SQL
>>>>>> PRs
>>>>>>>>> can
>>>>>>>>>>>> be
>>>>>>>>>>>>>>> included, since they are small or in reviewing and expected
>> to
>>>>>>> pass
>>>>>>>>>>>>> review
>>>>>>>>>>>>>>> phase soon.
>>>>>>>>>>>>>>> (And some small PRs. There're other valuable PRs in PR list
>>>>> but
>>>>>>> I'm
>>>>>>>>>>>> not
>>>>>>>>>>>>>>> sure we can review them soon. One example is unified API.)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> One issue which is not clear is STORM-2006
>>>>>>>>>>>>>>> <https://github.com/apache/storm/pull/1595>. This is a
>>>>>> candidate
>>>>>>>>> for
>>>>>>>>>>>>> me,
>>>>>>>>>>>>>>> but gets blocked while reviewing. If we plan to put great
>>>>> effort
>>>>>>> to
>>>>>>>>>>>>> revise
>>>>>>>>>>>>>>> Metric we can skip this.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Please enumerate other PRs as well if you want to include in
>>>>>>> 1.1.0.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Jungtaek Lim
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 2016년 9월 30일 (금) 오후 11:09, Bobby Evans
>>>>>>> <evans@yahoo-inc.com.invalid
>>>>>>>>>> 
>>>>>>>>>>>> 님이
>>>>>>>>>>>>> 작성:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Sounds good to me.  It would be nice to get some of the new
>>>>>>>>> features
>>>>>>>>>>>>> out.
>>>>>>>>>>>>>>> Do we expect to maintain both 1.0.x and 1.1.x lines with bug
>>>>>>> fixes?
>>>>>>>>>>>>> And if
>>>>>>>>>>>>>>> so for how long do we want to do this for? - Bobby
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Thursday, September 29, 2016 7:35 PM, Jungtaek Lim <
>>>>>>>>>>>>>>> kabhwan@gmail.com> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Hi devs,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> It's been 5 months after releasing Storm 1.0.0, and now 1.x
>>>>>> branch
>>>>>>>>>> has
>>>>>>>>>>>>> lots
>>>>>>>>>>>>>>> of CHANGELOG and also pending reviews.
>>>>>>>>>>>>>>> It's also been a long time after 1.1.0 RC1 is canceled.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I think it may be good to put some efforts to review and
>> merge
>>>>>>>>>> pending
>>>>>>>>>>>>> pull
>>>>>>>>>>>>>>> requests (except things which takes time to review and
> test),
>>>>>> and
>>>>>>>>>>>>> release
>>>>>>>>>>>>>>> 1.1.0 soon.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I'm also open to volunteer release manager for 1.1.0 after
> we
>>>>>>>>>> document
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> process of official release.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>> 
>>

Re: [DISCUSS] Release Storm 1.1.0

Posted by Jungtaek Lim <ka...@gmail.com>.

Now all issues on the Storm 1.1.0 epic got resolved. :)

There're still some bugfix pull requests for storm-kafka-client, but
releasing Storm 1.1.0 has been tremendously dragged (I initiated this at
Sep. 2016) so unless they're critical, I'm +1 on starting release phase on
Storm 1.1.0 ASAP.

- Jungtaek Lim

2017년 2월 14일 (화) 오전 11:41, Harsha Chintalapani <st...@harsha.io>님이 작성:

STORM-2340 is more of a feature . Auto-commit mode in storm-kafka used
rarely and most users
run the kafka spout with ackers and get at-least once guarantee.  If its
going to longer to address the PR reviews
I am +1 on moving this out of Storm 1.1.0. We already quite a few patches
storm-kafka-client and 1.1.0 release brings in lot of improvements
and bug-fixes.
-Harsha

On Wed, Feb 8, 2017 at 6:15 PM Jungtaek Lim <ka...@gmail.com> wrote:

> There seems some pull requests for bugfix/improvement on
> storm-kafka-client, and some authors in PRs are not availble for now.
> (waiting 7 days)
>
> If we plan to get 1.1.1 out soon (say 1 month later or even closer) we can
> postpone, but if not, it might be better to coordinate these things ASAP
> and include to 1.1.0.
>
> There seems to be other small PRs, but nothing seems critical so it would
> be OK to not wait for merging.
>
> - Jungtaek Lim
>
> 2017년 2월 9일 (목) 오전 6:48, P. Taylor Goetz <pt...@gmail.com>님이 작성:
>
> Right now we’re down to 1 open issue on the 1.1.0 release epic: STORM-2250
> which is under active review/discussion.
>
> Assuming that is mergeable in the near future, are there any other open
> issues that should be considered for this release?
>
> -Taylor
>
>
> > On Feb 2, 2017, at 4:48 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
> >
> > Thanks for putting this list together Jungtaek. I added a few to the 1.1
> release epic that I think are important. Feel free to do the same.
> >
> > Looks like we have a few to go, but there are pull requests for them.
> It’s mostly just a matter of reviews and review responses, so I think we
> are close.
> >
> > -Taylor
> >
> >> On Feb 2, 2017, at 1:41 AM, Jungtaek Lim <ka...@gmail.com> wrote:
> >>
> >> Seems like there're not blockers for 1.1.0, but some pull requests are
> >> worth to check.
> >> There're pending pull requests for storm-kafka-client waited on
> STORM-2225.
> >> Given that STORM-2225 is now merged, we might need to take a look at.
> >>
> >> *- reviewing*
> >>
> >> [storm-core]
> >>
> >>> STORM-2324 : Fix deployment failure if resources directory is missing
> in
> >> topology jar
> >> (master) https://github.com/apache/storm/pull/1908
> >> (1.x) https://github.com/apache/storm/pull/1898
> >>
> >>> STORM-2321 Handle blobstore zk key deletion in KeySequenceNumber
> >> (master) https://github.com/apache/storm/pull/1904
> >> (1.x) https://github.com/apache/storm/pull/1905
> >>
> >> [storm-kafka]
> >>
> >>> STORM-2270 Kafka spout should consume from latest when ZK partition
> >> commit offset bigger than the latest offset
> >> (1.x) https://github.com/apache/storm/pull/1851
> >>
> >> [storm-kafka-client]
> >>
> >>> STORM-2281: Running Multiple Kafka Spouts (Trident) Throws Illegal
> State
> >> Exception
> >> (1.x) https://github.com/apache/storm/pull/1902
> >>
> >>> STORM-2315 Storm kafka client does not commit offsets when ack is
> disabled
> >> (1.x) https://github.com/apache/storm/pull/1891
> >>
> >>> fix: KafkaSpout is blocked in AutoCommitMode
> >> (master) https://github.com/apache/storm/pull/1863
> >>
> >>> STORM-2250: Kafka Spout Refactoring to Increase Modularity and
> Testability
> >> (master) https://github.com/apache/storm/pull/1832
> >>
> >>> STORM-2014: Put logic around dropping messages into RetryService,
> remove
> >> maxRetry setting from new KafkaSpout
> >> (master) https://github.com/apache/storm/pull/1605
> >>
> >>> fix NullPointException with acked.get(rtp)
> >> (master) https://github.com/apache/storm/pull/1807
> >>
> >> [storm-sql]
> >>
> >>> STORM-1443 [Storm SQL] Support customizing parallelism in StormSQL
> >> https://github.com/apache/storm/pull/1739
> >>
> >> *- pending*
> >>
> >> [storm-kafka-client]
> >>
> >>> STORM-2296 Kafka spout no dup on leader changes
> >> (1.0.x) https://github.com/apache/storm/pull/1873
> >> (1.x) https://github.com/apache/storm/pull/1888
> >>
> >> [storm-sql]
> >>
> >>> STORM-2148 [Storm SQL] Trident mode: back to code generate and compile
> >> Trident topology
> >> https://github.com/apache/storm/pull/1743
> >>
> >> Thanks,
> >> Jungtaek Lim (HeartSaVioR)
> >>
> >> 2017년 2월 2일 (목) 오전 8:14, Harsha Chintalapani <st...@harsha.io>님이 작성:
> >>
> >>> Trying to check the status on this release of 1.1.0. Are we going to
do
> >>> this release anytime soon?
> >>>
> >>>
> >>> On Fri, Jan 13, 2017 at 7:50 PM S G <sg...@gmail.com> wrote:
> >>>
> >>>> Not sure if its a little late to include for the 1.1.0 and 1.0.3
> releases
> >>>> now, but can we consider using zookeeper 3.4.9 for the future
versions
> as
> >>>> 3.4.9 brings in a lot of stability improvements (
> >>>> http://zookeeper.apache.org/releases.html) and storm is still using
> >>> 3.4.6
> >>>> (
> >>>> https://github.com/apache/storm/blob/master/pom.xml)
> >>>>
> >>>> On Fri, Jan 6, 2017 at 11:54 AM, P. Taylor Goetz <pt...@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> Thanks for the update Jungtaek.
> >>>>>
> >>>>> I’m verifying the patches now. And they should be mergeable shortly.
> >>>>>
> >>>>> I think we’ll likely be ready for 1.1.0 and 1.0.3 releases next
week.
> >>>>>
> >>>>> -Taylor
> >>>>>
> >>>>>> On Jan 6, 2017, at 3:40 AM, Jungtaek Lim <ka...@gmail.com> wrote:
> >>>>>>
> >>>>>> I just submitted a patch for STORM-2176
> >>>>>> <https://issues.apache.org/jira/browse/STORM-2176>. Since it's a
> >>> small
> >>>>> fix,
> >>>>>> we just need to handle STORM-2228
> >>>>>> <https://issues.apache.org/jira/browse/STORM-2228> to release.
> >>>>>>
> >>>>>> - Jungtaek Lim (HeartSaVioR)
> >>>>>>
> >>>>>> 2017년 1월 5일 (목) 오후 1:43, Jungtaek Lim <ka...@gmail.com>님이 작성:
> >>>>>>
> >>>>>>> Recently we receive some requests regarding release Storm 1.0.3,
so
> >>>>> would
> >>>>>>> like to bump this again.
> >>>>>>>
> >>>>>>> Given that blocker issues for Storm 1.1.0 are also blocker for
> >>> 1.0.3,
> >>>>> I'd
> >>>>>>> like to ask a favor of taking care of 'open' / 'in progress'
issues
> >>> on
> >>>>>>> 1.1.0 epic.
> >>>>>>> https://issues.apache.org/jira/browse/STORM-1856
> >>>>>>>
> >>>>>>> There're one 'open' issue and three 'in progress' issues. Two of
> >>> three
> >>>>> 'in
> >>>>>>> progress' issues are tiny fix so easy to be handled, so actual
> >>>> blockers
> >>>>> are
> >>>>>>> STORM-2176 <https://issues.apache.org/jira/browse/STORM-2176> and
> >>>>>>> STORM-2228 <https://issues.apache.org/jira/browse/STORM-2228>.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Jungtaek Lim (HeartSaVioR)
> >>>>>>>
> >>>>>>> 2016년 11월 17일 (목) 오후 11:41, Satish Duggana <
> >>> satish.duggana@gmail.com
> >>>>> 님이
> >>>>>>> 작성:
> >>>>>>>
> >>>>>>>  STORM-2205: Race condition in getting nimbus summaries while ZK
> >>>>>>> connections are reconnected.
> >>>>>>>
> >>>>>>> This issue seems to occur in our environments and I would like
this
> >>> to
> >>>>> be
> >>>>>>> part of 1.1.0.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Satish.
> >>>>>>>
> >>>>>>> On Thu, Nov 17, 2016 at 9:36 AM, Jungtaek Lim <ka...@gmail.com>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> I have no idea on storm-kafka-client, but some bugfix issues for
> >>>>>>>> storm-kafka-client are waiting for reviewing / merging.
> >>>>>>>>
> >>>>>>>> STORM-2014 <https://issues.apache.org/jira/browse/STORM-2014>
> >>>>>>>> STORM-2087 <https://issues.apache.org/jira/browse/STORM-2087>
> >>>>>>>> STORM-2104 <https://issues.apache.org/jira/browse/STORM-2104>
> >>>>>>>>
> >>>>>>>> If someone can review them in several days it would be great.
> >>>>>>>>
> >>>>>>>> I hope that we include currently opened pull requests for Storm
> SQL
> >>>> so
> >>>>>>> that
> >>>>>>>> we can release 'usable Storm SQL' more usable, but I'm also OK to
> >>>>>>> postpone
> >>>>>>>> them to be included to next release if they drag the release.
> >>>>>>>>
> >>>>>>>> STORM-1446 <http://issues.apache.org/jira/browse/STORM-1446>
> >>>>>>>> STORM-1443 <https://issues.apache.org/jira/browse/STORM-1443>
> >>>>>>>> STORM-2148 <https://issues.apache.org/jira/browse/STORM-2148>
> >>>>>>>> STORM-2170 <https://issues.apache.org/jira/browse/STORM-2170>
> >>>>>>>>
> >>>>>>>> I can see some pull requests which address Trident
implementations
> >>>> for
> >>>>>>>> storm-kafka-client, storm-mongodb, storm-cassandra.
> >>>>>>>>
> >>>>>>>> storm-kafka-client: STORM-1694
> >>>>>>>> <https://issues.apache.org/jira/browse/STORM-1694> (patch for 2.0
> >>> is
> >>>>>>>> merged, patch for 1.x is ready for reviewing)
> >>>>>>>> storm-cassandra: STORM-1369
> >>>>>>>> <https://issues.apache.org/jira/browse/STORM-1369>
> >>>>>>>> storm-mongodb: STORM-1607 <https://issues.apache.org/
> >>>>>>>> jira/browse/STORM-1607>
> >>>>>>>>
> >>>>>>>> If we want to cut the release now, we could include only bugfix
> >>>> issues
> >>>>>>> and
> >>>>>>>> postpone others. Otherwise we could discuss and include some or
> all
> >>>> of
> >>>>>>> the
> >>>>>>>> above.
> >>>>>>>>
> >>>>>>>> What do you think? When we want to start the release process for
> >>>> 1.1.0?
> >>>>>>>>
> >>>>>>>> - Jungtaek Lim (HeartSaVioR)
> >>>>>>>>
> >>>>>>>> 2016년 11월 16일 (수) 오전 4:11, P. Taylor Goetz <pt...@gmail.com>님이
> >>> 작성:
> >>>>>>>>
> >>>>>>>> Thanks Xin, I added it to the 1.1.0 epic.
> >>>>>>>>
> >>>>>>>> -Taylor
> >>>>>>>>
> >>>>>>>>> On Nov 15, 2016, at 9:01 AM, Xin Wang <da...@gmail.com>
> >>>> wrote:
> >>>>>>>>>
> >>>>>>>>> STORM-2198 ( PR: https://github.com/apache/storm/pull/1773 )
> >>> fixes
> >>>> a
> >>>>>>> bug
> >>>>>>>> of
> >>>>>>>>> storm-hdfs. Do we have a consideration to include this?
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Xin Wang (vesense)
> >>>>>>>>>
> >>>>>>>>> 2016-11-15 10:03 GMT+08:00 Jungtaek Lim <ka...@gmail.com>:
> >>>>>>>>>
> >>>>>>>>>> Some issues on Storm SQL are resolved but not documented yet.
> >>> I'll
> >>>>>>> file
> >>>>>>>> an
> >>>>>>>>>> issue and assign to 1.1.0 release epic.
> >>>>>>>>>> And also I want to address dropping aggregation and join on
> Storm
> >>>> SQL
> >>>>>>>>>> Trident mode before releasing. I'll assign it too.
> >>>>>>>>>>
> >>>>>>>>>> - Jungtaek Lim (HeartSaVioR)
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> 2016년 11월 15일 (화) 오전 5:55, P. Taylor Goetz <ptgoetz@gmail.com
> >님이
> >>>> 작성:
> >>>>>>>>>>
> >>>>>>>>>>> I think we’re very close. I would like to confirm that the
> >>>>> 1.x-branch
> >>>>>>>> is
> >>>>>>>>>>> not affected by STORM-2176.
> >>>>>>>>>>>
> >>>>>>>>>>> The worker lifecycle API was added in 1.0, but doesn’t work in
> >>> any
> >>>>>>>>>>> released version due to STORM-2176.
> >>>>>>>>>>>
> >>>>>>>>>>> If there are any other open JIRAs that anyone is passionate
> >>> about,
> >>>>>>> now
> >>>>>>>>>>> would be a good time to assign them to the 1.1.0 release epic
> >>>>>>>>>> (STORM-1856).
> >>>>>>>>>>>
> >>>>>>>>>>> -Taylor
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>> On Oct 27, 2016, at 12:19 PM, Jungtaek Lim <kabhwan@gmail.com
> >
> >>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Finally Pacemaker H/A, Supervisor V2, and Storm SQL PRs which
> >>>> were
> >>>>>>>>>> opened
> >>>>>>>>>>>> at the last mail (4 weeks ago) are all merged to 1.x branch.
> >>>>>>>>>>>>
> >>>>>>>>>>>> There're some more PRs on Storm SQL opened, but given that we
> >>> can
> >>>>>>>>>> release
> >>>>>>>>>>>> new minor at any time when we feel it's enough change, I can
> >>> wait
> >>>>>>> for
> >>>>>>>>>> it.
> >>>>>>>>>>>> They didn't get reviewed yet indeed.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Is there something else we would want to include it to 1.1.0?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
> >>>>>>>>>>>>
> >>>>>>>>>>>> 2016년 10월 1일 (토) 오전 9:30, Jungtaek Lim <ka...@gmail.com>님이
> >>> 작성:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Personally, merging and porting back to three branches are
> >>>> painful
> >>>>>>>>>>> enough,
> >>>>>>>>>>>>> especially we don't have merging script and having verbose
> >>>> process
> >>>>>>> (I
> >>>>>>>>>>> mean
> >>>>>>>>>>>>> CHANGELOG).
> >>>>>>>>>>>>> It would be better if merging process is automated (by
> running
> >>>>>>> script
> >>>>>>>>>> or
> >>>>>>>>>>>>> so), so I'd +1 to revisit Harsha's suggestion (adopting
Kafka
> >>>>> merge
> >>>>>>>>>>> script)
> >>>>>>>>>>>>> and modify script to fit to Storm.
> >>>>>>>>>>>>> (It will not work if it's the case we need to handle PRs for
> >>>> each
> >>>>>>>>>>> version
> >>>>>>>>>>>>> line, since 'Close' in commit log doesn't close the PR if
its
> >>>>>>> target
> >>>>>>>>>>> branch
> >>>>>>>>>>>>> is not master.)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Anyway, without automation I don't want to maintain more
> >>> version
> >>>>>>>>>> lines.
> >>>>>>>>>>>>> I'm looking at the announces from other projects, and others
> >>> are
> >>>>>>> only
> >>>>>>>>>>>>> maintaining two version lines.
> >>>>>>>>>>>>> Since we maintain 2.0.0 version line we can't reduce version
> >>>> lines
> >>>>>>> to
> >>>>>>>>>> 2,
> >>>>>>>>>>>>> but hopefully at most 3.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Btw, let's check pending pull requests and enumerate which
> can
> >>>> be
> >>>>>>>>>>> included
> >>>>>>>>>>>>> in 1.0.0, and start/finish review and merge them soon.
> >>>>>>>>>>>>> For me Supervisor V2 and Pacemaker H/A, and pending Storm
SQL
> >>>> PRs
> >>>>>>> can
> >>>>>>>>>> be
> >>>>>>>>>>>>> included, since they are small or in reviewing and expected
> to
> >>>>> pass
> >>>>>>>>>>> review
> >>>>>>>>>>>>> phase soon.
> >>>>>>>>>>>>> (And some small PRs. There're other valuable PRs in PR list
> >>> but
> >>>>> I'm
> >>>>>>>>>> not
> >>>>>>>>>>>>> sure we can review them soon. One example is unified API.)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> One issue which is not clear is STORM-2006
> >>>>>>>>>>>>> <https://github.com/apache/storm/pull/1595>. This is a
> >>>> candidate
> >>>>>>> for
> >>>>>>>>>>> me,
> >>>>>>>>>>>>> but gets blocked while reviewing. If we plan to put great
> >>> effort
> >>>>> to
> >>>>>>>>>>> revise
> >>>>>>>>>>>>> Metric we can skip this.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Please enumerate other PRs as well if you want to include in
> >>>>> 1.1.0.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> Jungtaek Lim
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2016년 9월 30일 (금) 오후 11:09, Bobby Evans
> >>>>> <evans@yahoo-inc.com.invalid
> >>>>>>>>
> >>>>>>>>>> 님이
> >>>>>>>>>>> 작성:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Sounds good to me.  It would be nice to get some of the new
> >>>>>>> features
> >>>>>>>>>>> out.
> >>>>>>>>>>>>> Do we expect to maintain both 1.0.x and 1.1.x lines with bug
> >>>>> fixes?
> >>>>>>>>>>> And if
> >>>>>>>>>>>>> so for how long do we want to do this for? - Bobby
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thursday, September 29, 2016 7:35 PM, Jungtaek Lim <
> >>>>>>>>>>>>> kabhwan@gmail.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi devs,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> It's been 5 months after releasing Storm 1.0.0, and now 1.x
> >>>> branch
> >>>>>>>> has
> >>>>>>>>>>> lots
> >>>>>>>>>>>>> of CHANGELOG and also pending reviews.
> >>>>>>>>>>>>> It's also been a long time after 1.1.0 RC1 is canceled.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I think it may be good to put some efforts to review and
> merge
> >>>>>>>> pending
> >>>>>>>>>>> pull
> >>>>>>>>>>>>> requests (except things which takes time to review and
test),
> >>>> and
> >>>>>>>>>>> release
> >>>>>>>>>>>>> 1.1.0 soon.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I'm also open to volunteer release manager for 1.1.0 after
we
> >>>>>>>> document
> >>>>>>>>>>> the
> >>>>>>>>>>>>> process of official release.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >
>

Re: [DISCUSS] Release Storm 1.1.0

Posted by Harsha Chintalapani <st...@harsha.io>.

STORM-2340 is more of a feature . Auto-commit mode in storm-kafka used
rarely and most users
run the kafka spout with ackers and get at-least once guarantee.  If its
going to longer to address the PR reviews
I am +1 on moving this out of Storm 1.1.0. We already quite a few patches
storm-kafka-client and 1.1.0 release brings in lot of improvements
and bug-fixes.
-Harsha

On Wed, Feb 8, 2017 at 6:15 PM Jungtaek Lim <ka...@gmail.com> wrote:

> There seems some pull requests for bugfix/improvement on
> storm-kafka-client, and some authors in PRs are not availble for now.
> (waiting 7 days)
>
> If we plan to get 1.1.1 out soon (say 1 month later or even closer) we can
> postpone, but if not, it might be better to coordinate these things ASAP
> and include to 1.1.0.
>
> There seems to be other small PRs, but nothing seems critical so it would
> be OK to not wait for merging.
>
> - Jungtaek Lim
>
> 2017년 2월 9일 (목) 오전 6:48, P. Taylor Goetz <pt...@gmail.com>님이 작성:
>
> Right now we’re down to 1 open issue on the 1.1.0 release epic: STORM-2250
> which is under active review/discussion.
>
> Assuming that is mergeable in the near future, are there any other open
> issues that should be considered for this release?
>
> -Taylor
>
>
> > On Feb 2, 2017, at 4:48 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
> >
> > Thanks for putting this list together Jungtaek. I added a few to the 1.1
> release epic that I think are important. Feel free to do the same.
> >
> > Looks like we have a few to go, but there are pull requests for them.
> It’s mostly just a matter of reviews and review responses, so I think we
> are close.
> >
> > -Taylor
> >
> >> On Feb 2, 2017, at 1:41 AM, Jungtaek Lim <ka...@gmail.com> wrote:
> >>
> >> Seems like there're not blockers for 1.1.0, but some pull requests are
> >> worth to check.
> >> There're pending pull requests for storm-kafka-client waited on
> STORM-2225.
> >> Given that STORM-2225 is now merged, we might need to take a look at.
> >>
> >> *- reviewing*
> >>
> >> [storm-core]
> >>
> >>> STORM-2324 : Fix deployment failure if resources directory is missing
> in
> >> topology jar
> >> (master) https://github.com/apache/storm/pull/1908
> >> (1.x) https://github.com/apache/storm/pull/1898
> >>
> >>> STORM-2321 Handle blobstore zk key deletion in KeySequenceNumber
> >> (master) https://github.com/apache/storm/pull/1904
> >> (1.x) https://github.com/apache/storm/pull/1905
> >>
> >> [storm-kafka]
> >>
> >>> STORM-2270 Kafka spout should consume from latest when ZK partition
> >> commit offset bigger than the latest offset
> >> (1.x) https://github.com/apache/storm/pull/1851
> >>
> >> [storm-kafka-client]
> >>
> >>> STORM-2281: Running Multiple Kafka Spouts (Trident) Throws Illegal
> State
> >> Exception
> >> (1.x) https://github.com/apache/storm/pull/1902
> >>
> >>> STORM-2315 Storm kafka client does not commit offsets when ack is
> disabled
> >> (1.x) https://github.com/apache/storm/pull/1891
> >>
> >>> fix: KafkaSpout is blocked in AutoCommitMode
> >> (master) https://github.com/apache/storm/pull/1863
> >>
> >>> STORM-2250: Kafka Spout Refactoring to Increase Modularity and
> Testability
> >> (master) https://github.com/apache/storm/pull/1832
> >>
> >>> STORM-2014: Put logic around dropping messages into RetryService,
> remove
> >> maxRetry setting from new KafkaSpout
> >> (master) https://github.com/apache/storm/pull/1605
> >>
> >>> fix NullPointException with acked.get(rtp)
> >> (master) https://github.com/apache/storm/pull/1807
> >>
> >> [storm-sql]
> >>
> >>> STORM-1443 [Storm SQL] Support customizing parallelism in StormSQL
> >> https://github.com/apache/storm/pull/1739
> >>
> >> *- pending*
> >>
> >> [storm-kafka-client]
> >>
> >>> STORM-2296 Kafka spout no dup on leader changes
> >> (1.0.x) https://github.com/apache/storm/pull/1873
> >> (1.x) https://github.com/apache/storm/pull/1888
> >>
> >> [storm-sql]
> >>
> >>> STORM-2148 [Storm SQL] Trident mode: back to code generate and compile
> >> Trident topology
> >> https://github.com/apache/storm/pull/1743
> >>
> >> Thanks,
> >> Jungtaek Lim (HeartSaVioR)
> >>
> >> 2017년 2월 2일 (목) 오전 8:14, Harsha Chintalapani <st...@harsha.io>님이 작성:
> >>
> >>> Trying to check the status on this release of 1.1.0. Are we going to do
> >>> this release anytime soon?
> >>>
> >>>
> >>> On Fri, Jan 13, 2017 at 7:50 PM S G <sg...@gmail.com> wrote:
> >>>
> >>>> Not sure if its a little late to include for the 1.1.0 and 1.0.3
> releases
> >>>> now, but can we consider using zookeeper 3.4.9 for the future versions
> as
> >>>> 3.4.9 brings in a lot of stability improvements (
> >>>> http://zookeeper.apache.org/releases.html) and storm is still using
> >>> 3.4.6
> >>>> (
> >>>> https://github.com/apache/storm/blob/master/pom.xml)
> >>>>
> >>>> On Fri, Jan 6, 2017 at 11:54 AM, P. Taylor Goetz <pt...@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> Thanks for the update Jungtaek.
> >>>>>
> >>>>> I’m verifying the patches now. And they should be mergeable shortly.
> >>>>>
> >>>>> I think we’ll likely be ready for 1.1.0 and 1.0.3 releases next week.
> >>>>>
> >>>>> -Taylor
> >>>>>
> >>>>>> On Jan 6, 2017, at 3:40 AM, Jungtaek Lim <ka...@gmail.com> wrote:
> >>>>>>
> >>>>>> I just submitted a patch for STORM-2176
> >>>>>> <https://issues.apache.org/jira/browse/STORM-2176>. Since it's a
> >>> small
> >>>>> fix,
> >>>>>> we just need to handle STORM-2228
> >>>>>> <https://issues.apache.org/jira/browse/STORM-2228> to release.
> >>>>>>
> >>>>>> - Jungtaek Lim (HeartSaVioR)
> >>>>>>
> >>>>>> 2017년 1월 5일 (목) 오후 1:43, Jungtaek Lim <ka...@gmail.com>님이 작성:
> >>>>>>
> >>>>>>> Recently we receive some requests regarding release Storm 1.0.3, so
> >>>>> would
> >>>>>>> like to bump this again.
> >>>>>>>
> >>>>>>> Given that blocker issues for Storm 1.1.0 are also blocker for
> >>> 1.0.3,
> >>>>> I'd
> >>>>>>> like to ask a favor of taking care of 'open' / 'in progress' issues
> >>> on
> >>>>>>> 1.1.0 epic.
> >>>>>>> https://issues.apache.org/jira/browse/STORM-1856
> >>>>>>>
> >>>>>>> There're one 'open' issue and three 'in progress' issues. Two of
> >>> three
> >>>>> 'in
> >>>>>>> progress' issues are tiny fix so easy to be handled, so actual
> >>>> blockers
> >>>>> are
> >>>>>>> STORM-2176 <https://issues.apache.org/jira/browse/STORM-2176> and
> >>>>>>> STORM-2228 <https://issues.apache.org/jira/browse/STORM-2228>.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Jungtaek Lim (HeartSaVioR)
> >>>>>>>
> >>>>>>> 2016년 11월 17일 (목) 오후 11:41, Satish Duggana <
> >>> satish.duggana@gmail.com
> >>>>> 님이
> >>>>>>> 작성:
> >>>>>>>
> >>>>>>>  STORM-2205: Race condition in getting nimbus summaries while ZK
> >>>>>>> connections are reconnected.
> >>>>>>>
> >>>>>>> This issue seems to occur in our environments and I would like this
> >>> to
> >>>>> be
> >>>>>>> part of 1.1.0.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Satish.
> >>>>>>>
> >>>>>>> On Thu, Nov 17, 2016 at 9:36 AM, Jungtaek Lim <ka...@gmail.com>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> I have no idea on storm-kafka-client, but some bugfix issues for
> >>>>>>>> storm-kafka-client are waiting for reviewing / merging.
> >>>>>>>>
> >>>>>>>> STORM-2014 <https://issues.apache.org/jira/browse/STORM-2014>
> >>>>>>>> STORM-2087 <https://issues.apache.org/jira/browse/STORM-2087>
> >>>>>>>> STORM-2104 <https://issues.apache.org/jira/browse/STORM-2104>
> >>>>>>>>
> >>>>>>>> If someone can review them in several days it would be great.
> >>>>>>>>
> >>>>>>>> I hope that we include currently opened pull requests for Storm
> SQL
> >>>> so
> >>>>>>> that
> >>>>>>>> we can release 'usable Storm SQL' more usable, but I'm also OK to
> >>>>>>> postpone
> >>>>>>>> them to be included to next release if they drag the release.
> >>>>>>>>
> >>>>>>>> STORM-1446 <http://issues.apache.org/jira/browse/STORM-1446>
> >>>>>>>> STORM-1443 <https://issues.apache.org/jira/browse/STORM-1443>
> >>>>>>>> STORM-2148 <https://issues.apache.org/jira/browse/STORM-2148>
> >>>>>>>> STORM-2170 <https://issues.apache.org/jira/browse/STORM-2170>
> >>>>>>>>
> >>>>>>>> I can see some pull requests which address Trident implementations
> >>>> for
> >>>>>>>> storm-kafka-client, storm-mongodb, storm-cassandra.
> >>>>>>>>
> >>>>>>>> storm-kafka-client: STORM-1694
> >>>>>>>> <https://issues.apache.org/jira/browse/STORM-1694> (patch for 2.0
> >>> is
> >>>>>>>> merged, patch for 1.x is ready for reviewing)
> >>>>>>>> storm-cassandra: STORM-1369
> >>>>>>>> <https://issues.apache.org/jira/browse/STORM-1369>
> >>>>>>>> storm-mongodb: STORM-1607 <https://issues.apache.org/
> >>>>>>>> jira/browse/STORM-1607>
> >>>>>>>>
> >>>>>>>> If we want to cut the release now, we could include only bugfix
> >>>> issues
> >>>>>>> and
> >>>>>>>> postpone others. Otherwise we could discuss and include some or
> all
> >>>> of
> >>>>>>> the
> >>>>>>>> above.
> >>>>>>>>
> >>>>>>>> What do you think? When we want to start the release process for
> >>>> 1.1.0?
> >>>>>>>>
> >>>>>>>> - Jungtaek Lim (HeartSaVioR)
> >>>>>>>>
> >>>>>>>> 2016년 11월 16일 (수) 오전 4:11, P. Taylor Goetz <pt...@gmail.com>님이
> >>> 작성:
> >>>>>>>>
> >>>>>>>> Thanks Xin, I added it to the 1.1.0 epic.
> >>>>>>>>
> >>>>>>>> -Taylor
> >>>>>>>>
> >>>>>>>>> On Nov 15, 2016, at 9:01 AM, Xin Wang <da...@gmail.com>
> >>>> wrote:
> >>>>>>>>>
> >>>>>>>>> STORM-2198 ( PR: https://github.com/apache/storm/pull/1773 )
> >>> fixes
> >>>> a
> >>>>>>> bug
> >>>>>>>> of
> >>>>>>>>> storm-hdfs. Do we have a consideration to include this?
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Xin Wang (vesense)
> >>>>>>>>>
> >>>>>>>>> 2016-11-15 10:03 GMT+08:00 Jungtaek Lim <ka...@gmail.com>:
> >>>>>>>>>
> >>>>>>>>>> Some issues on Storm SQL are resolved but not documented yet.
> >>> I'll
> >>>>>>> file
> >>>>>>>> an
> >>>>>>>>>> issue and assign to 1.1.0 release epic.
> >>>>>>>>>> And also I want to address dropping aggregation and join on
> Storm
> >>>> SQL
> >>>>>>>>>> Trident mode before releasing. I'll assign it too.
> >>>>>>>>>>
> >>>>>>>>>> - Jungtaek Lim (HeartSaVioR)
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> 2016년 11월 15일 (화) 오전 5:55, P. Taylor Goetz <ptgoetz@gmail.com
> >님이
> >>>> 작성:
> >>>>>>>>>>
> >>>>>>>>>>> I think we’re very close. I would like to confirm that the
> >>>>> 1.x-branch
> >>>>>>>> is
> >>>>>>>>>>> not affected by STORM-2176.
> >>>>>>>>>>>
> >>>>>>>>>>> The worker lifecycle API was added in 1.0, but doesn’t work in
> >>> any
> >>>>>>>>>>> released version due to STORM-2176.
> >>>>>>>>>>>
> >>>>>>>>>>> If there are any other open JIRAs that anyone is passionate
> >>> about,
> >>>>>>> now
> >>>>>>>>>>> would be a good time to assign them to the 1.1.0 release epic
> >>>>>>>>>> (STORM-1856).
> >>>>>>>>>>>
> >>>>>>>>>>> -Taylor
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>> On Oct 27, 2016, at 12:19 PM, Jungtaek Lim <kabhwan@gmail.com
> >
> >>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Finally Pacemaker H/A, Supervisor V2, and Storm SQL PRs which
> >>>> were
> >>>>>>>>>> opened
> >>>>>>>>>>>> at the last mail (4 weeks ago) are all merged to 1.x branch.
> >>>>>>>>>>>>
> >>>>>>>>>>>> There're some more PRs on Storm SQL opened, but given that we
> >>> can
> >>>>>>>>>> release
> >>>>>>>>>>>> new minor at any time when we feel it's enough change, I can
> >>> wait
> >>>>>>> for
> >>>>>>>>>> it.
> >>>>>>>>>>>> They didn't get reviewed yet indeed.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Is there something else we would want to include it to 1.1.0?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
> >>>>>>>>>>>>
> >>>>>>>>>>>> 2016년 10월 1일 (토) 오전 9:30, Jungtaek Lim <ka...@gmail.com>님이
> >>> 작성:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Personally, merging and porting back to three branches are
> >>>> painful
> >>>>>>>>>>> enough,
> >>>>>>>>>>>>> especially we don't have merging script and having verbose
> >>>> process
> >>>>>>> (I
> >>>>>>>>>>> mean
> >>>>>>>>>>>>> CHANGELOG).
> >>>>>>>>>>>>> It would be better if merging process is automated (by
> running
> >>>>>>> script
> >>>>>>>>>> or
> >>>>>>>>>>>>> so), so I'd +1 to revisit Harsha's suggestion (adopting Kafka
> >>>>> merge
> >>>>>>>>>>> script)
> >>>>>>>>>>>>> and modify script to fit to Storm.
> >>>>>>>>>>>>> (It will not work if it's the case we need to handle PRs for
> >>>> each
> >>>>>>>>>>> version
> >>>>>>>>>>>>> line, since 'Close' in commit log doesn't close the PR if its
> >>>>>>> target
> >>>>>>>>>>> branch
> >>>>>>>>>>>>> is not master.)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Anyway, without automation I don't want to maintain more
> >>> version
> >>>>>>>>>> lines.
> >>>>>>>>>>>>> I'm looking at the announces from other projects, and others
> >>> are
> >>>>>>> only
> >>>>>>>>>>>>> maintaining two version lines.
> >>>>>>>>>>>>> Since we maintain 2.0.0 version line we can't reduce version
> >>>> lines
> >>>>>>> to
> >>>>>>>>>> 2,
> >>>>>>>>>>>>> but hopefully at most 3.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Btw, let's check pending pull requests and enumerate which
> can
> >>>> be
> >>>>>>>>>>> included
> >>>>>>>>>>>>> in 1.0.0, and start/finish review and merge them soon.
> >>>>>>>>>>>>> For me Supervisor V2 and Pacemaker H/A, and pending Storm SQL
> >>>> PRs
> >>>>>>> can
> >>>>>>>>>> be
> >>>>>>>>>>>>> included, since they are small or in reviewing and expected
> to
> >>>>> pass
> >>>>>>>>>>> review
> >>>>>>>>>>>>> phase soon.
> >>>>>>>>>>>>> (And some small PRs. There're other valuable PRs in PR list
> >>> but
> >>>>> I'm
> >>>>>>>>>> not
> >>>>>>>>>>>>> sure we can review them soon. One example is unified API.)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> One issue which is not clear is STORM-2006
> >>>>>>>>>>>>> <https://github.com/apache/storm/pull/1595>. This is a
> >>>> candidate
> >>>>>>> for
> >>>>>>>>>>> me,
> >>>>>>>>>>>>> but gets blocked while reviewing. If we plan to put great
> >>> effort
> >>>>> to
> >>>>>>>>>>> revise
> >>>>>>>>>>>>> Metric we can skip this.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Please enumerate other PRs as well if you want to include in
> >>>>> 1.1.0.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> Jungtaek Lim
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2016년 9월 30일 (금) 오후 11:09, Bobby Evans
> >>>>> <evans@yahoo-inc.com.invalid
> >>>>>>>>
> >>>>>>>>>> 님이
> >>>>>>>>>>> 작성:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Sounds good to me.  It would be nice to get some of the new
> >>>>>>> features
> >>>>>>>>>>> out.
> >>>>>>>>>>>>> Do we expect to maintain both 1.0.x and 1.1.x lines with bug
> >>>>> fixes?
> >>>>>>>>>>> And if
> >>>>>>>>>>>>> so for how long do we want to do this for? - Bobby
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thursday, September 29, 2016 7:35 PM, Jungtaek Lim <
> >>>>>>>>>>>>> kabhwan@gmail.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi devs,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> It's been 5 months after releasing Storm 1.0.0, and now 1.x
> >>>> branch
> >>>>>>>> has
> >>>>>>>>>>> lots
> >>>>>>>>>>>>> of CHANGELOG and also pending reviews.
> >>>>>>>>>>>>> It's also been a long time after 1.1.0 RC1 is canceled.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I think it may be good to put some efforts to review and
> merge
> >>>>>>>> pending
> >>>>>>>>>>> pull
> >>>>>>>>>>>>> requests (except things which takes time to review and test),
> >>>> and
> >>>>>>>>>>> release
> >>>>>>>>>>>>> 1.1.0 soon.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I'm also open to volunteer release manager for 1.1.0 after we
> >>>>>>>> document
> >>>>>>>>>>> the
> >>>>>>>>>>>>> process of official release.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >
>

Re: [DISCUSS] Release Storm 1.1.0

Posted by Jungtaek Lim <ka...@gmail.com>.

There seems some pull requests for bugfix/improvement on
storm-kafka-client, and some authors in PRs are not availble for now.
(waiting 7 days)

If we plan to get 1.1.1 out soon (say 1 month later or even closer) we can
postpone, but if not, it might be better to coordinate these things ASAP
and include to 1.1.0.

There seems to be other small PRs, but nothing seems critical so it would
be OK to not wait for merging.

- Jungtaek Lim

2017년 2월 9일 (목) 오전 6:48, P. Taylor Goetz <pt...@gmail.com>님이 작성:

Right now we’re down to 1 open issue on the 1.1.0 release epic: STORM-2250
which is under active review/discussion.

Assuming that is mergeable in the near future, are there any other open
issues that should be considered for this release?

-Taylor


> On Feb 2, 2017, at 4:48 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
>
> Thanks for putting this list together Jungtaek. I added a few to the 1.1
release epic that I think are important. Feel free to do the same.
>
> Looks like we have a few to go, but there are pull requests for them.
It’s mostly just a matter of reviews and review responses, so I think we
are close.
>
> -Taylor
>
>> On Feb 2, 2017, at 1:41 AM, Jungtaek Lim <ka...@gmail.com> wrote:
>>
>> Seems like there're not blockers for 1.1.0, but some pull requests are
>> worth to check.
>> There're pending pull requests for storm-kafka-client waited on
STORM-2225.
>> Given that STORM-2225 is now merged, we might need to take a look at.
>>
>> *- reviewing*
>>
>> [storm-core]
>>
>>> STORM-2324 : Fix deployment failure if resources directory is missing in
>> topology jar
>> (master) https://github.com/apache/storm/pull/1908
>> (1.x) https://github.com/apache/storm/pull/1898
>>
>>> STORM-2321 Handle blobstore zk key deletion in KeySequenceNumber
>> (master) https://github.com/apache/storm/pull/1904
>> (1.x) https://github.com/apache/storm/pull/1905
>>
>> [storm-kafka]
>>
>>> STORM-2270 Kafka spout should consume from latest when ZK partition
>> commit offset bigger than the latest offset
>> (1.x) https://github.com/apache/storm/pull/1851
>>
>> [storm-kafka-client]
>>
>>> STORM-2281: Running Multiple Kafka Spouts (Trident) Throws Illegal State
>> Exception
>> (1.x) https://github.com/apache/storm/pull/1902
>>
>>> STORM-2315 Storm kafka client does not commit offsets when ack is
disabled
>> (1.x) https://github.com/apache/storm/pull/1891
>>
>>> fix: KafkaSpout is blocked in AutoCommitMode
>> (master) https://github.com/apache/storm/pull/1863
>>
>>> STORM-2250: Kafka Spout Refactoring to Increase Modularity and
Testability
>> (master) https://github.com/apache/storm/pull/1832
>>
>>> STORM-2014: Put logic around dropping messages into RetryService, remove
>> maxRetry setting from new KafkaSpout
>> (master) https://github.com/apache/storm/pull/1605
>>
>>> fix NullPointException with acked.get(rtp)
>> (master) https://github.com/apache/storm/pull/1807
>>
>> [storm-sql]
>>
>>> STORM-1443 [Storm SQL] Support customizing parallelism in StormSQL
>> https://github.com/apache/storm/pull/1739
>>
>> *- pending*
>>
>> [storm-kafka-client]
>>
>>> STORM-2296 Kafka spout no dup on leader changes
>> (1.0.x) https://github.com/apache/storm/pull/1873
>> (1.x) https://github.com/apache/storm/pull/1888
>>
>> [storm-sql]
>>
>>> STORM-2148 [Storm SQL] Trident mode: back to code generate and compile
>> Trident topology
>> https://github.com/apache/storm/pull/1743
>>
>> Thanks,
>> Jungtaek Lim (HeartSaVioR)
>>
>> 2017년 2월 2일 (목) 오전 8:14, Harsha Chintalapani <st...@harsha.io>님이 작성:
>>
>>> Trying to check the status on this release of 1.1.0. Are we going to do
>>> this release anytime soon?
>>>
>>>
>>> On Fri, Jan 13, 2017 at 7:50 PM S G <sg...@gmail.com> wrote:
>>>
>>>> Not sure if its a little late to include for the 1.1.0 and 1.0.3
releases
>>>> now, but can we consider using zookeeper 3.4.9 for the future versions
as
>>>> 3.4.9 brings in a lot of stability improvements (
>>>> http://zookeeper.apache.org/releases.html) and storm is still using
>>> 3.4.6
>>>> (
>>>> https://github.com/apache/storm/blob/master/pom.xml)
>>>>
>>>> On Fri, Jan 6, 2017 at 11:54 AM, P. Taylor Goetz <pt...@gmail.com>
>>>> wrote:
>>>>
>>>>> Thanks for the update Jungtaek.
>>>>>
>>>>> I’m verifying the patches now. And they should be mergeable shortly.
>>>>>
>>>>> I think we’ll likely be ready for 1.1.0 and 1.0.3 releases next week.
>>>>>
>>>>> -Taylor
>>>>>
>>>>>> On Jan 6, 2017, at 3:40 AM, Jungtaek Lim <ka...@gmail.com> wrote:
>>>>>>
>>>>>> I just submitted a patch for STORM-2176
>>>>>> <https://issues.apache.org/jira/browse/STORM-2176>. Since it's a
>>> small
>>>>> fix,
>>>>>> we just need to handle STORM-2228
>>>>>> <https://issues.apache.org/jira/browse/STORM-2228> to release.
>>>>>>
>>>>>> - Jungtaek Lim (HeartSaVioR)
>>>>>>
>>>>>> 2017년 1월 5일 (목) 오후 1:43, Jungtaek Lim <ka...@gmail.com>님이 작성:
>>>>>>
>>>>>>> Recently we receive some requests regarding release Storm 1.0.3, so
>>>>> would
>>>>>>> like to bump this again.
>>>>>>>
>>>>>>> Given that blocker issues for Storm 1.1.0 are also blocker for
>>> 1.0.3,
>>>>> I'd
>>>>>>> like to ask a favor of taking care of 'open' / 'in progress' issues
>>> on
>>>>>>> 1.1.0 epic.
>>>>>>> https://issues.apache.org/jira/browse/STORM-1856
>>>>>>>
>>>>>>> There're one 'open' issue and three 'in progress' issues. Two of
>>> three
>>>>> 'in
>>>>>>> progress' issues are tiny fix so easy to be handled, so actual
>>>> blockers
>>>>> are
>>>>>>> STORM-2176 <https://issues.apache.org/jira/browse/STORM-2176> and
>>>>>>> STORM-2228 <https://issues.apache.org/jira/browse/STORM-2228>.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>
>>>>>>> 2016년 11월 17일 (목) 오후 11:41, Satish Duggana <
>>> satish.duggana@gmail.com
>>>>> 님이
>>>>>>> 작성:
>>>>>>>
>>>>>>>  STORM-2205: Race condition in getting nimbus summaries while ZK
>>>>>>> connections are reconnected.
>>>>>>>
>>>>>>> This issue seems to occur in our environments and I would like this
>>> to
>>>>> be
>>>>>>> part of 1.1.0.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Satish.
>>>>>>>
>>>>>>> On Thu, Nov 17, 2016 at 9:36 AM, Jungtaek Lim <ka...@gmail.com>
>>>>> wrote:
>>>>>>>
>>>>>>>> I have no idea on storm-kafka-client, but some bugfix issues for
>>>>>>>> storm-kafka-client are waiting for reviewing / merging.
>>>>>>>>
>>>>>>>> STORM-2014 <https://issues.apache.org/jira/browse/STORM-2014>
>>>>>>>> STORM-2087 <https://issues.apache.org/jira/browse/STORM-2087>
>>>>>>>> STORM-2104 <https://issues.apache.org/jira/browse/STORM-2104>
>>>>>>>>
>>>>>>>> If someone can review them in several days it would be great.
>>>>>>>>
>>>>>>>> I hope that we include currently opened pull requests for Storm SQL
>>>> so
>>>>>>> that
>>>>>>>> we can release 'usable Storm SQL' more usable, but I'm also OK to
>>>>>>> postpone
>>>>>>>> them to be included to next release if they drag the release.
>>>>>>>>
>>>>>>>> STORM-1446 <http://issues.apache.org/jira/browse/STORM-1446>
>>>>>>>> STORM-1443 <https://issues.apache.org/jira/browse/STORM-1443>
>>>>>>>> STORM-2148 <https://issues.apache.org/jira/browse/STORM-2148>
>>>>>>>> STORM-2170 <https://issues.apache.org/jira/browse/STORM-2170>
>>>>>>>>
>>>>>>>> I can see some pull requests which address Trident implementations
>>>> for
>>>>>>>> storm-kafka-client, storm-mongodb, storm-cassandra.
>>>>>>>>
>>>>>>>> storm-kafka-client: STORM-1694
>>>>>>>> <https://issues.apache.org/jira/browse/STORM-1694> (patch for 2.0
>>> is
>>>>>>>> merged, patch for 1.x is ready for reviewing)
>>>>>>>> storm-cassandra: STORM-1369
>>>>>>>> <https://issues.apache.org/jira/browse/STORM-1369>
>>>>>>>> storm-mongodb: STORM-1607 <https://issues.apache.org/
>>>>>>>> jira/browse/STORM-1607>
>>>>>>>>
>>>>>>>> If we want to cut the release now, we could include only bugfix
>>>> issues
>>>>>>> and
>>>>>>>> postpone others. Otherwise we could discuss and include some or all
>>>> of
>>>>>>> the
>>>>>>>> above.
>>>>>>>>
>>>>>>>> What do you think? When we want to start the release process for
>>>> 1.1.0?
>>>>>>>>
>>>>>>>> - Jungtaek Lim (HeartSaVioR)
>>>>>>>>
>>>>>>>> 2016년 11월 16일 (수) 오전 4:11, P. Taylor Goetz <pt...@gmail.com>님이
>>> 작성:
>>>>>>>>
>>>>>>>> Thanks Xin, I added it to the 1.1.0 epic.
>>>>>>>>
>>>>>>>> -Taylor
>>>>>>>>
>>>>>>>>> On Nov 15, 2016, at 9:01 AM, Xin Wang <da...@gmail.com>
>>>> wrote:
>>>>>>>>>
>>>>>>>>> STORM-2198 ( PR: https://github.com/apache/storm/pull/1773 )
>>> fixes
>>>> a
>>>>>>> bug
>>>>>>>> of
>>>>>>>>> storm-hdfs. Do we have a consideration to include this?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Xin Wang (vesense)
>>>>>>>>>
>>>>>>>>> 2016-11-15 10:03 GMT+08:00 Jungtaek Lim <ka...@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Some issues on Storm SQL are resolved but not documented yet.
>>> I'll
>>>>>>> file
>>>>>>>> an
>>>>>>>>>> issue and assign to 1.1.0 release epic.
>>>>>>>>>> And also I want to address dropping aggregation and join on Storm
>>>> SQL
>>>>>>>>>> Trident mode before releasing. I'll assign it too.
>>>>>>>>>>
>>>>>>>>>> - Jungtaek Lim (HeartSaVioR)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2016년 11월 15일 (화) 오전 5:55, P. Taylor Goetz <pt...@gmail.com>님이
>>>> 작성:
>>>>>>>>>>
>>>>>>>>>>> I think we’re very close. I would like to confirm that the
>>>>> 1.x-branch
>>>>>>>> is
>>>>>>>>>>> not affected by STORM-2176.
>>>>>>>>>>>
>>>>>>>>>>> The worker lifecycle API was added in 1.0, but doesn’t work in
>>> any
>>>>>>>>>>> released version due to STORM-2176.
>>>>>>>>>>>
>>>>>>>>>>> If there are any other open JIRAs that anyone is passionate
>>> about,
>>>>>>> now
>>>>>>>>>>> would be a good time to assign them to the 1.1.0 release epic
>>>>>>>>>> (STORM-1856).
>>>>>>>>>>>
>>>>>>>>>>> -Taylor
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> On Oct 27, 2016, at 12:19 PM, Jungtaek Lim <ka...@gmail.com>
>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Finally Pacemaker H/A, Supervisor V2, and Storm SQL PRs which
>>>> were
>>>>>>>>>> opened
>>>>>>>>>>>> at the last mail (4 weeks ago) are all merged to 1.x branch.
>>>>>>>>>>>>
>>>>>>>>>>>> There're some more PRs on Storm SQL opened, but given that we
>>> can
>>>>>>>>>> release
>>>>>>>>>>>> new minor at any time when we feel it's enough change, I can
>>> wait
>>>>>>> for
>>>>>>>>>> it.
>>>>>>>>>>>> They didn't get reviewed yet indeed.
>>>>>>>>>>>>
>>>>>>>>>>>> Is there something else we would want to include it to 1.1.0?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>>>>>
>>>>>>>>>>>> 2016년 10월 1일 (토) 오전 9:30, Jungtaek Lim <ka...@gmail.com>님이
>>> 작성:
>>>>>>>>>>>>
>>>>>>>>>>>>> Personally, merging and porting back to three branches are
>>>> painful
>>>>>>>>>>> enough,
>>>>>>>>>>>>> especially we don't have merging script and having verbose
>>>> process
>>>>>>> (I
>>>>>>>>>>> mean
>>>>>>>>>>>>> CHANGELOG).
>>>>>>>>>>>>> It would be better if merging process is automated (by running
>>>>>>> script
>>>>>>>>>> or
>>>>>>>>>>>>> so), so I'd +1 to revisit Harsha's suggestion (adopting Kafka
>>>>> merge
>>>>>>>>>>> script)
>>>>>>>>>>>>> and modify script to fit to Storm.
>>>>>>>>>>>>> (It will not work if it's the case we need to handle PRs for
>>>> each
>>>>>>>>>>> version
>>>>>>>>>>>>> line, since 'Close' in commit log doesn't close the PR if its
>>>>>>> target
>>>>>>>>>>> branch
>>>>>>>>>>>>> is not master.)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Anyway, without automation I don't want to maintain more
>>> version
>>>>>>>>>> lines.
>>>>>>>>>>>>> I'm looking at the announces from other projects, and others
>>> are
>>>>>>> only
>>>>>>>>>>>>> maintaining two version lines.
>>>>>>>>>>>>> Since we maintain 2.0.0 version line we can't reduce version
>>>> lines
>>>>>>> to
>>>>>>>>>> 2,
>>>>>>>>>>>>> but hopefully at most 3.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Btw, let's check pending pull requests and enumerate which can
>>>> be
>>>>>>>>>>> included
>>>>>>>>>>>>> in 1.0.0, and start/finish review and merge them soon.
>>>>>>>>>>>>> For me Supervisor V2 and Pacemaker H/A, and pending Storm SQL
>>>> PRs
>>>>>>> can
>>>>>>>>>> be
>>>>>>>>>>>>> included, since they are small or in reviewing and expected to
>>>>> pass
>>>>>>>>>>> review
>>>>>>>>>>>>> phase soon.
>>>>>>>>>>>>> (And some small PRs. There're other valuable PRs in PR list
>>> but
>>>>> I'm
>>>>>>>>>> not
>>>>>>>>>>>>> sure we can review them soon. One example is unified API.)
>>>>>>>>>>>>>
>>>>>>>>>>>>> One issue which is not clear is STORM-2006
>>>>>>>>>>>>> <https://github.com/apache/storm/pull/1595>. This is a
>>>> candidate
>>>>>>> for
>>>>>>>>>>> me,
>>>>>>>>>>>>> but gets blocked while reviewing. If we plan to put great
>>> effort
>>>>> to
>>>>>>>>>>> revise
>>>>>>>>>>>>> Metric we can skip this.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please enumerate other PRs as well if you want to include in
>>>>> 1.1.0.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Jungtaek Lim
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2016년 9월 30일 (금) 오후 11:09, Bobby Evans
>>>>> <evans@yahoo-inc.com.invalid
>>>>>>>>
>>>>>>>>>> 님이
>>>>>>>>>>> 작성:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sounds good to me.  It would be nice to get some of the new
>>>>>>> features
>>>>>>>>>>> out.
>>>>>>>>>>>>> Do we expect to maintain both 1.0.x and 1.1.x lines with bug
>>>>> fixes?
>>>>>>>>>>> And if
>>>>>>>>>>>>> so for how long do we want to do this for? - Bobby
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thursday, September 29, 2016 7:35 PM, Jungtaek Lim <
>>>>>>>>>>>>> kabhwan@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi devs,
>>>>>>>>>>>>>
>>>>>>>>>>>>> It's been 5 months after releasing Storm 1.0.0, and now 1.x
>>>> branch
>>>>>>>> has
>>>>>>>>>>> lots
>>>>>>>>>>>>> of CHANGELOG and also pending reviews.
>>>>>>>>>>>>> It's also been a long time after 1.1.0 RC1 is canceled.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think it may be good to put some efforts to review and merge
>>>>>>>> pending
>>>>>>>>>>> pull
>>>>>>>>>>>>> requests (except things which takes time to review and test),
>>>> and
>>>>>>>>>>> release
>>>>>>>>>>>>> 1.1.0 soon.
>>>>>>>>>>>>>
>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm also open to volunteer release manager for 1.1.0 after we
>>>>>>>> document
>>>>>>>>>>> the
>>>>>>>>>>>>> process of official release.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>>
>

Re: [DISCUSS] Release Storm 1.1.0

Posted by "P. Taylor Goetz" <pt...@gmail.com>.

Right now we’re down to 1 open issue on the 1.1.0 release epic: STORM-2250 which is under active review/discussion.

Assuming that is mergeable in the near future, are there any other open issues that should be considered for this release?

-Taylor


> On Feb 2, 2017, at 4:48 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
> 
> Thanks for putting this list together Jungtaek. I added a few to the 1.1 release epic that I think are important. Feel free to do the same.
> 
> Looks like we have a few to go, but there are pull requests for them. It’s mostly just a matter of reviews and review responses, so I think we are close.
> 
> -Taylor
> 
>> On Feb 2, 2017, at 1:41 AM, Jungtaek Lim <ka...@gmail.com> wrote:
>> 
>> Seems like there're not blockers for 1.1.0, but some pull requests are
>> worth to check.
>> There're pending pull requests for storm-kafka-client waited on STORM-2225.
>> Given that STORM-2225 is now merged, we might need to take a look at.
>> 
>> *- reviewing*
>> 
>> [storm-core]
>> 
>>> STORM-2324 : Fix deployment failure if resources directory is missing in
>> topology jar
>> (master) https://github.com/apache/storm/pull/1908
>> (1.x) https://github.com/apache/storm/pull/1898
>> 
>>> STORM-2321 Handle blobstore zk key deletion in KeySequenceNumber
>> (master) https://github.com/apache/storm/pull/1904
>> (1.x) https://github.com/apache/storm/pull/1905
>> 
>> [storm-kafka]
>> 
>>> STORM-2270 Kafka spout should consume from latest when ZK partition
>> commit offset bigger than the latest offset
>> (1.x) https://github.com/apache/storm/pull/1851
>> 
>> [storm-kafka-client]
>> 
>>> STORM-2281: Running Multiple Kafka Spouts (Trident) Throws Illegal State
>> Exception
>> (1.x) https://github.com/apache/storm/pull/1902
>> 
>>> STORM-2315 Storm kafka client does not commit offsets when ack is disabled
>> (1.x) https://github.com/apache/storm/pull/1891
>> 
>>> fix: KafkaSpout is blocked in AutoCommitMode
>> (master) https://github.com/apache/storm/pull/1863
>> 
>>> STORM-2250: Kafka Spout Refactoring to Increase Modularity and Testability
>> (master) https://github.com/apache/storm/pull/1832
>> 
>>> STORM-2014: Put logic around dropping messages into RetryService, remove
>> maxRetry setting from new KafkaSpout
>> (master) https://github.com/apache/storm/pull/1605
>> 
>>> fix NullPointException with acked.get(rtp)
>> (master) https://github.com/apache/storm/pull/1807
>> 
>> [storm-sql]
>> 
>>> STORM-1443 [Storm SQL] Support customizing parallelism in StormSQL
>> https://github.com/apache/storm/pull/1739
>> 
>> *- pending*
>> 
>> [storm-kafka-client]
>> 
>>> STORM-2296 Kafka spout no dup on leader changes
>> (1.0.x) https://github.com/apache/storm/pull/1873
>> (1.x) https://github.com/apache/storm/pull/1888
>> 
>> [storm-sql]
>> 
>>> STORM-2148 [Storm SQL] Trident mode: back to code generate and compile
>> Trident topology
>> https://github.com/apache/storm/pull/1743
>> 
>> Thanks,
>> Jungtaek Lim (HeartSaVioR)
>> 
>> 2017년 2월 2일 (목) 오전 8:14, Harsha Chintalapani <st...@harsha.io>님이 작성:
>> 
>>> Trying to check the status on this release of 1.1.0. Are we going to do
>>> this release anytime soon?
>>> 
>>> 
>>> On Fri, Jan 13, 2017 at 7:50 PM S G <sg...@gmail.com> wrote:
>>> 
>>>> Not sure if its a little late to include for the 1.1.0 and 1.0.3 releases
>>>> now, but can we consider using zookeeper 3.4.9 for the future versions as
>>>> 3.4.9 brings in a lot of stability improvements (
>>>> http://zookeeper.apache.org/releases.html) and storm is still using
>>> 3.4.6
>>>> (
>>>> https://github.com/apache/storm/blob/master/pom.xml)
>>>> 
>>>> On Fri, Jan 6, 2017 at 11:54 AM, P. Taylor Goetz <pt...@gmail.com>
>>>> wrote:
>>>> 
>>>>> Thanks for the update Jungtaek.
>>>>> 
>>>>> I’m verifying the patches now. And they should be mergeable shortly.
>>>>> 
>>>>> I think we’ll likely be ready for 1.1.0 and 1.0.3 releases next week.
>>>>> 
>>>>> -Taylor
>>>>> 
>>>>>> On Jan 6, 2017, at 3:40 AM, Jungtaek Lim <ka...@gmail.com> wrote:
>>>>>> 
>>>>>> I just submitted a patch for STORM-2176
>>>>>> <https://issues.apache.org/jira/browse/STORM-2176>. Since it's a
>>> small
>>>>> fix,
>>>>>> we just need to handle STORM-2228
>>>>>> <https://issues.apache.org/jira/browse/STORM-2228> to release.
>>>>>> 
>>>>>> - Jungtaek Lim (HeartSaVioR)
>>>>>> 
>>>>>> 2017년 1월 5일 (목) 오후 1:43, Jungtaek Lim <ka...@gmail.com>님이 작성:
>>>>>> 
>>>>>>> Recently we receive some requests regarding release Storm 1.0.3, so
>>>>> would
>>>>>>> like to bump this again.
>>>>>>> 
>>>>>>> Given that blocker issues for Storm 1.1.0 are also blocker for
>>> 1.0.3,
>>>>> I'd
>>>>>>> like to ask a favor of taking care of 'open' / 'in progress' issues
>>> on
>>>>>>> 1.1.0 epic.
>>>>>>> https://issues.apache.org/jira/browse/STORM-1856
>>>>>>> 
>>>>>>> There're one 'open' issue and three 'in progress' issues. Two of
>>> three
>>>>> 'in
>>>>>>> progress' issues are tiny fix so easy to be handled, so actual
>>>> blockers
>>>>> are
>>>>>>> STORM-2176 <https://issues.apache.org/jira/browse/STORM-2176> and
>>>>>>> STORM-2228 <https://issues.apache.org/jira/browse/STORM-2228>.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>> 
>>>>>>> 2016년 11월 17일 (목) 오후 11:41, Satish Duggana <
>>> satish.duggana@gmail.com
>>>>> 님이
>>>>>>> 작성:
>>>>>>> 
>>>>>>>  STORM-2205: Race condition in getting nimbus summaries while ZK
>>>>>>> connections are reconnected.
>>>>>>> 
>>>>>>> This issue seems to occur in our environments and I would like this
>>> to
>>>>> be
>>>>>>> part of 1.1.0.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Satish.
>>>>>>> 
>>>>>>> On Thu, Nov 17, 2016 at 9:36 AM, Jungtaek Lim <ka...@gmail.com>
>>>>> wrote:
>>>>>>> 
>>>>>>>> I have no idea on storm-kafka-client, but some bugfix issues for
>>>>>>>> storm-kafka-client are waiting for reviewing / merging.
>>>>>>>> 
>>>>>>>> STORM-2014 <https://issues.apache.org/jira/browse/STORM-2014>
>>>>>>>> STORM-2087 <https://issues.apache.org/jira/browse/STORM-2087>
>>>>>>>> STORM-2104 <https://issues.apache.org/jira/browse/STORM-2104>
>>>>>>>> 
>>>>>>>> If someone can review them in several days it would be great.
>>>>>>>> 
>>>>>>>> I hope that we include currently opened pull requests for Storm SQL
>>>> so
>>>>>>> that
>>>>>>>> we can release 'usable Storm SQL' more usable, but I'm also OK to
>>>>>>> postpone
>>>>>>>> them to be included to next release if they drag the release.
>>>>>>>> 
>>>>>>>> STORM-1446 <http://issues.apache.org/jira/browse/STORM-1446>
>>>>>>>> STORM-1443 <https://issues.apache.org/jira/browse/STORM-1443>
>>>>>>>> STORM-2148 <https://issues.apache.org/jira/browse/STORM-2148>
>>>>>>>> STORM-2170 <https://issues.apache.org/jira/browse/STORM-2170>
>>>>>>>> 
>>>>>>>> I can see some pull requests which address Trident implementations
>>>> for
>>>>>>>> storm-kafka-client, storm-mongodb, storm-cassandra.
>>>>>>>> 
>>>>>>>> storm-kafka-client: STORM-1694
>>>>>>>> <https://issues.apache.org/jira/browse/STORM-1694> (patch for 2.0
>>> is
>>>>>>>> merged, patch for 1.x is ready for reviewing)
>>>>>>>> storm-cassandra: STORM-1369
>>>>>>>> <https://issues.apache.org/jira/browse/STORM-1369>
>>>>>>>> storm-mongodb: STORM-1607 <https://issues.apache.org/
>>>>>>>> jira/browse/STORM-1607>
>>>>>>>> 
>>>>>>>> If we want to cut the release now, we could include only bugfix
>>>> issues
>>>>>>> and
>>>>>>>> postpone others. Otherwise we could discuss and include some or all
>>>> of
>>>>>>> the
>>>>>>>> above.
>>>>>>>> 
>>>>>>>> What do you think? When we want to start the release process for
>>>> 1.1.0?
>>>>>>>> 
>>>>>>>> - Jungtaek Lim (HeartSaVioR)
>>>>>>>> 
>>>>>>>> 2016년 11월 16일 (수) 오전 4:11, P. Taylor Goetz <pt...@gmail.com>님이
>>> 작성:
>>>>>>>> 
>>>>>>>> Thanks Xin, I added it to the 1.1.0 epic.
>>>>>>>> 
>>>>>>>> -Taylor
>>>>>>>> 
>>>>>>>>> On Nov 15, 2016, at 9:01 AM, Xin Wang <da...@gmail.com>
>>>> wrote:
>>>>>>>>> 
>>>>>>>>> STORM-2198 ( PR: https://github.com/apache/storm/pull/1773 )
>>> fixes
>>>> a
>>>>>>> bug
>>>>>>>> of
>>>>>>>>> storm-hdfs. Do we have a consideration to include this?
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Xin Wang (vesense)
>>>>>>>>> 
>>>>>>>>> 2016-11-15 10:03 GMT+08:00 Jungtaek Lim <ka...@gmail.com>:
>>>>>>>>> 
>>>>>>>>>> Some issues on Storm SQL are resolved but not documented yet.
>>> I'll
>>>>>>> file
>>>>>>>> an
>>>>>>>>>> issue and assign to 1.1.0 release epic.
>>>>>>>>>> And also I want to address dropping aggregation and join on Storm
>>>> SQL
>>>>>>>>>> Trident mode before releasing. I'll assign it too.
>>>>>>>>>> 
>>>>>>>>>> - Jungtaek Lim (HeartSaVioR)
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 2016년 11월 15일 (화) 오전 5:55, P. Taylor Goetz <pt...@gmail.com>님이
>>>> 작성:
>>>>>>>>>> 
>>>>>>>>>>> I think we’re very close. I would like to confirm that the
>>>>> 1.x-branch
>>>>>>>> is
>>>>>>>>>>> not affected by STORM-2176.
>>>>>>>>>>> 
>>>>>>>>>>> The worker lifecycle API was added in 1.0, but doesn’t work in
>>> any
>>>>>>>>>>> released version due to STORM-2176.
>>>>>>>>>>> 
>>>>>>>>>>> If there are any other open JIRAs that anyone is passionate
>>> about,
>>>>>>> now
>>>>>>>>>>> would be a good time to assign them to the 1.1.0 release epic
>>>>>>>>>> (STORM-1856).
>>>>>>>>>>> 
>>>>>>>>>>> -Taylor
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> On Oct 27, 2016, at 12:19 PM, Jungtaek Lim <ka...@gmail.com>
>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Finally Pacemaker H/A, Supervisor V2, and Storm SQL PRs which
>>>> were
>>>>>>>>>> opened
>>>>>>>>>>>> at the last mail (4 weeks ago) are all merged to 1.x branch.
>>>>>>>>>>>> 
>>>>>>>>>>>> There're some more PRs on Storm SQL opened, but given that we
>>> can
>>>>>>>>>> release
>>>>>>>>>>>> new minor at any time when we feel it's enough change, I can
>>> wait
>>>>>>> for
>>>>>>>>>> it.
>>>>>>>>>>>> They didn't get reviewed yet indeed.
>>>>>>>>>>>> 
>>>>>>>>>>>> Is there something else we would want to include it to 1.1.0?
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>>>>> 
>>>>>>>>>>>> 2016년 10월 1일 (토) 오전 9:30, Jungtaek Lim <ka...@gmail.com>님이
>>> 작성:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Personally, merging and porting back to three branches are
>>>> painful
>>>>>>>>>>> enough,
>>>>>>>>>>>>> especially we don't have merging script and having verbose
>>>> process
>>>>>>> (I
>>>>>>>>>>> mean
>>>>>>>>>>>>> CHANGELOG).
>>>>>>>>>>>>> It would be better if merging process is automated (by running
>>>>>>> script
>>>>>>>>>> or
>>>>>>>>>>>>> so), so I'd +1 to revisit Harsha's suggestion (adopting Kafka
>>>>> merge
>>>>>>>>>>> script)
>>>>>>>>>>>>> and modify script to fit to Storm.
>>>>>>>>>>>>> (It will not work if it's the case we need to handle PRs for
>>>> each
>>>>>>>>>>> version
>>>>>>>>>>>>> line, since 'Close' in commit log doesn't close the PR if its
>>>>>>> target
>>>>>>>>>>> branch
>>>>>>>>>>>>> is not master.)
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Anyway, without automation I don't want to maintain more
>>> version
>>>>>>>>>> lines.
>>>>>>>>>>>>> I'm looking at the announces from other projects, and others
>>> are
>>>>>>> only
>>>>>>>>>>>>> maintaining two version lines.
>>>>>>>>>>>>> Since we maintain 2.0.0 version line we can't reduce version
>>>> lines
>>>>>>> to
>>>>>>>>>> 2,
>>>>>>>>>>>>> but hopefully at most 3.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Btw, let's check pending pull requests and enumerate which can
>>>> be
>>>>>>>>>>> included
>>>>>>>>>>>>> in 1.0.0, and start/finish review and merge them soon.
>>>>>>>>>>>>> For me Supervisor V2 and Pacemaker H/A, and pending Storm SQL
>>>> PRs
>>>>>>> can
>>>>>>>>>> be
>>>>>>>>>>>>> included, since they are small or in reviewing and expected to
>>>>> pass
>>>>>>>>>>> review
>>>>>>>>>>>>> phase soon.
>>>>>>>>>>>>> (And some small PRs. There're other valuable PRs in PR list
>>> but
>>>>> I'm
>>>>>>>>>> not
>>>>>>>>>>>>> sure we can review them soon. One example is unified API.)
>>>>>>>>>>>>> 
>>>>>>>>>>>>> One issue which is not clear is STORM-2006
>>>>>>>>>>>>> <https://github.com/apache/storm/pull/1595>. This is a
>>>> candidate
>>>>>>> for
>>>>>>>>>>> me,
>>>>>>>>>>>>> but gets blocked while reviewing. If we plan to put great
>>> effort
>>>>> to
>>>>>>>>>>> revise
>>>>>>>>>>>>> Metric we can skip this.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Please enumerate other PRs as well if you want to include in
>>>>> 1.1.0.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Jungtaek Lim
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 2016년 9월 30일 (금) 오후 11:09, Bobby Evans
>>>>> <evans@yahoo-inc.com.invalid
>>>>>>>> 
>>>>>>>>>> 님이
>>>>>>>>>>> 작성:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Sounds good to me.  It would be nice to get some of the new
>>>>>>> features
>>>>>>>>>>> out.
>>>>>>>>>>>>> Do we expect to maintain both 1.0.x and 1.1.x lines with bug
>>>>> fixes?
>>>>>>>>>>> And if
>>>>>>>>>>>>> so for how long do we want to do this for? - Bobby
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Thursday, September 29, 2016 7:35 PM, Jungtaek Lim <
>>>>>>>>>>>>> kabhwan@gmail.com> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi devs,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> It's been 5 months after releasing Storm 1.0.0, and now 1.x
>>>> branch
>>>>>>>> has
>>>>>>>>>>> lots
>>>>>>>>>>>>> of CHANGELOG and also pending reviews.
>>>>>>>>>>>>> It's also been a long time after 1.1.0 RC1 is canceled.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I think it may be good to put some efforts to review and merge
>>>>>>>> pending
>>>>>>>>>>> pull
>>>>>>>>>>>>> requests (except things which takes time to review and test),
>>>> and
>>>>>>>>>>> release
>>>>>>>>>>>>> 1.1.0 soon.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I'm also open to volunteer release manager for 1.1.0 after we
>>>>>>>> document
>>>>>>>>>>> the
>>>>>>>>>>>>> process of official release.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>

Re: [DISCUSS] Release Storm 1.1.0

Posted by "P. Taylor Goetz" <pt...@gmail.com>.

Thanks for putting this list together Jungtaek. I added a few to the 1.1 release epic that I think are important. Feel free to do the same.

Looks like we have a few to go, but there are pull requests for them. It’s mostly just a matter of reviews and review responses, so I think we are close.

-Taylor

> On Feb 2, 2017, at 1:41 AM, Jungtaek Lim <ka...@gmail.com> wrote:
> 
> Seems like there're not blockers for 1.1.0, but some pull requests are
> worth to check.
> There're pending pull requests for storm-kafka-client waited on STORM-2225.
> Given that STORM-2225 is now merged, we might need to take a look at.
> 
> *- reviewing*
> 
> [storm-core]
> 
>> STORM-2324 : Fix deployment failure if resources directory is missing in
> topology jar
> (master) https://github.com/apache/storm/pull/1908
> (1.x) https://github.com/apache/storm/pull/1898
> 
>> STORM-2321 Handle blobstore zk key deletion in KeySequenceNumber
> (master) https://github.com/apache/storm/pull/1904
> (1.x) https://github.com/apache/storm/pull/1905
> 
> [storm-kafka]
> 
>> STORM-2270 Kafka spout should consume from latest when ZK partition
> commit offset bigger than the latest offset
> (1.x) https://github.com/apache/storm/pull/1851
> 
> [storm-kafka-client]
> 
>> STORM-2281: Running Multiple Kafka Spouts (Trident) Throws Illegal State
> Exception
> (1.x) https://github.com/apache/storm/pull/1902
> 
>> STORM-2315 Storm kafka client does not commit offsets when ack is disabled
> (1.x) https://github.com/apache/storm/pull/1891
> 
>> fix: KafkaSpout is blocked in AutoCommitMode
> (master) https://github.com/apache/storm/pull/1863
> 
>> STORM-2250: Kafka Spout Refactoring to Increase Modularity and Testability
> (master) https://github.com/apache/storm/pull/1832
> 
>> STORM-2014: Put logic around dropping messages into RetryService, remove
> maxRetry setting from new KafkaSpout
> (master) https://github.com/apache/storm/pull/1605
> 
>> fix NullPointException with acked.get(rtp)
> (master) https://github.com/apache/storm/pull/1807
> 
> [storm-sql]
> 
>> STORM-1443 [Storm SQL] Support customizing parallelism in StormSQL
> https://github.com/apache/storm/pull/1739
> 
> *- pending*
> 
> [storm-kafka-client]
> 
>> STORM-2296 Kafka spout no dup on leader changes
> (1.0.x) https://github.com/apache/storm/pull/1873
> (1.x) https://github.com/apache/storm/pull/1888
> 
> [storm-sql]
> 
>> STORM-2148 [Storm SQL] Trident mode: back to code generate and compile
> Trident topology
> https://github.com/apache/storm/pull/1743
> 
> Thanks,
> Jungtaek Lim (HeartSaVioR)
> 
> 2017년 2월 2일 (목) 오전 8:14, Harsha Chintalapani <st...@harsha.io>님이 작성:
> 
>> Trying to check the status on this release of 1.1.0. Are we going to do
>> this release anytime soon?
>> 
>> 
>> On Fri, Jan 13, 2017 at 7:50 PM S G <sg...@gmail.com> wrote:
>> 
>>> Not sure if its a little late to include for the 1.1.0 and 1.0.3 releases
>>> now, but can we consider using zookeeper 3.4.9 for the future versions as
>>> 3.4.9 brings in a lot of stability improvements (
>>> http://zookeeper.apache.org/releases.html) and storm is still using
>> 3.4.6
>>> (
>>> https://github.com/apache/storm/blob/master/pom.xml)
>>> 
>>> On Fri, Jan 6, 2017 at 11:54 AM, P. Taylor Goetz <pt...@gmail.com>
>>> wrote:
>>> 
>>>> Thanks for the update Jungtaek.
>>>> 
>>>> I’m verifying the patches now. And they should be mergeable shortly.
>>>> 
>>>> I think we’ll likely be ready for 1.1.0 and 1.0.3 releases next week.
>>>> 
>>>> -Taylor
>>>> 
>>>>> On Jan 6, 2017, at 3:40 AM, Jungtaek Lim <ka...@gmail.com> wrote:
>>>>> 
>>>>> I just submitted a patch for STORM-2176
>>>>> <https://issues.apache.org/jira/browse/STORM-2176>. Since it's a
>> small
>>>> fix,
>>>>> we just need to handle STORM-2228
>>>>> <https://issues.apache.org/jira/browse/STORM-2228> to release.
>>>>> 
>>>>> - Jungtaek Lim (HeartSaVioR)
>>>>> 
>>>>> 2017년 1월 5일 (목) 오후 1:43, Jungtaek Lim <ka...@gmail.com>님이 작성:
>>>>> 
>>>>>> Recently we receive some requests regarding release Storm 1.0.3, so
>>>> would
>>>>>> like to bump this again.
>>>>>> 
>>>>>> Given that blocker issues for Storm 1.1.0 are also blocker for
>> 1.0.3,
>>>> I'd
>>>>>> like to ask a favor of taking care of 'open' / 'in progress' issues
>> on
>>>>>> 1.1.0 epic.
>>>>>> https://issues.apache.org/jira/browse/STORM-1856
>>>>>> 
>>>>>> There're one 'open' issue and three 'in progress' issues. Two of
>> three
>>>> 'in
>>>>>> progress' issues are tiny fix so easy to be handled, so actual
>>> blockers
>>>> are
>>>>>> STORM-2176 <https://issues.apache.org/jira/browse/STORM-2176> and
>>>>>> STORM-2228 <https://issues.apache.org/jira/browse/STORM-2228>.
>>>>>> 
>>>>>> Thanks,
>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>> 
>>>>>> 2016년 11월 17일 (목) 오후 11:41, Satish Duggana <
>> satish.duggana@gmail.com
>>>> 님이
>>>>>> 작성:
>>>>>> 
>>>>>>   STORM-2205: Race condition in getting nimbus summaries while ZK
>>>>>> connections are reconnected.
>>>>>> 
>>>>>> This issue seems to occur in our environments and I would like this
>> to
>>>> be
>>>>>> part of 1.1.0.
>>>>>> 
>>>>>> Thanks,
>>>>>> Satish.
>>>>>> 
>>>>>> On Thu, Nov 17, 2016 at 9:36 AM, Jungtaek Lim <ka...@gmail.com>
>>>> wrote:
>>>>>> 
>>>>>>> I have no idea on storm-kafka-client, but some bugfix issues for
>>>>>>> storm-kafka-client are waiting for reviewing / merging.
>>>>>>> 
>>>>>>> STORM-2014 <https://issues.apache.org/jira/browse/STORM-2014>
>>>>>>> STORM-2087 <https://issues.apache.org/jira/browse/STORM-2087>
>>>>>>> STORM-2104 <https://issues.apache.org/jira/browse/STORM-2104>
>>>>>>> 
>>>>>>> If someone can review them in several days it would be great.
>>>>>>> 
>>>>>>> I hope that we include currently opened pull requests for Storm SQL
>>> so
>>>>>> that
>>>>>>> we can release 'usable Storm SQL' more usable, but I'm also OK to
>>>>>> postpone
>>>>>>> them to be included to next release if they drag the release.
>>>>>>> 
>>>>>>> STORM-1446 <http://issues.apache.org/jira/browse/STORM-1446>
>>>>>>> STORM-1443 <https://issues.apache.org/jira/browse/STORM-1443>
>>>>>>> STORM-2148 <https://issues.apache.org/jira/browse/STORM-2148>
>>>>>>> STORM-2170 <https://issues.apache.org/jira/browse/STORM-2170>
>>>>>>> 
>>>>>>> I can see some pull requests which address Trident implementations
>>> for
>>>>>>> storm-kafka-client, storm-mongodb, storm-cassandra.
>>>>>>> 
>>>>>>> storm-kafka-client: STORM-1694
>>>>>>> <https://issues.apache.org/jira/browse/STORM-1694> (patch for 2.0
>> is
>>>>>>> merged, patch for 1.x is ready for reviewing)
>>>>>>> storm-cassandra: STORM-1369
>>>>>>> <https://issues.apache.org/jira/browse/STORM-1369>
>>>>>>> storm-mongodb: STORM-1607 <https://issues.apache.org/
>>>>>>> jira/browse/STORM-1607>
>>>>>>> 
>>>>>>> If we want to cut the release now, we could include only bugfix
>>> issues
>>>>>> and
>>>>>>> postpone others. Otherwise we could discuss and include some or all
>>> of
>>>>>> the
>>>>>>> above.
>>>>>>> 
>>>>>>> What do you think? When we want to start the release process for
>>> 1.1.0?
>>>>>>> 
>>>>>>> - Jungtaek Lim (HeartSaVioR)
>>>>>>> 
>>>>>>> 2016년 11월 16일 (수) 오전 4:11, P. Taylor Goetz <pt...@gmail.com>님이
>> 작성:
>>>>>>> 
>>>>>>> Thanks Xin, I added it to the 1.1.0 epic.
>>>>>>> 
>>>>>>> -Taylor
>>>>>>> 
>>>>>>>> On Nov 15, 2016, at 9:01 AM, Xin Wang <da...@gmail.com>
>>> wrote:
>>>>>>>> 
>>>>>>>> STORM-2198 ( PR: https://github.com/apache/storm/pull/1773 )
>> fixes
>>> a
>>>>>> bug
>>>>>>> of
>>>>>>>> storm-hdfs. Do we have a consideration to include this?
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Xin Wang (vesense)
>>>>>>>> 
>>>>>>>> 2016-11-15 10:03 GMT+08:00 Jungtaek Lim <ka...@gmail.com>:
>>>>>>>> 
>>>>>>>>> Some issues on Storm SQL are resolved but not documented yet.
>> I'll
>>>>>> file
>>>>>>> an
>>>>>>>>> issue and assign to 1.1.0 release epic.
>>>>>>>>> And also I want to address dropping aggregation and join on Storm
>>> SQL
>>>>>>>>> Trident mode before releasing. I'll assign it too.
>>>>>>>>> 
>>>>>>>>> - Jungtaek Lim (HeartSaVioR)
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 2016년 11월 15일 (화) 오전 5:55, P. Taylor Goetz <pt...@gmail.com>님이
>>> 작성:
>>>>>>>>> 
>>>>>>>>>> I think we’re very close. I would like to confirm that the
>>>> 1.x-branch
>>>>>>> is
>>>>>>>>>> not affected by STORM-2176.
>>>>>>>>>> 
>>>>>>>>>> The worker lifecycle API was added in 1.0, but doesn’t work in
>> any
>>>>>>>>>> released version due to STORM-2176.
>>>>>>>>>> 
>>>>>>>>>> If there are any other open JIRAs that anyone is passionate
>> about,
>>>>>> now
>>>>>>>>>> would be a good time to assign them to the 1.1.0 release epic
>>>>>>>>> (STORM-1856).
>>>>>>>>>> 
>>>>>>>>>> -Taylor
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> On Oct 27, 2016, at 12:19 PM, Jungtaek Lim <ka...@gmail.com>
>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Finally Pacemaker H/A, Supervisor V2, and Storm SQL PRs which
>>> were
>>>>>>>>> opened
>>>>>>>>>>> at the last mail (4 weeks ago) are all merged to 1.x branch.
>>>>>>>>>>> 
>>>>>>>>>>> There're some more PRs on Storm SQL opened, but given that we
>> can
>>>>>>>>> release
>>>>>>>>>>> new minor at any time when we feel it's enough change, I can
>> wait
>>>>>> for
>>>>>>>>> it.
>>>>>>>>>>> They didn't get reviewed yet indeed.
>>>>>>>>>>> 
>>>>>>>>>>> Is there something else we would want to include it to 1.1.0?
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>>>> 
>>>>>>>>>>> 2016년 10월 1일 (토) 오전 9:30, Jungtaek Lim <ka...@gmail.com>님이
>> 작성:
>>>>>>>>>>> 
>>>>>>>>>>>> Personally, merging and porting back to three branches are
>>> painful
>>>>>>>>>> enough,
>>>>>>>>>>>> especially we don't have merging script and having verbose
>>> process
>>>>>> (I
>>>>>>>>>> mean
>>>>>>>>>>>> CHANGELOG).
>>>>>>>>>>>> It would be better if merging process is automated (by running
>>>>>> script
>>>>>>>>> or
>>>>>>>>>>>> so), so I'd +1 to revisit Harsha's suggestion (adopting Kafka
>>>> merge
>>>>>>>>>> script)
>>>>>>>>>>>> and modify script to fit to Storm.
>>>>>>>>>>>> (It will not work if it's the case we need to handle PRs for
>>> each
>>>>>>>>>> version
>>>>>>>>>>>> line, since 'Close' in commit log doesn't close the PR if its
>>>>>> target
>>>>>>>>>> branch
>>>>>>>>>>>> is not master.)
>>>>>>>>>>>> 
>>>>>>>>>>>> Anyway, without automation I don't want to maintain more
>> version
>>>>>>>>> lines.
>>>>>>>>>>>> I'm looking at the announces from other projects, and others
>> are
>>>>>> only
>>>>>>>>>>>> maintaining two version lines.
>>>>>>>>>>>> Since we maintain 2.0.0 version line we can't reduce version
>>> lines
>>>>>> to
>>>>>>>>> 2,
>>>>>>>>>>>> but hopefully at most 3.
>>>>>>>>>>>> 
>>>>>>>>>>>> Btw, let's check pending pull requests and enumerate which can
>>> be
>>>>>>>>>> included
>>>>>>>>>>>> in 1.0.0, and start/finish review and merge them soon.
>>>>>>>>>>>> For me Supervisor V2 and Pacemaker H/A, and pending Storm SQL
>>> PRs
>>>>>> can
>>>>>>>>> be
>>>>>>>>>>>> included, since they are small or in reviewing and expected to
>>>> pass
>>>>>>>>>> review
>>>>>>>>>>>> phase soon.
>>>>>>>>>>>> (And some small PRs. There're other valuable PRs in PR list
>> but
>>>> I'm
>>>>>>>>> not
>>>>>>>>>>>> sure we can review them soon. One example is unified API.)
>>>>>>>>>>>> 
>>>>>>>>>>>> One issue which is not clear is STORM-2006
>>>>>>>>>>>> <https://github.com/apache/storm/pull/1595>. This is a
>>> candidate
>>>>>> for
>>>>>>>>>> me,
>>>>>>>>>>>> but gets blocked while reviewing. If we plan to put great
>> effort
>>>> to
>>>>>>>>>> revise
>>>>>>>>>>>> Metric we can skip this.
>>>>>>>>>>>> 
>>>>>>>>>>>> Please enumerate other PRs as well if you want to include in
>>>> 1.1.0.
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Jungtaek Lim
>>>>>>>>>>>> 
>>>>>>>>>>>> 2016년 9월 30일 (금) 오후 11:09, Bobby Evans
>>>> <evans@yahoo-inc.com.invalid
>>>>>>> 
>>>>>>>>> 님이
>>>>>>>>>> 작성:
>>>>>>>>>>>> 
>>>>>>>>>>>> Sounds good to me.  It would be nice to get some of the new
>>>>>> features
>>>>>>>>>> out.
>>>>>>>>>>>> Do we expect to maintain both 1.0.x and 1.1.x lines with bug
>>>> fixes?
>>>>>>>>>> And if
>>>>>>>>>>>> so for how long do we want to do this for? - Bobby
>>>>>>>>>>>> 
>>>>>>>>>>>> On Thursday, September 29, 2016 7:35 PM, Jungtaek Lim <
>>>>>>>>>>>> kabhwan@gmail.com> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> Hi devs,
>>>>>>>>>>>> 
>>>>>>>>>>>> It's been 5 months after releasing Storm 1.0.0, and now 1.x
>>> branch
>>>>>>> has
>>>>>>>>>> lots
>>>>>>>>>>>> of CHANGELOG and also pending reviews.
>>>>>>>>>>>> It's also been a long time after 1.1.0 RC1 is canceled.
>>>>>>>>>>>> 
>>>>>>>>>>>> I think it may be good to put some efforts to review and merge
>>>>>>> pending
>>>>>>>>>> pull
>>>>>>>>>>>> requests (except things which takes time to review and test),
>>> and
>>>>>>>>>> release
>>>>>>>>>>>> 1.1.0 soon.
>>>>>>>>>>>> 
>>>>>>>>>>>> What do you think?
>>>>>>>>>>>> 
>>>>>>>>>>>> I'm also open to volunteer release manager for 1.1.0 after we
>>>>>>> document
>>>>>>>>>> the
>>>>>>>>>>>> process of official release.
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>> 
>>

Re: [DISCUSS] Release Storm 1.1.0

Posted by Jungtaek Lim <ka...@gmail.com>.

Seems like there're not blockers for 1.1.0, but some pull requests are
worth to check.
There're pending pull requests for storm-kafka-client waited on STORM-2225.
Given that STORM-2225 is now merged, we might need to take a look at.

*- reviewing*

[storm-core]

> STORM-2324 : Fix deployment failure if resources directory is missing in
topology jar
(master) https://github.com/apache/storm/pull/1908
(1.x) https://github.com/apache/storm/pull/1898

> STORM-2321 Handle blobstore zk key deletion in KeySequenceNumber
(master) https://github.com/apache/storm/pull/1904
(1.x) https://github.com/apache/storm/pull/1905

[storm-kafka]

> STORM-2270 Kafka spout should consume from latest when ZK partition
commit offset bigger than the latest offset
(1.x) https://github.com/apache/storm/pull/1851

[storm-kafka-client]

> STORM-2281: Running Multiple Kafka Spouts (Trident) Throws Illegal State
Exception
(1.x) https://github.com/apache/storm/pull/1902

> STORM-2315 Storm kafka client does not commit offsets when ack is disabled
(1.x) https://github.com/apache/storm/pull/1891

> fix: KafkaSpout is blocked in AutoCommitMode
(master) https://github.com/apache/storm/pull/1863

> STORM-2250: Kafka Spout Refactoring to Increase Modularity and Testability
(master) https://github.com/apache/storm/pull/1832

> STORM-2014: Put logic around dropping messages into RetryService, remove
maxRetry setting from new KafkaSpout
(master) https://github.com/apache/storm/pull/1605

> fix NullPointException with acked.get(rtp)
(master) https://github.com/apache/storm/pull/1807

[storm-sql]

> STORM-1443 [Storm SQL] Support customizing parallelism in StormSQL
https://github.com/apache/storm/pull/1739

*- pending*

[storm-kafka-client]

> STORM-2296 Kafka spout no dup on leader changes
(1.0.x) https://github.com/apache/storm/pull/1873
(1.x) https://github.com/apache/storm/pull/1888

[storm-sql]

> STORM-2148 [Storm SQL] Trident mode: back to code generate and compile
Trident topology
https://github.com/apache/storm/pull/1743

Thanks,
Jungtaek Lim (HeartSaVioR)

2017년 2월 2일 (목) 오전 8:14, Harsha Chintalapani <st...@harsha.io>님이 작성:

> Trying to check the status on this release of 1.1.0. Are we going to do
> this release anytime soon?
>
>
> On Fri, Jan 13, 2017 at 7:50 PM S G <sg...@gmail.com> wrote:
>
> > Not sure if its a little late to include for the 1.1.0 and 1.0.3 releases
> > now, but can we consider using zookeeper 3.4.9 for the future versions as
> > 3.4.9 brings in a lot of stability improvements (
> > http://zookeeper.apache.org/releases.html) and storm is still using
> 3.4.6
> > (
> > https://github.com/apache/storm/blob/master/pom.xml)
> >
> > On Fri, Jan 6, 2017 at 11:54 AM, P. Taylor Goetz <pt...@gmail.com>
> > wrote:
> >
> > > Thanks for the update Jungtaek.
> > >
> > > I’m verifying the patches now. And they should be mergeable shortly.
> > >
> > > I think we’ll likely be ready for 1.1.0 and 1.0.3 releases next week.
> > >
> > > -Taylor
> > >
> > > > On Jan 6, 2017, at 3:40 AM, Jungtaek Lim <ka...@gmail.com> wrote:
> > > >
> > > > I just submitted a patch for STORM-2176
> > > > <https://issues.apache.org/jira/browse/STORM-2176>. Since it's a
> small
> > > fix,
> > > > we just need to handle STORM-2228
> > > > <https://issues.apache.org/jira/browse/STORM-2228> to release.
> > > >
> > > > - Jungtaek Lim (HeartSaVioR)
> > > >
> > > > 2017년 1월 5일 (목) 오후 1:43, Jungtaek Lim <ka...@gmail.com>님이 작성:
> > > >
> > > >> Recently we receive some requests regarding release Storm 1.0.3, so
> > > would
> > > >> like to bump this again.
> > > >>
> > > >> Given that blocker issues for Storm 1.1.0 are also blocker for
> 1.0.3,
> > > I'd
> > > >> like to ask a favor of taking care of 'open' / 'in progress' issues
> on
> > > >> 1.1.0 epic.
> > > >> https://issues.apache.org/jira/browse/STORM-1856
> > > >>
> > > >> There're one 'open' issue and three 'in progress' issues. Two of
> three
> > > 'in
> > > >> progress' issues are tiny fix so easy to be handled, so actual
> > blockers
> > > are
> > > >> STORM-2176 <https://issues.apache.org/jira/browse/STORM-2176> and
> > > >> STORM-2228 <https://issues.apache.org/jira/browse/STORM-2228>.
> > > >>
> > > >> Thanks,
> > > >> Jungtaek Lim (HeartSaVioR)
> > > >>
> > > >> 2016년 11월 17일 (목) 오후 11:41, Satish Duggana <
> satish.duggana@gmail.com
> > >님이
> > > >> 작성:
> > > >>
> > > >>    STORM-2205: Race condition in getting nimbus summaries while ZK
> > > >> connections are reconnected.
> > > >>
> > > >> This issue seems to occur in our environments and I would like this
> to
> > > be
> > > >> part of 1.1.0.
> > > >>
> > > >> Thanks,
> > > >> Satish.
> > > >>
> > > >> On Thu, Nov 17, 2016 at 9:36 AM, Jungtaek Lim <ka...@gmail.com>
> > > wrote:
> > > >>
> > > >>> I have no idea on storm-kafka-client, but some bugfix issues for
> > > >>> storm-kafka-client are waiting for reviewing / merging.
> > > >>>
> > > >>> STORM-2014 <https://issues.apache.org/jira/browse/STORM-2014>
> > > >>> STORM-2087 <https://issues.apache.org/jira/browse/STORM-2087>
> > > >>> STORM-2104 <https://issues.apache.org/jira/browse/STORM-2104>
> > > >>>
> > > >>> If someone can review them in several days it would be great.
> > > >>>
> > > >>> I hope that we include currently opened pull requests for Storm SQL
> > so
> > > >> that
> > > >>> we can release 'usable Storm SQL' more usable, but I'm also OK to
> > > >> postpone
> > > >>> them to be included to next release if they drag the release.
> > > >>>
> > > >>> STORM-1446 <http://issues.apache.org/jira/browse/STORM-1446>
> > > >>> STORM-1443 <https://issues.apache.org/jira/browse/STORM-1443>
> > > >>> STORM-2148 <https://issues.apache.org/jira/browse/STORM-2148>
> > > >>> STORM-2170 <https://issues.apache.org/jira/browse/STORM-2170>
> > > >>>
> > > >>> I can see some pull requests which address Trident implementations
> > for
> > > >>> storm-kafka-client, storm-mongodb, storm-cassandra.
> > > >>>
> > > >>> storm-kafka-client: STORM-1694
> > > >>> <https://issues.apache.org/jira/browse/STORM-1694> (patch for 2.0
> is
> > > >>> merged, patch for 1.x is ready for reviewing)
> > > >>> storm-cassandra: STORM-1369
> > > >>> <https://issues.apache.org/jira/browse/STORM-1369>
> > > >>> storm-mongodb: STORM-1607 <https://issues.apache.org/
> > > >>> jira/browse/STORM-1607>
> > > >>>
> > > >>> If we want to cut the release now, we could include only bugfix
> > issues
> > > >> and
> > > >>> postpone others. Otherwise we could discuss and include some or all
> > of
> > > >> the
> > > >>> above.
> > > >>>
> > > >>> What do you think? When we want to start the release process for
> > 1.1.0?
> > > >>>
> > > >>> - Jungtaek Lim (HeartSaVioR)
> > > >>>
> > > >>> 2016년 11월 16일 (수) 오전 4:11, P. Taylor Goetz <pt...@gmail.com>님이
> 작성:
> > > >>>
> > > >>> Thanks Xin, I added it to the 1.1.0 epic.
> > > >>>
> > > >>> -Taylor
> > > >>>
> > > >>>> On Nov 15, 2016, at 9:01 AM, Xin Wang <da...@gmail.com>
> > wrote:
> > > >>>>
> > > >>>> STORM-2198 ( PR: https://github.com/apache/storm/pull/1773 )
> fixes
> > a
> > > >> bug
> > > >>> of
> > > >>>> storm-hdfs. Do we have a consideration to include this?
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Xin Wang (vesense)
> > > >>>>
> > > >>>> 2016-11-15 10:03 GMT+08:00 Jungtaek Lim <ka...@gmail.com>:
> > > >>>>
> > > >>>>> Some issues on Storm SQL are resolved but not documented yet.
> I'll
> > > >> file
> > > >>> an
> > > >>>>> issue and assign to 1.1.0 release epic.
> > > >>>>> And also I want to address dropping aggregation and join on Storm
> > SQL
> > > >>>>> Trident mode before releasing. I'll assign it too.
> > > >>>>>
> > > >>>>> - Jungtaek Lim (HeartSaVioR)
> > > >>>>>
> > > >>>>>
> > > >>>>> 2016년 11월 15일 (화) 오전 5:55, P. Taylor Goetz <pt...@gmail.com>님이
> > 작성:
> > > >>>>>
> > > >>>>>> I think we’re very close. I would like to confirm that the
> > > 1.x-branch
> > > >>> is
> > > >>>>>> not affected by STORM-2176.
> > > >>>>>>
> > > >>>>>> The worker lifecycle API was added in 1.0, but doesn’t work in
> any
> > > >>>>>> released version due to STORM-2176.
> > > >>>>>>
> > > >>>>>> If there are any other open JIRAs that anyone is passionate
> about,
> > > >> now
> > > >>>>>> would be a good time to assign them to the 1.1.0 release epic
> > > >>>>> (STORM-1856).
> > > >>>>>>
> > > >>>>>> -Taylor
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>> On Oct 27, 2016, at 12:19 PM, Jungtaek Lim <ka...@gmail.com>
> > > >> wrote:
> > > >>>>>>>
> > > >>>>>>> Finally Pacemaker H/A, Supervisor V2, and Storm SQL PRs which
> > were
> > > >>>>> opened
> > > >>>>>>> at the last mail (4 weeks ago) are all merged to 1.x branch.
> > > >>>>>>>
> > > >>>>>>> There're some more PRs on Storm SQL opened, but given that we
> can
> > > >>>>> release
> > > >>>>>>> new minor at any time when we feel it's enough change, I can
> wait
> > > >> for
> > > >>>>> it.
> > > >>>>>>> They didn't get reviewed yet indeed.
> > > >>>>>>>
> > > >>>>>>> Is there something else we would want to include it to 1.1.0?
> > > >>>>>>>
> > > >>>>>>> Thanks,
> > > >>>>>>> Jungtaek Lim (HeartSaVioR)
> > > >>>>>>>
> > > >>>>>>> 2016년 10월 1일 (토) 오전 9:30, Jungtaek Lim <ka...@gmail.com>님이
> 작성:
> > > >>>>>>>
> > > >>>>>>>> Personally, merging and porting back to three branches are
> > painful
> > > >>>>>> enough,
> > > >>>>>>>> especially we don't have merging script and having verbose
> > process
> > > >> (I
> > > >>>>>> mean
> > > >>>>>>>> CHANGELOG).
> > > >>>>>>>> It would be better if merging process is automated (by running
> > > >> script
> > > >>>>> or
> > > >>>>>>>> so), so I'd +1 to revisit Harsha's suggestion (adopting Kafka
> > > merge
> > > >>>>>> script)
> > > >>>>>>>> and modify script to fit to Storm.
> > > >>>>>>>> (It will not work if it's the case we need to handle PRs for
> > each
> > > >>>>>> version
> > > >>>>>>>> line, since 'Close' in commit log doesn't close the PR if its
> > > >> target
> > > >>>>>> branch
> > > >>>>>>>> is not master.)
> > > >>>>>>>>
> > > >>>>>>>> Anyway, without automation I don't want to maintain more
> version
> > > >>>>> lines.
> > > >>>>>>>> I'm looking at the announces from other projects, and others
> are
> > > >> only
> > > >>>>>>>> maintaining two version lines.
> > > >>>>>>>> Since we maintain 2.0.0 version line we can't reduce version
> > lines
> > > >> to
> > > >>>>> 2,
> > > >>>>>>>> but hopefully at most 3.
> > > >>>>>>>>
> > > >>>>>>>> Btw, let's check pending pull requests and enumerate which can
> > be
> > > >>>>>> included
> > > >>>>>>>> in 1.0.0, and start/finish review and merge them soon.
> > > >>>>>>>> For me Supervisor V2 and Pacemaker H/A, and pending Storm SQL
> > PRs
> > > >> can
> > > >>>>> be
> > > >>>>>>>> included, since they are small or in reviewing and expected to
> > > pass
> > > >>>>>> review
> > > >>>>>>>> phase soon.
> > > >>>>>>>> (And some small PRs. There're other valuable PRs in PR list
> but
> > > I'm
> > > >>>>> not
> > > >>>>>>>> sure we can review them soon. One example is unified API.)
> > > >>>>>>>>
> > > >>>>>>>> One issue which is not clear is STORM-2006
> > > >>>>>>>> <https://github.com/apache/storm/pull/1595>. This is a
> > candidate
> > > >> for
> > > >>>>>> me,
> > > >>>>>>>> but gets blocked while reviewing. If we plan to put great
> effort
> > > to
> > > >>>>>> revise
> > > >>>>>>>> Metric we can skip this.
> > > >>>>>>>>
> > > >>>>>>>> Please enumerate other PRs as well if you want to include in
> > > 1.1.0.
> > > >>>>>>>>
> > > >>>>>>>> Thanks,
> > > >>>>>>>> Jungtaek Lim
> > > >>>>>>>>
> > > >>>>>>>> 2016년 9월 30일 (금) 오후 11:09, Bobby Evans
> > > <evans@yahoo-inc.com.invalid
> > > >>>
> > > >>>>> 님이
> > > >>>>>> 작성:
> > > >>>>>>>>
> > > >>>>>>>> Sounds good to me.  It would be nice to get some of the new
> > > >> features
> > > >>>>>> out.
> > > >>>>>>>> Do we expect to maintain both 1.0.x and 1.1.x lines with bug
> > > fixes?
> > > >>>>>> And if
> > > >>>>>>>> so for how long do we want to do this for? - Bobby
> > > >>>>>>>>
> > > >>>>>>>>  On Thursday, September 29, 2016 7:35 PM, Jungtaek Lim <
> > > >>>>>>>> kabhwan@gmail.com> wrote:
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> Hi devs,
> > > >>>>>>>>
> > > >>>>>>>> It's been 5 months after releasing Storm 1.0.0, and now 1.x
> > branch
> > > >>> has
> > > >>>>>> lots
> > > >>>>>>>> of CHANGELOG and also pending reviews.
> > > >>>>>>>> It's also been a long time after 1.1.0 RC1 is canceled.
> > > >>>>>>>>
> > > >>>>>>>> I think it may be good to put some efforts to review and merge
> > > >>> pending
> > > >>>>>> pull
> > > >>>>>>>> requests (except things which takes time to review and test),
> > and
> > > >>>>>> release
> > > >>>>>>>> 1.1.0 soon.
> > > >>>>>>>>
> > > >>>>>>>> What do you think?
> > > >>>>>>>>
> > > >>>>>>>> I'm also open to volunteer release manager for 1.1.0 after we
> > > >>> document
> > > >>>>>> the
> > > >>>>>>>> process of official release.
> > > >>>>>>>>
> > > >>>>>>>> Thanks,
> > > >>>>>>>> Jungtaek Lim (HeartSaVioR)
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>
> > > >>
> > > >>
> > >
> > >
> >
>

Re: [DISCUSS] Release Storm 1.1.0

Posted by Harsha Chintalapani <st...@harsha.io>.

Trying to check the status on this release of 1.1.0. Are we going to do
this release anytime soon?


On Fri, Jan 13, 2017 at 7:50 PM S G <sg...@gmail.com> wrote:

> Not sure if its a little late to include for the 1.1.0 and 1.0.3 releases
> now, but can we consider using zookeeper 3.4.9 for the future versions as
> 3.4.9 brings in a lot of stability improvements (
> http://zookeeper.apache.org/releases.html) and storm is still using 3.4.6
> (
> https://github.com/apache/storm/blob/master/pom.xml)
>
> On Fri, Jan 6, 2017 at 11:54 AM, P. Taylor Goetz <pt...@gmail.com>
> wrote:
>
> > Thanks for the update Jungtaek.
> >
> > I’m verifying the patches now. And they should be mergeable shortly.
> >
> > I think we’ll likely be ready for 1.1.0 and 1.0.3 releases next week.
> >
> > -Taylor
> >
> > > On Jan 6, 2017, at 3:40 AM, Jungtaek Lim <ka...@gmail.com> wrote:
> > >
> > > I just submitted a patch for STORM-2176
> > > <https://issues.apache.org/jira/browse/STORM-2176>. Since it's a small
> > fix,
> > > we just need to handle STORM-2228
> > > <https://issues.apache.org/jira/browse/STORM-2228> to release.
> > >
> > > - Jungtaek Lim (HeartSaVioR)
> > >
> > > 2017년 1월 5일 (목) 오후 1:43, Jungtaek Lim <ka...@gmail.com>님이 작성:
> > >
> > >> Recently we receive some requests regarding release Storm 1.0.3, so
> > would
> > >> like to bump this again.
> > >>
> > >> Given that blocker issues for Storm 1.1.0 are also blocker for 1.0.3,
> > I'd
> > >> like to ask a favor of taking care of 'open' / 'in progress' issues on
> > >> 1.1.0 epic.
> > >> https://issues.apache.org/jira/browse/STORM-1856
> > >>
> > >> There're one 'open' issue and three 'in progress' issues. Two of three
> > 'in
> > >> progress' issues are tiny fix so easy to be handled, so actual
> blockers
> > are
> > >> STORM-2176 <https://issues.apache.org/jira/browse/STORM-2176> and
> > >> STORM-2228 <https://issues.apache.org/jira/browse/STORM-2228>.
> > >>
> > >> Thanks,
> > >> Jungtaek Lim (HeartSaVioR)
> > >>
> > >> 2016년 11월 17일 (목) 오후 11:41, Satish Duggana <satish.duggana@gmail.com
> >님이
> > >> 작성:
> > >>
> > >>    STORM-2205: Race condition in getting nimbus summaries while ZK
> > >> connections are reconnected.
> > >>
> > >> This issue seems to occur in our environments and I would like this to
> > be
> > >> part of 1.1.0.
> > >>
> > >> Thanks,
> > >> Satish.
> > >>
> > >> On Thu, Nov 17, 2016 at 9:36 AM, Jungtaek Lim <ka...@gmail.com>
> > wrote:
> > >>
> > >>> I have no idea on storm-kafka-client, but some bugfix issues for
> > >>> storm-kafka-client are waiting for reviewing / merging.
> > >>>
> > >>> STORM-2014 <https://issues.apache.org/jira/browse/STORM-2014>
> > >>> STORM-2087 <https://issues.apache.org/jira/browse/STORM-2087>
> > >>> STORM-2104 <https://issues.apache.org/jira/browse/STORM-2104>
> > >>>
> > >>> If someone can review them in several days it would be great.
> > >>>
> > >>> I hope that we include currently opened pull requests for Storm SQL
> so
> > >> that
> > >>> we can release 'usable Storm SQL' more usable, but I'm also OK to
> > >> postpone
> > >>> them to be included to next release if they drag the release.
> > >>>
> > >>> STORM-1446 <http://issues.apache.org/jira/browse/STORM-1446>
> > >>> STORM-1443 <https://issues.apache.org/jira/browse/STORM-1443>
> > >>> STORM-2148 <https://issues.apache.org/jira/browse/STORM-2148>
> > >>> STORM-2170 <https://issues.apache.org/jira/browse/STORM-2170>
> > >>>
> > >>> I can see some pull requests which address Trident implementations
> for
> > >>> storm-kafka-client, storm-mongodb, storm-cassandra.
> > >>>
> > >>> storm-kafka-client: STORM-1694
> > >>> <https://issues.apache.org/jira/browse/STORM-1694> (patch for 2.0 is
> > >>> merged, patch for 1.x is ready for reviewing)
> > >>> storm-cassandra: STORM-1369
> > >>> <https://issues.apache.org/jira/browse/STORM-1369>
> > >>> storm-mongodb: STORM-1607 <https://issues.apache.org/
> > >>> jira/browse/STORM-1607>
> > >>>
> > >>> If we want to cut the release now, we could include only bugfix
> issues
> > >> and
> > >>> postpone others. Otherwise we could discuss and include some or all
> of
> > >> the
> > >>> above.
> > >>>
> > >>> What do you think? When we want to start the release process for
> 1.1.0?
> > >>>
> > >>> - Jungtaek Lim (HeartSaVioR)
> > >>>
> > >>> 2016년 11월 16일 (수) 오전 4:11, P. Taylor Goetz <pt...@gmail.com>님이 작성:
> > >>>
> > >>> Thanks Xin, I added it to the 1.1.0 epic.
> > >>>
> > >>> -Taylor
> > >>>
> > >>>> On Nov 15, 2016, at 9:01 AM, Xin Wang <da...@gmail.com>
> wrote:
> > >>>>
> > >>>> STORM-2198 ( PR: https://github.com/apache/storm/pull/1773 ) fixes
> a
> > >> bug
> > >>> of
> > >>>> storm-hdfs. Do we have a consideration to include this?
> > >>>>
> > >>>> Thanks,
> > >>>> Xin Wang (vesense)
> > >>>>
> > >>>> 2016-11-15 10:03 GMT+08:00 Jungtaek Lim <ka...@gmail.com>:
> > >>>>
> > >>>>> Some issues on Storm SQL are resolved but not documented yet. I'll
> > >> file
> > >>> an
> > >>>>> issue and assign to 1.1.0 release epic.
> > >>>>> And also I want to address dropping aggregation and join on Storm
> SQL
> > >>>>> Trident mode before releasing. I'll assign it too.
> > >>>>>
> > >>>>> - Jungtaek Lim (HeartSaVioR)
> > >>>>>
> > >>>>>
> > >>>>> 2016년 11월 15일 (화) 오전 5:55, P. Taylor Goetz <pt...@gmail.com>님이
> 작성:
> > >>>>>
> > >>>>>> I think we’re very close. I would like to confirm that the
> > 1.x-branch
> > >>> is
> > >>>>>> not affected by STORM-2176.
> > >>>>>>
> > >>>>>> The worker lifecycle API was added in 1.0, but doesn’t work in any
> > >>>>>> released version due to STORM-2176.
> > >>>>>>
> > >>>>>> If there are any other open JIRAs that anyone is passionate about,
> > >> now
> > >>>>>> would be a good time to assign them to the 1.1.0 release epic
> > >>>>> (STORM-1856).
> > >>>>>>
> > >>>>>> -Taylor
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>> On Oct 27, 2016, at 12:19 PM, Jungtaek Lim <ka...@gmail.com>
> > >> wrote:
> > >>>>>>>
> > >>>>>>> Finally Pacemaker H/A, Supervisor V2, and Storm SQL PRs which
> were
> > >>>>> opened
> > >>>>>>> at the last mail (4 weeks ago) are all merged to 1.x branch.
> > >>>>>>>
> > >>>>>>> There're some more PRs on Storm SQL opened, but given that we can
> > >>>>> release
> > >>>>>>> new minor at any time when we feel it's enough change, I can wait
> > >> for
> > >>>>> it.
> > >>>>>>> They didn't get reviewed yet indeed.
> > >>>>>>>
> > >>>>>>> Is there something else we would want to include it to 1.1.0?
> > >>>>>>>
> > >>>>>>> Thanks,
> > >>>>>>> Jungtaek Lim (HeartSaVioR)
> > >>>>>>>
> > >>>>>>> 2016년 10월 1일 (토) 오전 9:30, Jungtaek Lim <ka...@gmail.com>님이 작성:
> > >>>>>>>
> > >>>>>>>> Personally, merging and porting back to three branches are
> painful
> > >>>>>> enough,
> > >>>>>>>> especially we don't have merging script and having verbose
> process
> > >> (I
> > >>>>>> mean
> > >>>>>>>> CHANGELOG).
> > >>>>>>>> It would be better if merging process is automated (by running
> > >> script
> > >>>>> or
> > >>>>>>>> so), so I'd +1 to revisit Harsha's suggestion (adopting Kafka
> > merge
> > >>>>>> script)
> > >>>>>>>> and modify script to fit to Storm.
> > >>>>>>>> (It will not work if it's the case we need to handle PRs for
> each
> > >>>>>> version
> > >>>>>>>> line, since 'Close' in commit log doesn't close the PR if its
> > >> target
> > >>>>>> branch
> > >>>>>>>> is not master.)
> > >>>>>>>>
> > >>>>>>>> Anyway, without automation I don't want to maintain more version
> > >>>>> lines.
> > >>>>>>>> I'm looking at the announces from other projects, and others are
> > >> only
> > >>>>>>>> maintaining two version lines.
> > >>>>>>>> Since we maintain 2.0.0 version line we can't reduce version
> lines
> > >> to
> > >>>>> 2,
> > >>>>>>>> but hopefully at most 3.
> > >>>>>>>>
> > >>>>>>>> Btw, let's check pending pull requests and enumerate which can
> be
> > >>>>>> included
> > >>>>>>>> in 1.0.0, and start/finish review and merge them soon.
> > >>>>>>>> For me Supervisor V2 and Pacemaker H/A, and pending Storm SQL
> PRs
> > >> can
> > >>>>> be
> > >>>>>>>> included, since they are small or in reviewing and expected to
> > pass
> > >>>>>> review
> > >>>>>>>> phase soon.
> > >>>>>>>> (And some small PRs. There're other valuable PRs in PR list but
> > I'm
> > >>>>> not
> > >>>>>>>> sure we can review them soon. One example is unified API.)
> > >>>>>>>>
> > >>>>>>>> One issue which is not clear is STORM-2006
> > >>>>>>>> <https://github.com/apache/storm/pull/1595>. This is a
> candidate
> > >> for
> > >>>>>> me,
> > >>>>>>>> but gets blocked while reviewing. If we plan to put great effort
> > to
> > >>>>>> revise
> > >>>>>>>> Metric we can skip this.
> > >>>>>>>>
> > >>>>>>>> Please enumerate other PRs as well if you want to include in
> > 1.1.0.
> > >>>>>>>>
> > >>>>>>>> Thanks,
> > >>>>>>>> Jungtaek Lim
> > >>>>>>>>
> > >>>>>>>> 2016년 9월 30일 (금) 오후 11:09, Bobby Evans
> > <evans@yahoo-inc.com.invalid
> > >>>
> > >>>>> 님이
> > >>>>>> 작성:
> > >>>>>>>>
> > >>>>>>>> Sounds good to me.  It would be nice to get some of the new
> > >> features
> > >>>>>> out.
> > >>>>>>>> Do we expect to maintain both 1.0.x and 1.1.x lines with bug
> > fixes?
> > >>>>>> And if
> > >>>>>>>> so for how long do we want to do this for? - Bobby
> > >>>>>>>>
> > >>>>>>>>  On Thursday, September 29, 2016 7:35 PM, Jungtaek Lim <
> > >>>>>>>> kabhwan@gmail.com> wrote:
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Hi devs,
> > >>>>>>>>
> > >>>>>>>> It's been 5 months after releasing Storm 1.0.0, and now 1.x
> branch
> > >>> has
> > >>>>>> lots
> > >>>>>>>> of CHANGELOG and also pending reviews.
> > >>>>>>>> It's also been a long time after 1.1.0 RC1 is canceled.
> > >>>>>>>>
> > >>>>>>>> I think it may be good to put some efforts to review and merge
> > >>> pending
> > >>>>>> pull
> > >>>>>>>> requests (except things which takes time to review and test),
> and
> > >>>>>> release
> > >>>>>>>> 1.1.0 soon.
> > >>>>>>>>
> > >>>>>>>> What do you think?
> > >>>>>>>>
> > >>>>>>>> I'm also open to volunteer release manager for 1.1.0 after we
> > >>> document
> > >>>>>> the
> > >>>>>>>> process of official release.
> > >>>>>>>>
> > >>>>>>>> Thanks,
> > >>>>>>>> Jungtaek Lim (HeartSaVioR)
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> > >>
> > >>
> >
> >
>

Re: [DISCUSS] Release Storm 1.1.0

Posted by S G <sg...@gmail.com>.

Not sure if its a little late to include for the 1.1.0 and 1.0.3 releases
now, but can we consider using zookeeper 3.4.9 for the future versions as
3.4.9 brings in a lot of stability improvements (
http://zookeeper.apache.org/releases.html) and storm is still using 3.4.6 (
https://github.com/apache/storm/blob/master/pom.xml)

On Fri, Jan 6, 2017 at 11:54 AM, P. Taylor Goetz <pt...@gmail.com> wrote:

> Thanks for the update Jungtaek.
>
> I’m verifying the patches now. And they should be mergeable shortly.
>
> I think we’ll likely be ready for 1.1.0 and 1.0.3 releases next week.
>
> -Taylor
>
> > On Jan 6, 2017, at 3:40 AM, Jungtaek Lim <ka...@gmail.com> wrote:
> >
> > I just submitted a patch for STORM-2176
> > <https://issues.apache.org/jira/browse/STORM-2176>. Since it's a small
> fix,
> > we just need to handle STORM-2228
> > <https://issues.apache.org/jira/browse/STORM-2228> to release.
> >
> > - Jungtaek Lim (HeartSaVioR)
> >
> > 2017년 1월 5일 (목) 오후 1:43, Jungtaek Lim <ka...@gmail.com>님이 작성:
> >
> >> Recently we receive some requests regarding release Storm 1.0.3, so
> would
> >> like to bump this again.
> >>
> >> Given that blocker issues for Storm 1.1.0 are also blocker for 1.0.3,
> I'd
> >> like to ask a favor of taking care of 'open' / 'in progress' issues on
> >> 1.1.0 epic.
> >> https://issues.apache.org/jira/browse/STORM-1856
> >>
> >> There're one 'open' issue and three 'in progress' issues. Two of three
> 'in
> >> progress' issues are tiny fix so easy to be handled, so actual blockers
> are
> >> STORM-2176 <https://issues.apache.org/jira/browse/STORM-2176> and
> >> STORM-2228 <https://issues.apache.org/jira/browse/STORM-2228>.
> >>
> >> Thanks,
> >> Jungtaek Lim (HeartSaVioR)
> >>
> >> 2016년 11월 17일 (목) 오후 11:41, Satish Duggana <sa...@gmail.com>님이
> >> 작성:
> >>
> >>    STORM-2205: Race condition in getting nimbus summaries while ZK
> >> connections are reconnected.
> >>
> >> This issue seems to occur in our environments and I would like this to
> be
> >> part of 1.1.0.
> >>
> >> Thanks,
> >> Satish.
> >>
> >> On Thu, Nov 17, 2016 at 9:36 AM, Jungtaek Lim <ka...@gmail.com>
> wrote:
> >>
> >>> I have no idea on storm-kafka-client, but some bugfix issues for
> >>> storm-kafka-client are waiting for reviewing / merging.
> >>>
> >>> STORM-2014 <https://issues.apache.org/jira/browse/STORM-2014>
> >>> STORM-2087 <https://issues.apache.org/jira/browse/STORM-2087>
> >>> STORM-2104 <https://issues.apache.org/jira/browse/STORM-2104>
> >>>
> >>> If someone can review them in several days it would be great.
> >>>
> >>> I hope that we include currently opened pull requests for Storm SQL so
> >> that
> >>> we can release 'usable Storm SQL' more usable, but I'm also OK to
> >> postpone
> >>> them to be included to next release if they drag the release.
> >>>
> >>> STORM-1446 <http://issues.apache.org/jira/browse/STORM-1446>
> >>> STORM-1443 <https://issues.apache.org/jira/browse/STORM-1443>
> >>> STORM-2148 <https://issues.apache.org/jira/browse/STORM-2148>
> >>> STORM-2170 <https://issues.apache.org/jira/browse/STORM-2170>
> >>>
> >>> I can see some pull requests which address Trident implementations for
> >>> storm-kafka-client, storm-mongodb, storm-cassandra.
> >>>
> >>> storm-kafka-client: STORM-1694
> >>> <https://issues.apache.org/jira/browse/STORM-1694> (patch for 2.0 is
> >>> merged, patch for 1.x is ready for reviewing)
> >>> storm-cassandra: STORM-1369
> >>> <https://issues.apache.org/jira/browse/STORM-1369>
> >>> storm-mongodb: STORM-1607 <https://issues.apache.org/
> >>> jira/browse/STORM-1607>
> >>>
> >>> If we want to cut the release now, we could include only bugfix issues
> >> and
> >>> postpone others. Otherwise we could discuss and include some or all of
> >> the
> >>> above.
> >>>
> >>> What do you think? When we want to start the release process for 1.1.0?
> >>>
> >>> - Jungtaek Lim (HeartSaVioR)
> >>>
> >>> 2016년 11월 16일 (수) 오전 4:11, P. Taylor Goetz <pt...@gmail.com>님이 작성:
> >>>
> >>> Thanks Xin, I added it to the 1.1.0 epic.
> >>>
> >>> -Taylor
> >>>
> >>>> On Nov 15, 2016, at 9:01 AM, Xin Wang <da...@gmail.com> wrote:
> >>>>
> >>>> STORM-2198 ( PR: https://github.com/apache/storm/pull/1773 ) fixes a
> >> bug
> >>> of
> >>>> storm-hdfs. Do we have a consideration to include this?
> >>>>
> >>>> Thanks,
> >>>> Xin Wang (vesense)
> >>>>
> >>>> 2016-11-15 10:03 GMT+08:00 Jungtaek Lim <ka...@gmail.com>:
> >>>>
> >>>>> Some issues on Storm SQL are resolved but not documented yet. I'll
> >> file
> >>> an
> >>>>> issue and assign to 1.1.0 release epic.
> >>>>> And also I want to address dropping aggregation and join on Storm SQL
> >>>>> Trident mode before releasing. I'll assign it too.
> >>>>>
> >>>>> - Jungtaek Lim (HeartSaVioR)
> >>>>>
> >>>>>
> >>>>> 2016년 11월 15일 (화) 오전 5:55, P. Taylor Goetz <pt...@gmail.com>님이 작성:
> >>>>>
> >>>>>> I think we’re very close. I would like to confirm that the
> 1.x-branch
> >>> is
> >>>>>> not affected by STORM-2176.
> >>>>>>
> >>>>>> The worker lifecycle API was added in 1.0, but doesn’t work in any
> >>>>>> released version due to STORM-2176.
> >>>>>>
> >>>>>> If there are any other open JIRAs that anyone is passionate about,
> >> now
> >>>>>> would be a good time to assign them to the 1.1.0 release epic
> >>>>> (STORM-1856).
> >>>>>>
> >>>>>> -Taylor
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> On Oct 27, 2016, at 12:19 PM, Jungtaek Lim <ka...@gmail.com>
> >> wrote:
> >>>>>>>
> >>>>>>> Finally Pacemaker H/A, Supervisor V2, and Storm SQL PRs which were
> >>>>> opened
> >>>>>>> at the last mail (4 weeks ago) are all merged to 1.x branch.
> >>>>>>>
> >>>>>>> There're some more PRs on Storm SQL opened, but given that we can
> >>>>> release
> >>>>>>> new minor at any time when we feel it's enough change, I can wait
> >> for
> >>>>> it.
> >>>>>>> They didn't get reviewed yet indeed.
> >>>>>>>
> >>>>>>> Is there something else we would want to include it to 1.1.0?
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Jungtaek Lim (HeartSaVioR)
> >>>>>>>
> >>>>>>> 2016년 10월 1일 (토) 오전 9:30, Jungtaek Lim <ka...@gmail.com>님이 작성:
> >>>>>>>
> >>>>>>>> Personally, merging and porting back to three branches are painful
> >>>>>> enough,
> >>>>>>>> especially we don't have merging script and having verbose process
> >> (I
> >>>>>> mean
> >>>>>>>> CHANGELOG).
> >>>>>>>> It would be better if merging process is automated (by running
> >> script
> >>>>> or
> >>>>>>>> so), so I'd +1 to revisit Harsha's suggestion (adopting Kafka
> merge
> >>>>>> script)
> >>>>>>>> and modify script to fit to Storm.
> >>>>>>>> (It will not work if it's the case we need to handle PRs for each
> >>>>>> version
> >>>>>>>> line, since 'Close' in commit log doesn't close the PR if its
> >> target
> >>>>>> branch
> >>>>>>>> is not master.)
> >>>>>>>>
> >>>>>>>> Anyway, without automation I don't want to maintain more version
> >>>>> lines.
> >>>>>>>> I'm looking at the announces from other projects, and others are
> >> only
> >>>>>>>> maintaining two version lines.
> >>>>>>>> Since we maintain 2.0.0 version line we can't reduce version lines
> >> to
> >>>>> 2,
> >>>>>>>> but hopefully at most 3.
> >>>>>>>>
> >>>>>>>> Btw, let's check pending pull requests and enumerate which can be
> >>>>>> included
> >>>>>>>> in 1.0.0, and start/finish review and merge them soon.
> >>>>>>>> For me Supervisor V2 and Pacemaker H/A, and pending Storm SQL PRs
> >> can
> >>>>> be
> >>>>>>>> included, since they are small or in reviewing and expected to
> pass
> >>>>>> review
> >>>>>>>> phase soon.
> >>>>>>>> (And some small PRs. There're other valuable PRs in PR list but
> I'm
> >>>>> not
> >>>>>>>> sure we can review them soon. One example is unified API.)
> >>>>>>>>
> >>>>>>>> One issue which is not clear is STORM-2006
> >>>>>>>> <https://github.com/apache/storm/pull/1595>. This is a candidate
> >> for
> >>>>>> me,
> >>>>>>>> but gets blocked while reviewing. If we plan to put great effort
> to
> >>>>>> revise
> >>>>>>>> Metric we can skip this.
> >>>>>>>>
> >>>>>>>> Please enumerate other PRs as well if you want to include in
> 1.1.0.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Jungtaek Lim
> >>>>>>>>
> >>>>>>>> 2016년 9월 30일 (금) 오후 11:09, Bobby Evans
> <evans@yahoo-inc.com.invalid
> >>>
> >>>>> 님이
> >>>>>> 작성:
> >>>>>>>>
> >>>>>>>> Sounds good to me.  It would be nice to get some of the new
> >> features
> >>>>>> out.
> >>>>>>>> Do we expect to maintain both 1.0.x and 1.1.x lines with bug
> fixes?
> >>>>>> And if
> >>>>>>>> so for how long do we want to do this for? - Bobby
> >>>>>>>>
> >>>>>>>>  On Thursday, September 29, 2016 7:35 PM, Jungtaek Lim <
> >>>>>>>> kabhwan@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Hi devs,
> >>>>>>>>
> >>>>>>>> It's been 5 months after releasing Storm 1.0.0, and now 1.x branch
> >>> has
> >>>>>> lots
> >>>>>>>> of CHANGELOG and also pending reviews.
> >>>>>>>> It's also been a long time after 1.1.0 RC1 is canceled.
> >>>>>>>>
> >>>>>>>> I think it may be good to put some efforts to review and merge
> >>> pending
> >>>>>> pull
> >>>>>>>> requests (except things which takes time to review and test), and
> >>>>>> release
> >>>>>>>> 1.1.0 soon.
> >>>>>>>>
> >>>>>>>> What do you think?
> >>>>>>>>
> >>>>>>>> I'm also open to volunteer release manager for 1.1.0 after we
> >>> document
> >>>>>> the
> >>>>>>>> process of official release.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Jungtaek Lim (HeartSaVioR)
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>
> >>
> >>
>
>

Re: [DISCUSS] Release Storm 1.1.0

Posted by "P. Taylor Goetz" <pt...@gmail.com>.

Thanks for the update Jungtaek.

I’m verifying the patches now. And they should be mergeable shortly.

I think we’ll likely be ready for 1.1.0 and 1.0.3 releases next week.

-Taylor

> On Jan 6, 2017, at 3:40 AM, Jungtaek Lim <ka...@gmail.com> wrote:
> 
> I just submitted a patch for STORM-2176
> <https://issues.apache.org/jira/browse/STORM-2176>. Since it's a small fix,
> we just need to handle STORM-2228
> <https://issues.apache.org/jira/browse/STORM-2228> to release.
> 
> - Jungtaek Lim (HeartSaVioR)
> 
> 2017년 1월 5일 (목) 오후 1:43, Jungtaek Lim <ka...@gmail.com>님이 작성:
> 
>> Recently we receive some requests regarding release Storm 1.0.3, so would
>> like to bump this again.
>> 
>> Given that blocker issues for Storm 1.1.0 are also blocker for 1.0.3, I'd
>> like to ask a favor of taking care of 'open' / 'in progress' issues on
>> 1.1.0 epic.
>> https://issues.apache.org/jira/browse/STORM-1856
>> 
>> There're one 'open' issue and three 'in progress' issues. Two of three 'in
>> progress' issues are tiny fix so easy to be handled, so actual blockers are
>> STORM-2176 <https://issues.apache.org/jira/browse/STORM-2176> and
>> STORM-2228 <https://issues.apache.org/jira/browse/STORM-2228>.
>> 
>> Thanks,
>> Jungtaek Lim (HeartSaVioR)
>> 
>> 2016년 11월 17일 (목) 오후 11:41, Satish Duggana <sa...@gmail.com>님이
>> 작성:
>> 
>>    STORM-2205: Race condition in getting nimbus summaries while ZK
>> connections are reconnected.
>> 
>> This issue seems to occur in our environments and I would like this to be
>> part of 1.1.0.
>> 
>> Thanks,
>> Satish.
>> 
>> On Thu, Nov 17, 2016 at 9:36 AM, Jungtaek Lim <ka...@gmail.com> wrote:
>> 
>>> I have no idea on storm-kafka-client, but some bugfix issues for
>>> storm-kafka-client are waiting for reviewing / merging.
>>> 
>>> STORM-2014 <https://issues.apache.org/jira/browse/STORM-2014>
>>> STORM-2087 <https://issues.apache.org/jira/browse/STORM-2087>
>>> STORM-2104 <https://issues.apache.org/jira/browse/STORM-2104>
>>> 
>>> If someone can review them in several days it would be great.
>>> 
>>> I hope that we include currently opened pull requests for Storm SQL so
>> that
>>> we can release 'usable Storm SQL' more usable, but I'm also OK to
>> postpone
>>> them to be included to next release if they drag the release.
>>> 
>>> STORM-1446 <http://issues.apache.org/jira/browse/STORM-1446>
>>> STORM-1443 <https://issues.apache.org/jira/browse/STORM-1443>
>>> STORM-2148 <https://issues.apache.org/jira/browse/STORM-2148>
>>> STORM-2170 <https://issues.apache.org/jira/browse/STORM-2170>
>>> 
>>> I can see some pull requests which address Trident implementations for
>>> storm-kafka-client, storm-mongodb, storm-cassandra.
>>> 
>>> storm-kafka-client: STORM-1694
>>> <https://issues.apache.org/jira/browse/STORM-1694> (patch for 2.0 is
>>> merged, patch for 1.x is ready for reviewing)
>>> storm-cassandra: STORM-1369
>>> <https://issues.apache.org/jira/browse/STORM-1369>
>>> storm-mongodb: STORM-1607 <https://issues.apache.org/
>>> jira/browse/STORM-1607>
>>> 
>>> If we want to cut the release now, we could include only bugfix issues
>> and
>>> postpone others. Otherwise we could discuss and include some or all of
>> the
>>> above.
>>> 
>>> What do you think? When we want to start the release process for 1.1.0?
>>> 
>>> - Jungtaek Lim (HeartSaVioR)
>>> 
>>> 2016년 11월 16일 (수) 오전 4:11, P. Taylor Goetz <pt...@gmail.com>님이 작성:
>>> 
>>> Thanks Xin, I added it to the 1.1.0 epic.
>>> 
>>> -Taylor
>>> 
>>>> On Nov 15, 2016, at 9:01 AM, Xin Wang <da...@gmail.com> wrote:
>>>> 
>>>> STORM-2198 ( PR: https://github.com/apache/storm/pull/1773 ) fixes a
>> bug
>>> of
>>>> storm-hdfs. Do we have a consideration to include this?
>>>> 
>>>> Thanks,
>>>> Xin Wang (vesense)
>>>> 
>>>> 2016-11-15 10:03 GMT+08:00 Jungtaek Lim <ka...@gmail.com>:
>>>> 
>>>>> Some issues on Storm SQL are resolved but not documented yet. I'll
>> file
>>> an
>>>>> issue and assign to 1.1.0 release epic.
>>>>> And also I want to address dropping aggregation and join on Storm SQL
>>>>> Trident mode before releasing. I'll assign it too.
>>>>> 
>>>>> - Jungtaek Lim (HeartSaVioR)
>>>>> 
>>>>> 
>>>>> 2016년 11월 15일 (화) 오전 5:55, P. Taylor Goetz <pt...@gmail.com>님이 작성:
>>>>> 
>>>>>> I think we’re very close. I would like to confirm that the 1.x-branch
>>> is
>>>>>> not affected by STORM-2176.
>>>>>> 
>>>>>> The worker lifecycle API was added in 1.0, but doesn’t work in any
>>>>>> released version due to STORM-2176.
>>>>>> 
>>>>>> If there are any other open JIRAs that anyone is passionate about,
>> now
>>>>>> would be a good time to assign them to the 1.1.0 release epic
>>>>> (STORM-1856).
>>>>>> 
>>>>>> -Taylor
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> On Oct 27, 2016, at 12:19 PM, Jungtaek Lim <ka...@gmail.com>
>> wrote:
>>>>>>> 
>>>>>>> Finally Pacemaker H/A, Supervisor V2, and Storm SQL PRs which were
>>>>> opened
>>>>>>> at the last mail (4 weeks ago) are all merged to 1.x branch.
>>>>>>> 
>>>>>>> There're some more PRs on Storm SQL opened, but given that we can
>>>>> release
>>>>>>> new minor at any time when we feel it's enough change, I can wait
>> for
>>>>> it.
>>>>>>> They didn't get reviewed yet indeed.
>>>>>>> 
>>>>>>> Is there something else we would want to include it to 1.1.0?
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>> 
>>>>>>> 2016년 10월 1일 (토) 오전 9:30, Jungtaek Lim <ka...@gmail.com>님이 작성:
>>>>>>> 
>>>>>>>> Personally, merging and porting back to three branches are painful
>>>>>> enough,
>>>>>>>> especially we don't have merging script and having verbose process
>> (I
>>>>>> mean
>>>>>>>> CHANGELOG).
>>>>>>>> It would be better if merging process is automated (by running
>> script
>>>>> or
>>>>>>>> so), so I'd +1 to revisit Harsha's suggestion (adopting Kafka merge
>>>>>> script)
>>>>>>>> and modify script to fit to Storm.
>>>>>>>> (It will not work if it's the case we need to handle PRs for each
>>>>>> version
>>>>>>>> line, since 'Close' in commit log doesn't close the PR if its
>> target
>>>>>> branch
>>>>>>>> is not master.)
>>>>>>>> 
>>>>>>>> Anyway, without automation I don't want to maintain more version
>>>>> lines.
>>>>>>>> I'm looking at the announces from other projects, and others are
>> only
>>>>>>>> maintaining two version lines.
>>>>>>>> Since we maintain 2.0.0 version line we can't reduce version lines
>> to
>>>>> 2,
>>>>>>>> but hopefully at most 3.
>>>>>>>> 
>>>>>>>> Btw, let's check pending pull requests and enumerate which can be
>>>>>> included
>>>>>>>> in 1.0.0, and start/finish review and merge them soon.
>>>>>>>> For me Supervisor V2 and Pacemaker H/A, and pending Storm SQL PRs
>> can
>>>>> be
>>>>>>>> included, since they are small or in reviewing and expected to pass
>>>>>> review
>>>>>>>> phase soon.
>>>>>>>> (And some small PRs. There're other valuable PRs in PR list but I'm
>>>>> not
>>>>>>>> sure we can review them soon. One example is unified API.)
>>>>>>>> 
>>>>>>>> One issue which is not clear is STORM-2006
>>>>>>>> <https://github.com/apache/storm/pull/1595>. This is a candidate
>> for
>>>>>> me,
>>>>>>>> but gets blocked while reviewing. If we plan to put great effort to
>>>>>> revise
>>>>>>>> Metric we can skip this.
>>>>>>>> 
>>>>>>>> Please enumerate other PRs as well if you want to include in 1.1.0.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Jungtaek Lim
>>>>>>>> 
>>>>>>>> 2016년 9월 30일 (금) 오후 11:09, Bobby Evans <evans@yahoo-inc.com.invalid
>>> 
>>>>> 님이
>>>>>> 작성:
>>>>>>>> 
>>>>>>>> Sounds good to me.  It would be nice to get some of the new
>> features
>>>>>> out.
>>>>>>>> Do we expect to maintain both 1.0.x and 1.1.x lines with bug fixes?
>>>>>> And if
>>>>>>>> so for how long do we want to do this for? - Bobby
>>>>>>>> 
>>>>>>>>  On Thursday, September 29, 2016 7:35 PM, Jungtaek Lim <
>>>>>>>> kabhwan@gmail.com> wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Hi devs,
>>>>>>>> 
>>>>>>>> It's been 5 months after releasing Storm 1.0.0, and now 1.x branch
>>> has
>>>>>> lots
>>>>>>>> of CHANGELOG and also pending reviews.
>>>>>>>> It's also been a long time after 1.1.0 RC1 is canceled.
>>>>>>>> 
>>>>>>>> I think it may be good to put some efforts to review and merge
>>> pending
>>>>>> pull
>>>>>>>> requests (except things which takes time to review and test), and
>>>>>> release
>>>>>>>> 1.1.0 soon.
>>>>>>>> 
>>>>>>>> What do you think?
>>>>>>>> 
>>>>>>>> I'm also open to volunteer release manager for 1.1.0 after we
>>> document
>>>>>> the
>>>>>>>> process of official release.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>> 
>> 
>>

Re: [DISCUSS] Release Storm 1.1.0

Posted by Jungtaek Lim <ka...@gmail.com>.

I just submitted a patch for STORM-2176
<https://issues.apache.org/jira/browse/STORM-2176>. Since it's a small fix,
we just need to handle STORM-2228
<https://issues.apache.org/jira/browse/STORM-2228> to release.

- Jungtaek Lim (HeartSaVioR)

2017년 1월 5일 (목) 오후 1:43, Jungtaek Lim <ka...@gmail.com>님이 작성:

> Recently we receive some requests regarding release Storm 1.0.3, so would
> like to bump this again.
>
> Given that blocker issues for Storm 1.1.0 are also blocker for 1.0.3, I'd
> like to ask a favor of taking care of 'open' / 'in progress' issues on
> 1.1.0 epic.
> https://issues.apache.org/jira/browse/STORM-1856
>
> There're one 'open' issue and three 'in progress' issues. Two of three 'in
> progress' issues are tiny fix so easy to be handled, so actual blockers are
> STORM-2176 <https://issues.apache.org/jira/browse/STORM-2176> and
> STORM-2228 <https://issues.apache.org/jira/browse/STORM-2228>.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 2016년 11월 17일 (목) 오후 11:41, Satish Duggana <sa...@gmail.com>님이
> 작성:
>
>     STORM-2205: Race condition in getting nimbus summaries while ZK
> connections are reconnected.
>
> This issue seems to occur in our environments and I would like this to be
> part of 1.1.0.
>
> Thanks,
> Satish.
>
> On Thu, Nov 17, 2016 at 9:36 AM, Jungtaek Lim <ka...@gmail.com> wrote:
>
> > I have no idea on storm-kafka-client, but some bugfix issues for
> > storm-kafka-client are waiting for reviewing / merging.
> >
> > STORM-2014 <https://issues.apache.org/jira/browse/STORM-2014>
> > STORM-2087 <https://issues.apache.org/jira/browse/STORM-2087>
> > STORM-2104 <https://issues.apache.org/jira/browse/STORM-2104>
> >
> > If someone can review them in several days it would be great.
> >
> > I hope that we include currently opened pull requests for Storm SQL so
> that
> > we can release 'usable Storm SQL' more usable, but I'm also OK to
> postpone
> > them to be included to next release if they drag the release.
> >
> > STORM-1446 <http://issues.apache.org/jira/browse/STORM-1446>
> > STORM-1443 <https://issues.apache.org/jira/browse/STORM-1443>
> > STORM-2148 <https://issues.apache.org/jira/browse/STORM-2148>
> > STORM-2170 <https://issues.apache.org/jira/browse/STORM-2170>
> >
> > I can see some pull requests which address Trident implementations for
> > storm-kafka-client, storm-mongodb, storm-cassandra.
> >
> > storm-kafka-client: STORM-1694
> > <https://issues.apache.org/jira/browse/STORM-1694> (patch for 2.0 is
> > merged, patch for 1.x is ready for reviewing)
> > storm-cassandra: STORM-1369
> > <https://issues.apache.org/jira/browse/STORM-1369>
> > storm-mongodb: STORM-1607 <https://issues.apache.org/
> > jira/browse/STORM-1607>
> >
> > If we want to cut the release now, we could include only bugfix issues
> and
> > postpone others. Otherwise we could discuss and include some or all of
> the
> > above.
> >
> > What do you think? When we want to start the release process for 1.1.0?
> >
> > - Jungtaek Lim (HeartSaVioR)
> >
> > 2016년 11월 16일 (수) 오전 4:11, P. Taylor Goetz <pt...@gmail.com>님이 작성:
> >
> > Thanks Xin, I added it to the 1.1.0 epic.
> >
> > -Taylor
> >
> > > On Nov 15, 2016, at 9:01 AM, Xin Wang <da...@gmail.com> wrote:
> > >
> > > STORM-2198 ( PR: https://github.com/apache/storm/pull/1773 ) fixes a
> bug
> > of
> > > storm-hdfs. Do we have a consideration to include this?
> > >
> > > Thanks,
> > > Xin Wang (vesense)
> > >
> > > 2016-11-15 10:03 GMT+08:00 Jungtaek Lim <ka...@gmail.com>:
> > >
> > >> Some issues on Storm SQL are resolved but not documented yet. I'll
> file
> > an
> > >> issue and assign to 1.1.0 release epic.
> > >> And also I want to address dropping aggregation and join on Storm SQL
> > >> Trident mode before releasing. I'll assign it too.
> > >>
> > >> - Jungtaek Lim (HeartSaVioR)
> > >>
> > >>
> > >> 2016년 11월 15일 (화) 오전 5:55, P. Taylor Goetz <pt...@gmail.com>님이 작성:
> > >>
> > >>> I think we’re very close. I would like to confirm that the 1.x-branch
> > is
> > >>> not affected by STORM-2176.
> > >>>
> > >>> The worker lifecycle API was added in 1.0, but doesn’t work in any
> > >>> released version due to STORM-2176.
> > >>>
> > >>> If there are any other open JIRAs that anyone is passionate about,
> now
> > >>> would be a good time to assign them to the 1.1.0 release epic
> > >> (STORM-1856).
> > >>>
> > >>> -Taylor
> > >>>
> > >>>
> > >>>
> > >>>> On Oct 27, 2016, at 12:19 PM, Jungtaek Lim <ka...@gmail.com>
> wrote:
> > >>>>
> > >>>> Finally Pacemaker H/A, Supervisor V2, and Storm SQL PRs which were
> > >> opened
> > >>>> at the last mail (4 weeks ago) are all merged to 1.x branch.
> > >>>>
> > >>>> There're some more PRs on Storm SQL opened, but given that we can
> > >> release
> > >>>> new minor at any time when we feel it's enough change, I can wait
> for
> > >> it.
> > >>>> They didn't get reviewed yet indeed.
> > >>>>
> > >>>> Is there something else we would want to include it to 1.1.0?
> > >>>>
> > >>>> Thanks,
> > >>>> Jungtaek Lim (HeartSaVioR)
> > >>>>
> > >>>> 2016년 10월 1일 (토) 오전 9:30, Jungtaek Lim <ka...@gmail.com>님이 작성:
> > >>>>
> > >>>>> Personally, merging and porting back to three branches are painful
> > >>> enough,
> > >>>>> especially we don't have merging script and having verbose process
> (I
> > >>> mean
> > >>>>> CHANGELOG).
> > >>>>> It would be better if merging process is automated (by running
> script
> > >> or
> > >>>>> so), so I'd +1 to revisit Harsha's suggestion (adopting Kafka merge
> > >>> script)
> > >>>>> and modify script to fit to Storm.
> > >>>>> (It will not work if it's the case we need to handle PRs for each
> > >>> version
> > >>>>> line, since 'Close' in commit log doesn't close the PR if its
> target
> > >>> branch
> > >>>>> is not master.)
> > >>>>>
> > >>>>> Anyway, without automation I don't want to maintain more version
> > >> lines.
> > >>>>> I'm looking at the announces from other projects, and others are
> only
> > >>>>> maintaining two version lines.
> > >>>>> Since we maintain 2.0.0 version line we can't reduce version lines
> to
> > >> 2,
> > >>>>> but hopefully at most 3.
> > >>>>>
> > >>>>> Btw, let's check pending pull requests and enumerate which can be
> > >>> included
> > >>>>> in 1.0.0, and start/finish review and merge them soon.
> > >>>>> For me Supervisor V2 and Pacemaker H/A, and pending Storm SQL PRs
> can
> > >> be
> > >>>>> included, since they are small or in reviewing and expected to pass
> > >>> review
> > >>>>> phase soon.
> > >>>>> (And some small PRs. There're other valuable PRs in PR list but I'm
> > >> not
> > >>>>> sure we can review them soon. One example is unified API.)
> > >>>>>
> > >>>>> One issue which is not clear is STORM-2006
> > >>>>> <https://github.com/apache/storm/pull/1595>. This is a candidate
> for
> > >>> me,
> > >>>>> but gets blocked while reviewing. If we plan to put great effort to
> > >>> revise
> > >>>>> Metric we can skip this.
> > >>>>>
> > >>>>> Please enumerate other PRs as well if you want to include in 1.1.0.
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Jungtaek Lim
> > >>>>>
> > >>>>> 2016년 9월 30일 (금) 오후 11:09, Bobby Evans <evans@yahoo-inc.com.invalid
> >
> > >> 님이
> > >>> 작성:
> > >>>>>
> > >>>>> Sounds good to me.  It would be nice to get some of the new
> features
> > >>> out.
> > >>>>> Do we expect to maintain both 1.0.x and 1.1.x lines with bug fixes?
> > >>> And if
> > >>>>> so for how long do we want to do this for? - Bobby
> > >>>>>
> > >>>>>   On Thursday, September 29, 2016 7:35 PM, Jungtaek Lim <
> > >>>>> kabhwan@gmail.com> wrote:
> > >>>>>
> > >>>>>
> > >>>>> Hi devs,
> > >>>>>
> > >>>>> It's been 5 months after releasing Storm 1.0.0, and now 1.x branch
> > has
> > >>> lots
> > >>>>> of CHANGELOG and also pending reviews.
> > >>>>> It's also been a long time after 1.1.0 RC1 is canceled.
> > >>>>>
> > >>>>> I think it may be good to put some efforts to review and merge
> > pending
> > >>> pull
> > >>>>> requests (except things which takes time to review and test), and
> > >>> release
> > >>>>> 1.1.0 soon.
> > >>>>>
> > >>>>> What do you think?
> > >>>>>
> > >>>>> I'm also open to volunteer release manager for 1.1.0 after we
> > document
> > >>> the
> > >>>>> process of official release.
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Jungtaek Lim (HeartSaVioR)
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>
> > >>>
> > >>
> >
>
>