You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by Ethan Li <et...@gmail.com> on 2019/09/05 14:55:28 UTC
Re: [VOTE] Release Apache Storm 2.1.0 (rc3)
Once another bug fix https://github.com/apache/storm/pull/3123 <https://github.com/apache/storm/pull/3123> is merged, I think we can run another release candidate.
> On Aug 28, 2019, at 9:27 AM, Ethan Li <et...@gmail.com> wrote:
>
> Thanks for finding the issue.
>
> I just uploaded my key to http://pgp.mit.edu/ <http://pgp.mit.edu/>. It’s a good idea to link to https://www.apache.org/info/verification.html <https://www.apache.org/info/verification.html> on download page.
>
>> On Aug 27, 2019, at 3:35 PM, Stig Rohde Døssing <stigdoessing@gmail.com <ma...@gmail.com>> wrote:
>>
>> -1, I think this issue is a blocker. It's caused by
>> https://github.com/apache/storm/pull/3093/files <https://github.com/apache/storm/pull/3093/files>, which prevents the
>> localizer from deleting blobs that are deleted from Nimbus, even if those
>> blobs are only scheduled for download.
>>
>> Have opened https://github.com/apache/storm/pull/3119 <https://github.com/apache/storm/pull/3119>
>>
>> Den tir. 27. aug. 2019 kl. 19.54 skrev Stig Rohde Døssing <
>> stigdoessing@gmail.com <ma...@gmail.com>>:
>>
>>> I think there may be an issue with topology cleanup
>>> I ran WordCountTopology for a bit, then killed it. The supervisor seems to
>>> keep trying to download the topology jar indefinitely.
>>>
>>> 2019-08-27 19:49:53.459 o.a.s.l.AsyncLocalizer AsyncLocalizer Executor - 1
>>> [WARN] Failed to download blob LOCAL TOPO BLOB TOPO_JAR
>>> word-count-1-1566926844 will try again in 100 ms
>>> org.apache.storm.generated.KeyNotFoundException: null
>>> at
>>> org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25919)
>>> ~[storm-client-2.1.0.jar:2.1.0]
>>> at
>>> org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25887)
>>> ~[storm-client-2.1.0.jar:2.1.0]
>>> at
>>> org.apache.storm.generated.Nimbus$getBlobMeta_result.read(Nimbus.java:25818)
>>> ~[storm-client-2.1.0.jar:2.1.0]
>>> at
>>> org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:88)
>>> ~[storm-shaded-deps-2.1.0.jar:2.1.0]
>>> at
>>> org.apache.storm.generated.Nimbus$Client.recv_getBlobMeta(Nimbus.java:794)
>>> ~[storm-client-2.1.0.jar:2.1.0]
>>> at org.apache.storm.generated.Nimbus$Client.getBlobMeta(Nimbus.java:781)
>>> ~[storm-client-2.1.0.jar:2.1.0]
>>> at
>>> org.apache.storm.blobstore.NimbusBlobStore.getBlobMeta(NimbusBlobStore.java:85)
>>> ~[storm-client-2.1.0.jar:2.1.0]
>>> at
>>> org.apache.storm.localizer.LocallyCachedTopologyBlob.getRemoteVersion(LocallyCachedTopologyBlob.java:127)
>>> ~[storm-server-2.1.0.jar:2.1.0]
>>> at
>>> org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:264)
>>> ~[storm-server-2.1.0.jar:2.1.0]
>>> at
>>> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
>>> [?:1.8.0_144]
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> [?:1.8.0_144]
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_144]
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>> [?:1.8.0_144]
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>> [?:1.8.0_144]
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> [?:1.8.0_144]
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> [?:1.8.0_144]
>>> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
>>>
>>> These repeat, intermittently broken up by
>>>
>>> Caused by: java.lang.RuntimeException: Could not download...
>>> at
>>> org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:285)
>>> ~[storm-server-2.1.0.jar:2.1.0]
>>> at
>>> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
>>> ~[?:1.8.0_144]
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> ~[?:1.8.0_144]
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_144]
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>> ~[?:1.8.0_144]
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>> ~[?:1.8.0_144]
>>> ... 3 more
>>> Caused by: org.apache.storm.generated.KeyNotFoundException
>>> at
>>> org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25919)
>>> ~[storm-client-2.1.0.jar:2.1.0]
>>> at
>>> org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25887)
>>> ~[storm-client-2.1.0.jar:2.1.0]
>>> at
>>> org.apache.storm.generated.Nimbus$getBlobMeta_result.read(Nimbus.java:25818)
>>> ~[storm-client-2.1.0.jar:2.1.0]
>>> at
>>> org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:88)
>>> ~[storm-shaded-deps-2.1.0.jar:2.1.0]
>>> at
>>> org.apache.storm.generated.Nimbus$Client.recv_getBlobMeta(Nimbus.java:794)
>>> ~[storm-client-2.1.0.jar:2.1.0]
>>> at org.apache.storm.generated.Nimbus$Client.getBlobMeta(Nimbus.java:781)
>>> ~[storm-client-2.1.0.jar:2.1.0]
>>> at
>>> org.apache.storm.blobstore.NimbusBlobStore.getBlobMeta(NimbusBlobStore.java:85)
>>> ~[storm-client-2.1.0.jar:2.1.0]
>>> at
>>> org.apache.storm.localizer.LocallyCachedTopologyBlob.getRemoteVersion(LocallyCachedTopologyBlob.java:127)
>>> ~[storm-server-2.1.0.jar:2.1.0]
>>> at
>>> org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:264)
>>> ~[storm-server-2.1.0.jar:2.1.0]
>>> at
>>> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
>>> ~[?:1.8.0_144]
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> ~[?:1.8.0_144]
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_144]
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>> ~[?:1.8.0_144]
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>> ~[?:1.8.0_144]
>>> ... 3 more
>>>
>>> Restarting the supervisor seems to fix it.
>>>
>>> Here's what I did before hitting this issue
>>>
>>> Verified asc and sha512 files.
>>> Built Storm from the zipped source.
>>> Extracted zipped binary and started dev-zookeeper, nimbus, supervisor,
>>> logviewer, ui.
>>> Ran WordCountTopology for a while.
>>> Checked rebalance, deactivate/activate, worker log viewing, kill from the
>>> UI.
>>>
>>> Minor things we might want to handle:
>>> It might be good to put your key up on a keyserver e.g.
>>> http://pgp.mit.edu/ <http://pgp.mit.edu/>, as that is where the
>>> https://www.apache.org/info/verification.html <https://www.apache.org/info/verification.html> page suggests users go to
>>> find keys.
>>> We should add a note to the Storm downloads page that links to the
>>> verification page. Currently users aren't informed how to check signatures
>>> and checksums.
>>>
>>> Den tir. 20. aug. 2019 kl. 23.05 skrev Ethan Li <ethanli@apache.org <ma...@apache.org>>:
>>>
>>>> This is a call to vote on releasing Apache Storm 2.1.0 (rc3)
>>>>
>>>> Full list of changes in this release:
>>>>
>>>>
>>>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc3/RELEASE_NOTES.html <https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc3/RELEASE_NOTES.html>
>>>>
>>>> The tag/commit to be voted upon is v2.1.0:
>>>>
>>>>
>>>> https://gitbox.apache.org/repos/asf?p=storm.git;a=tree;h=4288646ef99b97f9737dd9c21b7b6b62c8b17afa;hb=6562d6ee438c2858d92a51bd30ca0a1f2f623c77
>>>>
>>>> The source archive being voted upon can be found here:
>>>>
>>>>
>>>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc3/apache-storm-2.1.0-src.tar.gz
>>>>
>>>> Other release files, signatures and digests can be found here:
>>>>
>>>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc3
>>>>
>>>> The release artifacts are signed with the following key:
>>>>
>>>> https://www.apache.org/dist/storm/KEYS
>>>>
>>>> The Nexus staging repository for this release is:
>>>>
>>>> https://repository.apache.org/content/repositories/orgapachestorm-1087/
>>>>
>>>> Please vote on releasing this package as Apache Storm 2.1.0.
>>>>
>>>> When voting, please list the actions taken to verify the release.
>>>>
>>>> This vote will be open for at least 72 hours.
>>>>
>>>> [ ] +1 Release this package as Apache Storm 2.1.0
>>>> [ ] 0 No opinion
>>>> [ ] -1 Do not release this package because...
>>>>
>>>> Thanks to everyone who contributed to this release.
>>>>
>>>> Ethan
>>>>
>>>
>
Re: [VOTE] Release Apache Storm 2.1.0 (rc3)
Posted by Stig Rohde Døssing <st...@gmail.com>.
Sounds great, thanks Ethan.
Den tor. 5. sep. 2019 kl. 16.55 skrev Ethan Li <et...@gmail.com>:
> Once another bug fix https://github.com/apache/storm/pull/3123 <
> https://github.com/apache/storm/pull/3123> is merged, I think we can run
> another release candidate.
>
>
>
> > On Aug 28, 2019, at 9:27 AM, Ethan Li <et...@gmail.com> wrote:
> >
> > Thanks for finding the issue.
> >
> > I just uploaded my key to http://pgp.mit.edu/ <http://pgp.mit.edu/>.
> It’s a good idea to link to https://www.apache.org/info/verification.html
> <https://www.apache.org/info/verification.html> on download page.
> >
> >> On Aug 27, 2019, at 3:35 PM, Stig Rohde Døssing <stigdoessing@gmail.com
> <ma...@gmail.com>> wrote:
> >>
> >> -1, I think this issue is a blocker. It's caused by
> >> https://github.com/apache/storm/pull/3093/files <
> https://github.com/apache/storm/pull/3093/files>, which prevents the
> >> localizer from deleting blobs that are deleted from Nimbus, even if
> those
> >> blobs are only scheduled for download.
> >>
> >> Have opened https://github.com/apache/storm/pull/3119 <
> https://github.com/apache/storm/pull/3119>
> >>
> >> Den tir. 27. aug. 2019 kl. 19.54 skrev Stig Rohde Døssing <
> >> stigdoessing@gmail.com <ma...@gmail.com>>:
> >>
> >>> I think there may be an issue with topology cleanup
> >>> I ran WordCountTopology for a bit, then killed it. The supervisor
> seems to
> >>> keep trying to download the topology jar indefinitely.
> >>>
> >>> 2019-08-27 19:49:53.459 o.a.s.l.AsyncLocalizer AsyncLocalizer Executor
> - 1
> >>> [WARN] Failed to download blob LOCAL TOPO BLOB TOPO_JAR
> >>> word-count-1-1566926844 will try again in 100 ms
> >>> org.apache.storm.generated.KeyNotFoundException: null
> >>> at
> >>>
> org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25919)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25887)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.generated.Nimbus$getBlobMeta_result.read(Nimbus.java:25818)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:88)
> >>> ~[storm-shaded-deps-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.generated.Nimbus$Client.recv_getBlobMeta(Nimbus.java:794)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> org.apache.storm.generated.Nimbus$Client.getBlobMeta(Nimbus.java:781)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.blobstore.NimbusBlobStore.getBlobMeta(NimbusBlobStore.java:85)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.localizer.LocallyCachedTopologyBlob.getRemoteVersion(LocallyCachedTopologyBlob.java:127)
> >>> ~[storm-server-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:264)
> >>> ~[storm-server-2.1.0.jar:2.1.0]
> >>> at
> >>>
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
> >>> [?:1.8.0_144]
> >>> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> [?:1.8.0_144]
> >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [?:1.8.0_144]
> >>> at
> >>>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> >>> [?:1.8.0_144]
> >>> at
> >>>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> >>> [?:1.8.0_144]
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >>> [?:1.8.0_144]
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >>> [?:1.8.0_144]
> >>> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
> >>>
> >>> These repeat, intermittently broken up by
> >>>
> >>> Caused by: java.lang.RuntimeException: Could not download...
> >>> at
> >>>
> org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:285)
> >>> ~[storm-server-2.1.0.jar:2.1.0]
> >>> at
> >>>
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
> >>> ~[?:1.8.0_144]
> >>> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> ~[?:1.8.0_144]
> >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[?:1.8.0_144]
> >>> at
> >>>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> >>> ~[?:1.8.0_144]
> >>> at
> >>>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> >>> ~[?:1.8.0_144]
> >>> ... 3 more
> >>> Caused by: org.apache.storm.generated.KeyNotFoundException
> >>> at
> >>>
> org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25919)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25887)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.generated.Nimbus$getBlobMeta_result.read(Nimbus.java:25818)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:88)
> >>> ~[storm-shaded-deps-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.generated.Nimbus$Client.recv_getBlobMeta(Nimbus.java:794)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> org.apache.storm.generated.Nimbus$Client.getBlobMeta(Nimbus.java:781)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.blobstore.NimbusBlobStore.getBlobMeta(NimbusBlobStore.java:85)
> >>> ~[storm-client-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.localizer.LocallyCachedTopologyBlob.getRemoteVersion(LocallyCachedTopologyBlob.java:127)
> >>> ~[storm-server-2.1.0.jar:2.1.0]
> >>> at
> >>>
> org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:264)
> >>> ~[storm-server-2.1.0.jar:2.1.0]
> >>> at
> >>>
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
> >>> ~[?:1.8.0_144]
> >>> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> ~[?:1.8.0_144]
> >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[?:1.8.0_144]
> >>> at
> >>>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> >>> ~[?:1.8.0_144]
> >>> at
> >>>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> >>> ~[?:1.8.0_144]
> >>> ... 3 more
> >>>
> >>> Restarting the supervisor seems to fix it.
> >>>
> >>> Here's what I did before hitting this issue
> >>>
> >>> Verified asc and sha512 files.
> >>> Built Storm from the zipped source.
> >>> Extracted zipped binary and started dev-zookeeper, nimbus, supervisor,
> >>> logviewer, ui.
> >>> Ran WordCountTopology for a while.
> >>> Checked rebalance, deactivate/activate, worker log viewing, kill from
> the
> >>> UI.
> >>>
> >>> Minor things we might want to handle:
> >>> It might be good to put your key up on a keyserver e.g.
> >>> http://pgp.mit.edu/ <http://pgp.mit.edu/>, as that is where the
> >>> https://www.apache.org/info/verification.html <
> https://www.apache.org/info/verification.html> page suggests users go to
> >>> find keys.
> >>> We should add a note to the Storm downloads page that links to the
> >>> verification page. Currently users aren't informed how to check
> signatures
> >>> and checksums.
> >>>
> >>> Den tir. 20. aug. 2019 kl. 23.05 skrev Ethan Li <ethanli@apache.org
> <ma...@apache.org>>:
> >>>
> >>>> This is a call to vote on releasing Apache Storm 2.1.0 (rc3)
> >>>>
> >>>> Full list of changes in this release:
> >>>>
> >>>>
> >>>>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc3/RELEASE_NOTES.html
> <
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc3/RELEASE_NOTES.html
> >
> >>>>
> >>>> The tag/commit to be voted upon is v2.1.0:
> >>>>
> >>>>
> >>>>
> https://gitbox.apache.org/repos/asf?p=storm.git;a=tree;h=4288646ef99b97f9737dd9c21b7b6b62c8b17afa;hb=6562d6ee438c2858d92a51bd30ca0a1f2f623c77
> >>>>
> >>>> The source archive being voted upon can be found here:
> >>>>
> >>>>
> >>>>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc3/apache-storm-2.1.0-src.tar.gz
> >>>>
> >>>> Other release files, signatures and digests can be found here:
> >>>>
> >>>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc3
> >>>>
> >>>> The release artifacts are signed with the following key:
> >>>>
> >>>> https://www.apache.org/dist/storm/KEYS
> >>>>
> >>>> The Nexus staging repository for this release is:
> >>>>
> >>>>
> https://repository.apache.org/content/repositories/orgapachestorm-1087/
> >>>>
> >>>> Please vote on releasing this package as Apache Storm 2.1.0.
> >>>>
> >>>> When voting, please list the actions taken to verify the release.
> >>>>
> >>>> This vote will be open for at least 72 hours.
> >>>>
> >>>> [ ] +1 Release this package as Apache Storm 2.1.0
> >>>> [ ] 0 No opinion
> >>>> [ ] -1 Do not release this package because...
> >>>>
> >>>> Thanks to everyone who contributed to this release.
> >>>>
> >>>> Ethan
> >>>>
> >>>
> >
>
>