You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by Richard Zowalla <rz...@apache.org> on 2023/11/02 19:24:53 UTC
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Short update. Sorted out an issue with the tar.gz/zip with Julien today
and re-uploaded them to the nightlies area.
This new bundle works as expected in my deployment but happy to receive
additional feedback before getting up a first release candidate ;-)
Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard Zowalla:
> Hi Alexandre,
>
> we are not in a hurry here :) - take as much as time you need.
>
> Gruß
> Richard
>
>
> Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> Vermeerbergen:
> > Hello Richard,
> >
> > Okay, I'm more than happy to do that with my pre-production cluster
> > (~10 topologies) using the binary artifacts.
> > Would it be OK if I can use up to end of this week so that I'll be
> > able to have enough time to check all potential issues that this
> > upgrade bring?
> >
> > Thanks,
> > Alexandre
> >
> > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla <rz...@apache.org> a
> > écrit :
> > >
> > > Hi all,
> > >
> > > before starting a release of 2.6.0-SNAPSHOT, I would like to
> > > receive
> > > some feedback on the current SNAPSHOT build.
> > >
> > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > >
> > > It is available via the ASF Snapshot repository:
> > > https://repository.apache.org/content/repositories/snapshots/
> > >
> > > You can consume by adding
> > >
> > > <repository>
> > > <id>apache.snapshots</id>
> > > <name>Apache Snapshot Repository</name>
> > > <url>https://repository.apache.org/snapshots</url>
> > > <releases>
> > > <enabled>false</enabled>
> > > </releases>
> > > </repository>
> > >
> > > to your project pom. As we do not deploy SNAPSHOTS automatically,
> > > it
> > > should be easy to just consume the latest SNAPSHOT.
> > >
> > > The packaged binaries are available at nightlies.apache.org:
> > >
> > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > >
> > > If you have some minutes left: Please test and report any issues
> > > with
> > > this binaries, so we can fix before attempting to release.
> > >
> > > The most significant changes are in the are of hadoop/hbase/hdfs
> > > as
> > > we
> > > upgraded from 2.x to 3.x - our own test coverage within the build
> > > looks
> > > good but would be nice to get some real world use-case feedback
> > > on
> > > this. In addition, we had quite a lof of 3rd party dependency
> > > upgrades.
> > >
> > > In addition, it contains the pruning of external modules as
> > > listed
> > > in
> > >
> > > https://issues.apache.org/jira/browse/STORM-3988
> > >
> > > A summaryin Jira is here:
> > >
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > >
> > > Gruß
> > > Richard
> > >
> > >
>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Apologies: I have been way too busy many unrelated stuff (including,
but not limited to bad weather, which is ironic when thinking about
Storm ;); so I haven't yet started testing pre-vote Storm 2.6.0.
Trying to bootstrap this activity tomorrow, if the sky is nice enough
not try falling on my head twice!
Alexandre
Le jeu. 2 nov. 2023 à 20:24, Richard Zowalla <rz...@apache.org> a écrit :
>
> Short update. Sorted out an issue with the tar.gz/zip with Julien today
> and re-uploaded them to the nightlies area.
>
> This new bundle works as expected in my deployment but happy to receive
> additional feedback before getting up a first release candidate ;-)
>
> Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard Zowalla:
> > Hi Alexandre,
> >
> > we are not in a hurry here :) - take as much as time you need.
> >
> > Gruß
> > Richard
> >
> >
> > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > Vermeerbergen:
> > > Hello Richard,
> > >
> > > Okay, I'm more than happy to do that with my pre-production cluster
> > > (~10 topologies) using the binary artifacts.
> > > Would it be OK if I can use up to end of this week so that I'll be
> > > able to have enough time to check all potential issues that this
> > > upgrade bring?
> > >
> > > Thanks,
> > > Alexandre
> > >
> > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla <rz...@apache.org> a
> > > écrit :
> > > >
> > > > Hi all,
> > > >
> > > > before starting a release of 2.6.0-SNAPSHOT, I would like to
> > > > receive
> > > > some feedback on the current SNAPSHOT build.
> > > >
> > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > >
> > > > It is available via the ASF Snapshot repository:
> > > > https://repository.apache.org/content/repositories/snapshots/
> > > >
> > > > You can consume by adding
> > > >
> > > > <repository>
> > > > <id>apache.snapshots</id>
> > > > <name>Apache Snapshot Repository</name>
> > > > <url>https://repository.apache.org/snapshots</url>
> > > > <releases>
> > > > <enabled>false</enabled>
> > > > </releases>
> > > > </repository>
> > > >
> > > > to your project pom. As we do not deploy SNAPSHOTS automatically,
> > > > it
> > > > should be easy to just consume the latest SNAPSHOT.
> > > >
> > > > The packaged binaries are available at nightlies.apache.org:
> > > >
> > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > >
> > > > If you have some minutes left: Please test and report any issues
> > > > with
> > > > this binaries, so we can fix before attempting to release.
> > > >
> > > > The most significant changes are in the are of hadoop/hbase/hdfs
> > > > as
> > > > we
> > > > upgraded from 2.x to 3.x - our own test coverage within the build
> > > > looks
> > > > good but would be nice to get some real world use-case feedback
> > > > on
> > > > this. In addition, we had quite a lof of 3rd party dependency
> > > > upgrades.
> > > >
> > > > In addition, it contains the pruning of external modules as
> > > > listed
> > > > in
> > > >
> > > > https://issues.apache.org/jira/browse/STORM-3988
> > > >
> > > > A summaryin Jira is here:
> > > >
> > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > >
> > > > Gruß
> > > > Richard
> > > >
> > > >
> >
>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
At this moment that's the CVE which I found using Storm 2.6.0 snapshot 20231102.
And since there's a commons-text-1.11.0.jar available, I guess we
should directly jump to this version.
Le jeu. 9 nov. 2023 à 08:49, Alexandre Vermeerbergen
<av...@gmail.com> a écrit :
>
> Yet another finding on my side to "not call yet for vote on a Storm
> 2.6.0", we have some CVEs and we shoudn't release a version having
> known CVEs.
>
> For example, we have
>
> find . -name "commons-text*jar" -print
> ./lib-webapp/commons-text-1.9.jar
> ./external/storm-autocreds/commons-text-1.10.0.jar
>
> The second one is okay from
> https://www.cvedetails.com/cve/CVE-2022-42889/ perspective, but not
> the first one.
>
> I will continue checking if there are other CVEs...
>
> Thanks,
> Alexandre
>
> Le mar. 7 nov. 2023 à 17:43, Alexandre Vermeerbergen
> <av...@gmail.com> a écrit :
> >
> > Oh thank you very much for this cherry-pinking, I was too busy doing
> > other unrelated stuff..
> >
> > Le mar. 7 nov. 2023 à 17:35, Richard Zowalla <rz...@apache.org> a écrit :
> > >
> > > I cherry-picked your commit and put that on master
> > >
> > >
> > > Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard Zowalla:
> > > > You pushed on your own branch. You would need to push to apache/storm
> > > > main branch or open a PR ;-)
> > > >
> > > > Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb Alexandre
> > > > Vermeerbergen:
> > > > > Just did a 'git push' with that trivial fix... is that enough?
> > > > >
> > > > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla <ri...@zowalla.com>
> > > > > a
> > > > > écrit :
> > > > > >
> > > > > > You can do as you like ;-)
> > > > > >
> > > > > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen
> > > > > > <av...@gmail.com>:
> > > > > > > I mean :
> > > > > > >
> > > > > > > Can I keep using my
> > > > > > > 'STORM-3958-Capacity-to-set-St
> > > > > > > orm-UI's-title-in-conf/storm.yaml'
> > > > > > > branch to push this fix, or should I create a new one?
> > > > > > >
> > > > > > > Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > > > > > <av...@gmail.com> a écrit :
> > > > > > > >
> > > > > > > > Can you keep using my
> > > > > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > > > > > conf/storm.yaml'
> > > > > > > > branch to push this fix, or should I create a new one?
> > > > > > > >
> > > > > > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > > > > > <ri...@zowalla.com> a écrit :
> > > > > > > > >
> > > > > > > > > Just push a fix for it (referencing the original issue) ;-)
> > > > > > > > > -
> > > > > > > > > we hadn't a release in between, so not a big deal.
> > > > > > > > >
> > > > > > > > > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre
> > > > > > > > > Vermeerbergen <av...@gmail.com>:
> > > > > > > > > > Temporary workaround for the Storm UI title's regression:
> > > > > > > > > > run this
> > > > > > > > > > command on the title-template.html file (in
> > > > > > > > > > storm-stable/public/templates directory) after having
> > > > > > > > > > extracted the
> > > > > > > > > > binary archive:
> > > > > > > > > >
> > > > > > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > > > > > > > > >
> > > > > > > > > > Again sorry for having introduced this regression.
> > > > > > > > > >
> > > > > > > > > > Alexandre
> > > > > > > > > >
> > > > > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > >
> > > > > > > > > > > Updates regarding my test with Storm 2.6.0 Snapshot
> > > > > > > > > > > 2023/11/02:
> > > > > > > > > > > 1. I have modified my KO bolts using this pattern in
> > > > > > > > > > > their prepare() method:
> > > > > > > > > > >
> > > > > > > > > > > long cacheMaxSize =
> > > > > > > > > > > Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZ
> > > > > > > > > > > E)
> > > > > > > > > > > .toString());
> > > > > > > > > > >
> > > > > > > > > > > instead of:
> > > > > > > > > > >
> > > > > > > > > > > long cacheMaxSize = (Long)
> > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > > > >
> > > > > > > > > > > 2. I have found the cause of the regression in loss of
> > > > > > > > > > > 'Storm UI'
> > > > > > > > > > > title at the top left-hand side of all Storm UI's
> > > > > > > > > > > pages.
> > > > > > > > > > > This is a mistake of mine in this PR:
> > > > > > > > > > >
> > > > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > > > > > > > > More precisely, at line 19 of this file:
> > > > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > > > > > > > > > we must have:
> > > > > > > > > > > <h1><a href="/">{{title}}</a></h1>
> > > > > > > > > > >
> > > > > > > > > > > instead of:
> > > > > > > > > > >
> > > > > > > > > > > <h1><a href="/">{{ui.title}}</a></h1>
> > > > > > > > > > >
> > > > > > > > > > > I think I missed this during my tests because of
> > > > > > > > > > > browser's cache
> > > > > > > > > > > (bad excuse, I know...): probably I tested by-patching
> > > > > > > > > > > my
> > > > > > > > > > > deployed
> > > > > > > > > > > Storm cluster, and I finally forgot to checkout this
> > > > > > > > > > > latest change of
> > > > > > > > > > > my delivery.
> > > > > > > > > > >
> > > > > > > > > > > I had to use "private navigation" to make sure this
> > > > > > > > > > > time it fixes this issue
> > > > > > > > > > >
> > > > > > > > > > > What would be the quickest way for this very simple fix
> > > > > > > > > > > to be delivered ?
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > Alexandre
> > > > > > > > > > >
> > > > > > > > > > > Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks Richard for the analysis.
> > > > > > > > > > > >
> > > > > > > > > > > > My thought on this are very simple: let me fix my own
> > > > > > > > > > > > old code, after
> > > > > > > > > > > > all this was impacting only one of my topologies, the
> > > > > > > > > > > > other one were
> > > > > > > > > > > > using more careful Integer.parseInt or
> > > > > > > > > > > > Integer.parseLong to avoid
> > > > > > > > > > > > making assumption on the way the values in the config
> > > > > > > > > > > > Map were
> > > > > > > > > > > > serialized.
> > > > > > > > > > > >
> > > > > > > > > > > > My other concern with 2.6.0 Snapshot is that in Storm
> > > > > > > > > > > > UI, the "Storm
> > > > > > > > > > > > UI" title at the top left hand side of each page is
> > > > > > > > > > > > missing. This may
> > > > > > > > > > > > be the result of one of my own PR, I'll have to check
> > > > > > > > > > > > why...
> > > > > > > > > > > >
> > > > > > > > > > > > Alex
> > > > > > > > > > > >
> > > > > > > > > > > > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla
> > > > > > > > > > > > <rz...@apache.org> a écrit :
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hi,
> > > > > > > > > > > > >
> > > > > > > > > > > > > thanks for the feedback.
> > > > > > > > > > > > >
> > > > > > > > > > > > > The main difference to 2.5.0 is, that we switched
> > > > > > > > > > > > > the
> > > > > > > > > > > > > JSON
> > > > > > > > > > > > > implementation in Utils, which relied on a
> > > > > > > > > > > > > super old JSON library. What is happening here is,
> > > > > > > > > > > > > that the new JSON
> > > > > > > > > > > > > library (net.minidev) behaves differently.
> > > > > > > > > > > > >
> > > > > > > > > > > > > We can see it by writing some simple code without
> > > > > > > > > > > > > Storm (The related
> > > > > > > > > > > > > code line with the parsing is taken from Storm
> > > > > > > > > > > > > Utils)
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Map<String, Long> conf = new HashMap<>();
> > > > > > > > > > > > > conf.put("test", 100000000L);
> > > > > > > > > > > > >
> > > > > > > > > > > > > Map<String, Object> deserTopoConf =
> > > > > > > > > > > > > (Map<String,
> > > > > > > > > > > > > Object>)
> > > > > > > > > > > > > JSONValue.parseWithException(JSONValue.toJSONString
> > > > > > > > > > > > > (c
> > > > > > > > > > > > > onf));
> > > > > > > > > > > > >
> > > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > >
> > > > > > > > > > > > > System.out.println(e.getValue().getClass().getSimpl
> > > > > > > > > > > > > eN
> > > > > > > > > > > > > ame());
> > > > > > > > > > > > > }
> > > > > > > > > > > > >
> > > > > > > > > > > > > This will print "Integer". Using
> > > > > > > > > > > > > "10000000000000000L"
> > > > > > > > > > > > > as a value will
> > > > > > > > > > > > > result in "Long".
> > > > > > > > > > > > >
> > > > > > > > > > > > > The main issue derives from the fact, that we
> > > > > > > > > > > > > serialize the config map
> > > > > > > > > > > > > into a JSON String and parse the result.
> > > > > > > > > > > > > Based on the String representation, "100000000L"
> > > > > > > > > > > > > will
> > > > > > > > > > > > > still fit into an
> > > > > > > > > > > > > Integer, so the parser will not use Long ;-)
> > > > > > > > > > > > >
> > > > > > > > > > > > > If the number is big enough, the parser will switch
> > > > > > > > > > > > > to a Long> >> > >> > > The old library (> 10 years
> > > > > > > > > > > > > old now):
> > > > > > > > > > > > >
> > > > > > > > > > > > > deserTopoConf = (Map<String, Object>)
> > > > > > > > > > > > > org.json.simple.JSONValue.parseWithException(org.js
> > > > > > > > > > > > > on
> > > > > > > > > > > > > .simple.JSONValue.
> > > > > > > > > > > > > toJSONString(conf));
> > > > > > > > > > > > >
> > > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > >
> > > > > > > > > > > > > System.out.println(e.getValue().getClass().getSimpl
> > > > > > > > > > > > > eN
> > > > > > > > > > > > > ame());
> > > > > > > > > > > > > }
> > > > > > > > > > > > >
> > > > > > > > > > > > > This will print "Long".
> > > > > > > > > > > > >
> > > > > > > > > > > > > The difference between these libraries is, that the
> > > > > > > > > > > > > super old
> > > > > > > > > > > > > org.json.simple just always returns Long as the
> > > > > > > > > > > > > type.
> > > > > > > > > > > > >
> > > > > > > > > > > > > If can eventually tune the new JSON parser to
> > > > > > > > > > > > > something like:
> > > > > > > > > > > > >
> > > > > > > > > > > > > deserTopoConf = (Map<String, Object>) new
> > > > > > > > > > > > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse
> > > > > > > > > > > > > (J
> > > > > > > > > > > > > SONValue= toJSONStr
> > > > > > > > > > > > > ing(conf), JSONValue.defaultReader.DEFAULT);
> > > > > > > > > > > > >
> > > > > > > > > > > > > This will result in "Long" but might have other
> > > > > > > > > > > > > side-
> > > > > > > > > > > > > effects.
> > > > > > > > > > > > >
> > > > > > > > > > > > > From my point of view, the newer JSON library works
> > > > > > > > > > > > > as expected
> > > > > > > > > > > > > (although it might break backward compatibility).
> > > > > > > > > > > > > Unchecked casts are always evil, imho. In your
> > > > > > > > > > > > > specific case, it might
> > > > > > > > > > > > > be more save to cast to "Number" and than invoke
> > > > > > > > > > > > > "longValue()" instead.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Nevertheless, we might being able to restore the
> > > > > > > > > > > > > old
> > > > > > > > > > > > > behaviour by
> > > > > > > > > > > > > tuning the JSONParser configuration.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thoughts?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > Richard
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb
> > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > On a side note, the Long value that "MyKOBolt" is
> > > > > > > > > > > > > > set by a
> > > > > > > > > > > > > > DefaultTopologyConfig to a value with is a
> > > > > > > > > > > > > > java.lang.Long instance...
> > > > > > > > > > > > > > hence it's surprising to see it internally
> > > > > > > > > > > > > > converted into a
> > > > > > > > > > > > > > java.lang.Integer in the Map passed to prepare()
> > > > > > > > > > > > > > method...
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 18:27, Alexandre
> > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Hello,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I found a breaking change using Apache Storm
> > > > > > > > > > > > > > > 2.6.0 snapshot build
> > > > > > > > > > > > > > > dated 20231103 and without changing my
> > > > > > > > > > > > > > > topologies' code (keeping
> > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > same big jars).
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > This seems to be tied to a datatype change in
> > > > > > > > > > > > > > > Map
> > > > > > > > > > > > > > > argument of
> > > > > > > > > > > > > > > prepare() method of Bolt.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > For example, this Bolt is KO at runtime:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > public class MyKOBolt extends BaseRichBolt {
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > public void prepare(Map stormConf,
> > > > > > > > > > > > > > > TopologyContext context,
> > > > > > > > > > > > > > > OutputCollector collector) {
> > > > > > > > > > > > > > > cacheMaxSize = (Long)
> > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > > > > > > > > cacheTtlMin = (Long)
> > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > as it throws this error:
> > > > > > > > > > > > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > > > > > > > > > > > > > > Thread-14-__system-executor[-1, -1] [INFO]
> > > > > > > > > > > > > > > Preparing bolt
> > > > > > > > > > > > > > > __system:[-1]
> > > > > > > > > > > > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > > > > > > > > > > > > > Thread-15-evaluateTriggers-executor[11, 11]
> > > > > > > > > > > > > > > [ERROR] Async loop
> > > > > > > > > > > > > > > died!
> > > > > > > > > > > > > > > java.lang.ClassCastException: java.lang.Integer
> > > > > > > > > > > > > > > incompatible with
> > > > > > > > > > > > > > > java.lang.Long
> > > > > > > > > > > > > > > at com.acme.storm.alerting.MyKOBolt
> > > > > > > > > > > > > > > .prepare(EvaluationBolt.java:71)
> > > > > > > > > > > > > > > ~[stormjar.jar:?]
> > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.ini
> > > > > > > > > > > > > > > t(
> > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > 128)
> > > > > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.cal
> > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > 138)
> > > > > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.cal
> > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > 54)
> > > > > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > org.apache.storm.utils.Utils$1.run(Utils.java:3
> > > > > > > > > > > > > > > 93
> > > > > > > > > > > > > > > )
> > > > > > > > > > > > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > whereas this bolt is fine with Storm 2.6.0:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > public class MyOKBolt extends BaseBasicBolt {
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > public void prepare(final Map stormConf,
> > > > > > > > > > > > > > > final TopologyContext
> > > > > > > > > > > > > > > context) {
> > > > > > > > > > > > > > > super.prepare(stormConf, context);
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > this.redisPort =
> > > > > > > > > > > > > > > Integer.parseInt((String)
> > > > > > > > > > > > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > That said, I can modify the code of "MyKOBolt"
> > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > use same
> > > > > > > > > > > > > > > Integer.parseInt or Integer.parseLong trick,
> > > > > > > > > > > > > > > but
> > > > > > > > > > > > > > > it's the first
> > > > > > > > > > > > > > > time
> > > > > > > > > > > > > > > in my long history of upgrades that I have seen
> > > > > > > > > > > > > > > such runtime
> > > > > > > > > > > > > > > incompatibility.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Maybe "MyKOBolt" was badly written since the
> > > > > > > > > > > > > > > beginning and I have
> > > > > > > > > > > > > > > just
> > > > > > > > > > > > > > > hit the punishment for it: is there a
> > > > > > > > > > > > > > > documentation which clarifies
> > > > > > > > > > > > > > > datatypes of the Map argument of prepare()
> > > > > > > > > > > > > > > method?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Also worth noting: "MyKOBolt" derives from
> > > > > > > > > > > > > > > BaseRichBolt , while
> > > > > > > > > > > > > > > "MyOKBolt" derives from BaseBasicBolt => could
> > > > > > > > > > > > > > > this have any impact
> > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > this finding ?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Last, I'm running this on Redhat Linux 8 and
> > > > > > > > > > > > > > > IBM
> > > > > > > > > > > > > > > Semeru JDK
> > > > > > > > > > > > > > > 17.0.8.1.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > > > > > > > > > > > > > > <li...@gmail.com> a écrit :
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Thanks Richard.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Tried the latest snapshot with StormCrawler
> > > > > > > > > > > > > > > > both in local and
> > > > > > > > > > > > > > > > deployed mode
> > > > > > > > > > > > > > > > and did not find any issues.
> > > > > > > > > > > > > > > > Will try it on a topology generating WARC
> > > > > > > > > > > > > > > > files
> > > > > > > > > > > > > > > > next week to
> > > > > > > > > > > > > > > > check that the
> > > > > > > > > > > > > > > > dependency changes on Hadoop have not broken
> > > > > > > > > > > > > > > > anything.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Have a good week end
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Julien
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla
> > > > > > > > > > > > > > > > <rz...@apache.org>
> > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Short update. Sorted out an issue with the
> > > > > > > > > > > > > > > > > tar.gz/zip with
> > > > > > > > > > > > > > > > > Julien today
> > > > > > > > > > > > > > > > > and re-uploaded them to the nightlies area.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > This new bundle works as expected in my
> > > > > > > > > > > > > > > > > deployment but happy to
> > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > additional feedback before getting up a
> > > > > > > > > > > > > > > > > first
> > > > > > > > > > > > > > > > > release candidate
> > > > > > > > > > > > > > > > > ;-)
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100
> > > > > > > > > > > > > > > > > schrieb Richard
> > > > > > > > > > > > > > > > > Zowalla:
> > > > > > > > > > > > > > > > > > Hi Alexandre,
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > we are not in a hurry here :) - take as
> > > > > > > > > > > > > > > > > > much as time you
> > > > > > > > > > > > > > > > > > need.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100
> > > > > > > > > > > > > > > > > > schrieb Alexandre
> > > > > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > > > > Hello Richard,
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Okay, I'm more than happy to do that
> > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > my pre-production
> > > > > > > > > > > > > > > > > > > cluster
> > > > > > > > > > > > > > > > > > > (~10 topologies) using the binary
> > > > > > > > > > > > > > > > > > > artifacts.
> > > > > > > > > > > > > > > > > > > Would it be OK if I can use up to end
> > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > this week so that
> > > > > > > > > > > > > > > > > > > I'll be
> > > > > > > > > > > > > > > > > > > able to have enough time to check all
> > > > > > > > > > > > > > > > > > > potential issues that
> > > > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > > > upgrade bring?
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard
> > > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > > <rz...@apache.org> a
> > > > > > > > > > > > > > > > > > > écrit :
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > before starting a release of 2.6.0-
> > > > > > > > > > > > > > > > > > > > SNAPSHOT, I would like
> > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > > > some feedback on the current SNAPSHOT
> > > > > > > > > > > > > > > > > > > > build.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of
> > > > > > > > > > > > > > > > > > > > Storm build from
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > It is available via the ASF Snapshot
> > > > > > > > > > > > > > > > > > > > repository:
> > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > You can consume by adding
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > <repository>
> > > > > > > > > > > > > > > > > > > > <id>apache.snapshots</id>
> > > > > > > > > > > > > > > > > > > > <name>Apache Snapshot
> > > > > > > > > > > > > > > > > > > > Repository</name>
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > <url>
> > > > > > > > > > > > > > > > > > > > https://repository.apache.org/snap
> > > > > > > > > > > > > > > > > > > > shots</url>
> > > > > > > > > > > > > > > > > > > > <releases>
> > > > > > > > > > > > > > > > > > > > <enabled>false</enabled>
> > > > > > > > > > > > > > > > > > > > </releases>
> > > > > > > > > > > > > > > > > > > > </repository>
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > to your project pom. As we do not
> > > > > > > > > > > > > > > > > > > > deploy SNAPSHOTS
> > > > > > > > > > > > > > > > > > > > automatically,
> > > > > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > > > should be easy to just consume the
> > > > > > > > > > > > > > > > > > > > latest SNAPSHOT.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > The packaged binaries are available
> > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > nightlies.apache.org:
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > If you have some minutes left: Please
> > > > > > > > > > > > > > > > > > > > test and report any
> > > > > > > > > > > > > > > > > > > > issues
> > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > this binaries, so we can fix before
> > > > > > > > > > > > > > > > > > > > attempting to
> > > > > > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > The most significant changes are in
> > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > are of
> > > > > > > > > > > > > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > > upgraded from 2.x to 3.x - our own
> > > > > > > > > > > > > > > > > > > > test
> > > > > > > > > > > > > > > > > > > > coverage within
> > > > > > > > > > > > > > > > > > > > the build
> > > > > > > > > > > > > > > > > > > > looks
> > > > > > > > > > > > > > > > > > > > good but would be nice to get some
> > > > > > > > > > > > > > > > > > > > real world use-case
> > > > > > > > > > > > > > > > > > > > feedback
> > > > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > > this. In addition, we had quite a lof
> > > > > > > > > > > > > > > > > > > > of 3rd party
> > > > > > > > > > > > > > > > > > > > dependency
> > > > > > > > > > > > > > > > > > > > upgrades.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > In addition, it contains the pruning
> > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > external modules
> > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > listed
> > > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > A summaryin Jira is here:
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > *Open Source Solutions for Text Engineering*
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > http://www.digitalpebble.com
> > > > > > > > > > > > > > > > http://digitalpebble.blogspot.com/
> > > > > > > > > > > > > > > > #digitalpebble
> > > > > > > > > > > > > > > > <http://twitter.com/digitalpebble>
> > > > > > > > > > > > >
> > > > >
> > > >
> > >
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Richard Zowalla <ri...@zowalla.com>.
Here is the fix:
https://github.com/apache/storm/commit/d640cea65b1cbc71c85c220073e68442c3269a93
I will provide a new snapshot soon (= new thread as this one is really
long), so Julien can do his tests on the hdfs/hadoop stuff in the
context of crawling work ;-)
Gruß
Richard
Am Montag, dem 13.11.2023 um 22:54 +0100 schrieb Alexandre
Vermeerbergen:
> Yet another regression on in-work 2.6.0 : the supervisor.html view is
> broken, there's a Javascript syntax error (probably my fault again).
> It' s bit late for me, I'll check that tomorrow...
>
> Alexandre
>
> Le lun. 13 nov. 2023 à 15:36, Julien Nioche
> <li...@gmail.com> a écrit :
> >
> > I agree with Richard. The next release contains enough big changes
> > and it
> > would be good to release it ASAP before embarking on another big
> > modification.
> >
> > Julien
> >
> > On Mon, 13 Nov 2023 at 12:08, Richard Zowalla <rz...@apache.org>
> > wrote:
> >
> > > Hi,
> > >
> > > yes - in principal I agree, that switching to slf4j 2.x would be
> > > great
> > > in the near future. However, it requires some more work due to
> > > our old
> > > dependency stack ("external modules", ...) and most likely a lot
> > > of
> > > exclusions.
> > >
> > > Therefore, personally, I wouldn't see it in a 2.6.0 release but
> > > as a
> > > task on which we could work after having a new version in the
> > > pipeline
> > >
> > > wdyt?
> > >
> > > Gruß
> > > Richard
> > >
> > > Am Sonntag, dem 12.11.2023 um 09:40 +0100 schrieb Alexandre
> > > Vermeerbergen:
> > > > Thank Richard for the great work to cleanup CVEs on in-work
> > > > 2.6.0 !
> > > > One last "regret" I have with current in-work 2.6.0 is that we
> > > > still
> > > > rely on very old slf4j 1.x.
> > > > While it was not on slf4j itself, the log4shell CVE should
> > > > remind us
> > > > that we should keep loggers as up to date as possible.
> > > >
> > > > I see in a deployed Storm runtime the following hits:
> > > >
> > > > [root]# find . -name "slf*jar" -print
> > > > ./lib/slf4j-api-1.7.36.jar
> > > > ./lib-tools/storm-kafka-monitor/slf4j-api-1.7.36.jar
> > > > ./lib-tools/submit-tools/slf4j-api-1.7.36.jar
> > > > ./lib-tools/sql/core/slf4j-api-1.7.36.jar
> > > > ./lib-tools/sql/runtime/slf4j-api-1.7.36.jar
> > > > ./external/storm-autocreds/slf4j-api-1.7.36.jar
> > > > ./lib-worker/slf4j-api-1.7.36.jar
> > > > ./lib-webapp/slf4j-api-1.7.36.jar
> > > >
> > > > From https://www.slf4j.org/download.html it is said that "The
> > > > older
> > > > stable SLF4J version is 1.7.36. It is no longer actively
> > > > developed."
> > > >
> > > > The most current version of slf4j is 2.0.9.
> > > >
> > > > But I think at same time we would need to update these ones
> > > > too,
> > > > otherwise we will have a mismatch between SLF4J v1 logger
> > > > implementations vs new v2 APIs:
> > > >
> > > > [root@ip-172-31-43-21 storm-stable]# find . -name "log*jar" -
> > > > print
> > > > ./lib/log4j-over-slf4j-1.7.36.jar
> > > > ./lib/log4j-api-2.20.0.jar
> > > > ./lib/log4j-slf4j-impl-2.20.0.jar
> > > > ./lib/log4j-core-2.20.0.jar
> > > > ./external/storm-autocreds/log4j-api-2.20.0.jar
> > > > ./external/storm-autocreds/log4j-web-2.20.0.jar
> > > > ./external/storm-autocreds/log4j-1.2-api-2.20.0.jar
> > > > ./external/storm-autocreds/log4j-core-2.20.0.jar
> > > > ./lib-worker/log4j-over-slf4j-1.7.36.jar
> > > > ./lib-worker/log4j-api-2.20.0.jar
> > > > ./lib-worker/log4j-slf4j-impl-2.20.0.jar
> > > > ./lib-worker/log4j-core-2.20.0.jar
> > > > ./lib-webapp/log4j-over-slf4j-1.7.36.jar
> > > > ./lib-webapp/log4j-api-2.20.0.jar
> > > > ./lib-webapp/log4j-core-2.20.0.jar
> > > >
> > > > Disclaimer: I have no idea how hard this update would be.
> > > >
> > > > Anyone with good experience at updating SLF4J & related loggers
> > > > could
> > > > react on this proposal ?
> > > >
> > > > Thanks,
> > > > Alexandre
> > > >
> > > >
> > > > Le jeu. 9 nov. 2023 à 15:04, Richard Zowalla <rz...@apache.org>
> > > > a
> > > > écrit :
> > > > >
> > > > > Ok, most of the stuff reported by grype should be fixed now.
> > > > > I might be able to push a new snapshot within the next week,
> > > > > so we
> > > > > can
> > > > > do some testing and than go for a release candiate / vote :)
> > > > >
> > > > > Gruß
> > > > > Richard
> > > > >
> > > > >
> > > > >
> > > > > Am Donnerstag, dem 09.11.2023 um 08:53 +0100 schrieb Richard
> > > > > Zowalla:
> > > > > > Maybe just run grype [1] on the lib directories and post
> > > > > > the
> > > > > > output
> > > > > > on
> > > > > > the list / thread, so we can create tickets for it
> > > > > >
> > > > > > There might be some false positives but will give good
> > > > > > insights.
> > > > > >
> > > > > > Gruß
> > > > > > Richard
> > > > > >
> > > > > >
> > > > > > [1] https://github.com/anchore/grype
> > > > > > Am Donnerstag, dem 09.11.2023 um 08:49 +0100 schrieb
> > > > > > Alexandre
> > > > > > Vermeerbergen:
> > > > > > > Yet another finding on my side to "not call yet for vote
> > > > > > > on a
> > > > > > > Storm
> > > > > > > 2.6.0", we have some CVEs and we shoudn't release a
> > > > > > > version
> > > > > > > having
> > > > > > > known CVEs.
> > > > > > >
> > > > > > > For example, we have
> > > > > > >
> > > > > > > find . -name "commons-text*jar" -print
> > > > > > > ./lib-webapp/commons-text-1.9.jar
> > > > > > > ./external/storm-autocreds/commons-text-1.10.0.jar
> > > > > > >
> > > > > > > The second one is okay from
> > > > > > > https://www.cvedetails.com/cve/CVE-2022-42889/ perspectiv
> > > > > > > e, but
> > > > > > > not
> > > > > > > the first one.
> > > > > > >
> > > > > > > I will continue checking if there are other CVEs...
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Alexandre
> > > > > > >
> > > > > > > Le mar. 7 nov. 2023 à 17:43, Alexandre Vermeerbergen
> > > > > > > <av...@gmail.com> a écrit :
> > > > > > > >
> > > > > > > > Oh thank you very much for this cherry-pinking, I was
> > > > > > > > too
> > > > > > > > busy
> > > > > > > > doing
> > > > > > > > other unrelated stuff..
> > > > > > > >
> > > > > > > > Le mar. 7 nov. 2023 à 17:35, Richard Zowalla
> > > > > > > > <rz...@apache.org> a
> > > > > > > > écrit :
> > > > > > > > >
> > > > > > > > > I cherry-picked your commit and put that on master
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb
> > > > > > > > > Richard
> > > > > > > > > Zowalla:
> > > > > > > > > > You pushed on your own branch. You would need to
> > > > > > > > > > push to
> > > > > > > > > > apache/storm
> > > > > > > > > > main branch or open a PR ;-)
> > > > > > > > > >
> > > > > > > > > > Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb
> > > > > > > > > > Alexandre
> > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > Just did a 'git push' with that trivial fix... is
> > > > > > > > > > > that
> > > > > > > > > > > enough?
> > > > > > > > > > >
> > > > > > > > > > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla
> > > > > > > > > > > <ri...@zowalla.com>
> > > > > > > > > > > a
> > > > > > > > > > > écrit :
> > > > > > > > > > > >
> > > > > > > > > > > > You can do as you like ;-)
> > > > > > > > > > > >
> > > > > > > > > > > > Am 4. November 2023 08:54:48 MEZ schrieb
> > > > > > > > > > > > Alexandre
> > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > <av...@gmail.com>:
> > > > > > > > > > > > > I mean :
> > > > > > > > > > > > >
> > > > > > > > > > > > > Can I keep using my
> > > > > > > > > > > > > 'STORM-3958-Capacity-to-set-St
> > > > > > > > > > > > > orm-UI's-title-in-conf/storm.yaml'
> > > > > > > > > > > > > branch to push this fix, or should I create a
> > > > > > > > > > > > > new
> > > > > > > > > > > > > one?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Le sam. 4 nov. 2023 à 08:53, Alexandre
> > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Can you keep using my
> > > > > > > > > > > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-
> > > > > > > > > > > > > > title-in-
> > > > > > > > > > > > > > conf/storm.yaml'
> > > > > > > > > > > > > > branch to push this fix, or should I create
> > > > > > > > > > > > > > a new
> > > > > > > > > > > > > > one?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Le sam. 4 nov. 2023 à 08:14, Richard
> > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > <ri...@zowalla.com> a écrit :
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Just push a fix for it (referencing the
> > > > > > > > > > > > > > > original
> > > > > > > > > > > > > > > issue) ;-)
> > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > we hadn't a release in between, so not a
> > > > > > > > > > > > > > > big
> > > > > > > > > > > > > > > deal.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Am 4. November 2023 08:01:54 MEZ schrieb
> > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > Vermeerbergen <av...@gmail.com>:
> > > > > > > > > > > > > > > > Temporary workaround for the Storm UI
> > > > > > > > > > > > > > > > title's
> > > > > > > > > > > > > > > > regression:
> > > > > > > > > > > > > > > > run this
> > > > > > > > > > > > > > > > command on the title-template.html file
> > > > > > > > > > > > > > > > (in
> > > > > > > > > > > > > > > > storm-stable/public/templates
> > > > > > > > > > > > > > > > directory)
> > > > > > > > > > > > > > > > after
> > > > > > > > > > > > > > > > having
> > > > > > > > > > > > > > > > extracted the
> > > > > > > > > > > > > > > > binary archive:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > sed -i 's/{{ui\.title}}/{{title}}/'
> > > > > > > > > > > > > > > > title-
> > > > > > > > > > > > > > > > template.html
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Again sorry for having introduced this
> > > > > > > > > > > > > > > > regression.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre
> > > > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Updates regarding my test with Storm
> > > > > > > > > > > > > > > > > 2.6.0
> > > > > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > > > > 2023/11/02:
> > > > > > > > > > > > > > > > > 1. I have modified my KO bolts using
> > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > pattern in
> > > > > > > > > > > > > > > > > their prepare() method:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > long cacheMaxSize =
> > > > > > > > > > > > > > > > > Long.parseLong(stormConf.get(ConfKeys
> > > > > > > > > > > > > > > > > .SVC_D
> > > > > > > > > > > > > > > > > EF_C
> > > > > > > > > > > > > > > > > AC
> > > > > > > > > > > > > > > > > HE_SIZ
> > > > > > > > > > > > > > > > > E)
> > > > > > > > > > > > > > > > > .toString());
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > instead of:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > long cacheMaxSize = (Long)
> > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_
> > > > > > > > > > > > > > > > > SIZE);
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > 2. I have found the cause of the
> > > > > > > > > > > > > > > > > regression
> > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > loss of
> > > > > > > > > > > > > > > > > 'Storm UI'
> > > > > > > > > > > > > > > > > title at the top left-hand side of
> > > > > > > > > > > > > > > > > all
> > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > UI's
> > > > > > > > > > > > > > > > > pages.
> > > > > > > > > > > > > > > > > This is a mistake of mine in this
> > > > > > > > > > > > > > > > > PR:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > > > > > > > > > > > > > > More precisely, at line 19 of
> > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > file:
> > > > > > > > > > > > > > > > >
> > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > > > > > > > > > > > > > > > we must have:
> > > > > > > > > > > > > > > > > <h1><a href="/">{{title}}</a></h1>
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > instead of:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > <h1><a href="/">{{ui.title}}</a></h1>
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I think I missed this during my
> > > > > > > > > > > > > > > > > tests
> > > > > > > > > > > > > > > > > because
> > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > browser's cache
> > > > > > > > > > > > > > > > > (bad excuse, I know...): probably I
> > > > > > > > > > > > > > > > > tested
> > > > > > > > > > > > > > > > > by-
> > > > > > > > > > > > > > > > > patching
> > > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > > deployed
> > > > > > > > > > > > > > > > > Storm cluster, and I finally forgot
> > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > checkout
> > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > latest change of
> > > > > > > > > > > > > > > > > my delivery.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I had to use "private navigation"
> > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > make
> > > > > > > > > > > > > > > > > sure
> > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > time it fixes this issue
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > What would be the quickest way for
> > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > very
> > > > > > > > > > > > > > > > > simple fix
> > > > > > > > > > > > > > > > > to be delivered ?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 20:57,
> > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Thanks Richard for the analysis.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > My thought on this are very simple:
> > > > > > > > > > > > > > > > > > let
> > > > > > > > > > > > > > > > > > me
> > > > > > > > > > > > > > > > > > fix
> > > > > > > > > > > > > > > > > > my own
> > > > > > > > > > > > > > > > > > old code, after
> > > > > > > > > > > > > > > > > > all this was impacting only one of
> > > > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > > > topologies, the
> > > > > > > > > > > > > > > > > > other one were
> > > > > > > > > > > > > > > > > > using more careful Integer.parseInt
> > > > > > > > > > > > > > > > > > or
> > > > > > > > > > > > > > > > > > Integer.parseLong to avoid
> > > > > > > > > > > > > > > > > > making assumption on the way the
> > > > > > > > > > > > > > > > > > values
> > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > config
> > > > > > > > > > > > > > > > > > Map were
> > > > > > > > > > > > > > > > > > serialized.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > My other concern with 2.6.0
> > > > > > > > > > > > > > > > > > Snapshot is
> > > > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > > UI, the "Storm
> > > > > > > > > > > > > > > > > > UI" title at the top left hand side
> > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > each
> > > > > > > > > > > > > > > > > > page is
> > > > > > > > > > > > > > > > > > missing. This may
> > > > > > > > > > > > > > > > > > be the result of one of my own PR,
> > > > > > > > > > > > > > > > > > I'll
> > > > > > > > > > > > > > > > > > have
> > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > check
> > > > > > > > > > > > > > > > > > why...
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Alex
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 20:23,
> > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > <rz...@apache.org> a écrit :
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > thanks for the feedback.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > The main difference to 2.5.0 is,
> > > > > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > switched
> > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > JSON
> > > > > > > > > > > > > > > > > > > implementation in Utils, which
> > > > > > > > > > > > > > > > > > > relied
> > > > > > > > > > > > > > > > > > > on a
> > > > > > > > > > > > > > > > > > > super old JSON library. What is
> > > > > > > > > > > > > > > > > > > happening
> > > > > > > > > > > > > > > > > > > here is,
> > > > > > > > > > > > > > > > > > > that the new JSON
> > > > > > > > > > > > > > > > > > > library (net.minidev) behaves
> > > > > > > > > > > > > > > > > > > differently.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > We can see it by writing some
> > > > > > > > > > > > > > > > > > > simple
> > > > > > > > > > > > > > > > > > > code
> > > > > > > > > > > > > > > > > > > without
> > > > > > > > > > > > > > > > > > > Storm (The related
> > > > > > > > > > > > > > > > > > > code line with the parsing is
> > > > > > > > > > > > > > > > > > > taken
> > > > > > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > > > Utils)
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Map<String, Long> conf = new
> > > > > > > > > > > > > > > > > > > HashMap<>();
> > > > > > > > > > > > > > > > > > > conf.put("test", 100000000L);
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Map<String, Object>
> > > > > > > > > > > > > > > > > > > deserTopoConf =
> > > > > > > > > > > > > > > > > > > (Map<String,
> > > > > > > > > > > > > > > > > > > Object>)
> > > > > > > > > > > > > > > > > > > JSONValue.parseWithException(JSON
> > > > > > > > > > > > > > > > > > > Value.
> > > > > > > > > > > > > > > > > > > toJS
> > > > > > > > > > > > > > > > > > > ON
> > > > > > > > > > > > > > > > > > > String
> > > > > > > > > > > > > > > > > > > (c
> > > > > > > > > > > > > > > > > > > onf));
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > for(Map.Entry<String, Object>
> > > > > > > > > > > > > > > > > > > e :
> > > > > > > > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > System.out.println(e.getValue().g
> > > > > > > > > > > > > > > > > > > etClas
> > > > > > > > > > > > > > > > > > > s().
> > > > > > > > > > > > > > > > > > > ge
> > > > > > > > > > > > > > > > > > > tSimpl
> > > > > > > > > > > > > > > > > > > eN
> > > > > > > > > > > > > > > > > > > ame());
> > > > > > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > This will print "Integer". Using
> > > > > > > > > > > > > > > > > > > "10000000000000000L"
> > > > > > > > > > > > > > > > > > > as a value will
> > > > > > > > > > > > > > > > > > > result in "Long".
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > The main issue derives from the
> > > > > > > > > > > > > > > > > > > fact,
> > > > > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > serialize the config map
> > > > > > > > > > > > > > > > > > > into a JSON String and parse the
> > > > > > > > > > > > > > > > > > > result.
> > > > > > > > > > > > > > > > > > > Based on the String
> > > > > > > > > > > > > > > > > > > representation,
> > > > > > > > > > > > > > > > > > > "100000000L"
> > > > > > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > > > still fit into an
> > > > > > > > > > > > > > > > > > > Integer, so the parser will not
> > > > > > > > > > > > > > > > > > > use
> > > > > > > > > > > > > > > > > > > Long ;-
> > > > > > > > > > > > > > > > > > > )
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > If the number is big enough, the
> > > > > > > > > > > > > > > > > > > parser
> > > > > > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > > > switch
> > > > > > > > > > > > > > > > > > > to a Long> >> > >> > > The old
> > > > > > > > > > > > > > > > > > > library
> > > > > > > > > > > > > > > > > > > (>
> > > > > > > > > > > > > > > > > > > 10
> > > > > > > > > > > > > > > > > > > years
> > > > > > > > > > > > > > > > > > > old now):
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > deserTopoConf = (Map<String,
> > > > > > > > > > > > > > > > > > > Object>)
> > > > > > > > > > > > > > > > > > > org.json.simple.JSONValue.parseWi
> > > > > > > > > > > > > > > > > > > thExce
> > > > > > > > > > > > > > > > > > > ptio
> > > > > > > > > > > > > > > > > > > n(
> > > > > > > > > > > > > > > > > > > org.js
> > > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > .simple.JSONValue.
> > > > > > > > > > > > > > > > > > > toJSONString(conf));
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > for(Map.Entry<String, Object>
> > > > > > > > > > > > > > > > > > > e :
> > > > > > > > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > System.out.println(e.getValue().g
> > > > > > > > > > > > > > > > > > > etClas
> > > > > > > > > > > > > > > > > > > s().
> > > > > > > > > > > > > > > > > > > ge
> > > > > > > > > > > > > > > > > > > tSimpl
> > > > > > > > > > > > > > > > > > > eN
> > > > > > > > > > > > > > > > > > > ame());
> > > > > > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > This will print "Long".
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > The difference between these
> > > > > > > > > > > > > > > > > > > libraries
> > > > > > > > > > > > > > > > > > > is,
> > > > > > > > > > > > > > > > > > > that the
> > > > > > > > > > > > > > > > > > > super old
> > > > > > > > > > > > > > > > > > > org.json.simple just always
> > > > > > > > > > > > > > > > > > > returns
> > > > > > > > > > > > > > > > > > > Long as
> > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > type.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > If can eventually tune the new
> > > > > > > > > > > > > > > > > > > JSON
> > > > > > > > > > > > > > > > > > > parser
> > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > something like:
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > deserTopoConf = (Map<String,
> > > > > > > > > > > > > > > > > > > Object>)
> > > > > > > > > > > > > > > > > > > new
> > > > > > > > > > > > > > > > > > > JSONParser(JSONParser.BIG_DIGIT_U
> > > > > > > > > > > > > > > > > > > NRESTR
> > > > > > > > > > > > > > > > > > > ICTE
> > > > > > > > > > > > > > > > > > > D)
> > > > > > > > > > > > > > > > > > > .parse
> > > > > > > > > > > > > > > > > > > (J
> > > > > > > > > > > > > > > > > > > SONValue= toJSONStr
> > > > > > > > > > > > > > > > > > > ing(conf),
> > > > > > > > > > > > > > > > > > > JSONValue.defaultReader.DEFAULT);
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > This will result in "Long" but
> > > > > > > > > > > > > > > > > > > might
> > > > > > > > > > > > > > > > > > > have
> > > > > > > > > > > > > > > > > > > other
> > > > > > > > > > > > > > > > > > > side-
> > > > > > > > > > > > > > > > > > > effects.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > From my point of view, the newer
> > > > > > > > > > > > > > > > > > > JSON
> > > > > > > > > > > > > > > > > > > library
> > > > > > > > > > > > > > > > > > > works
> > > > > > > > > > > > > > > > > > > as expected
> > > > > > > > > > > > > > > > > > > (although it might break backward
> > > > > > > > > > > > > > > > > > > compatibility).
> > > > > > > > > > > > > > > > > > > Unchecked casts are always evil,
> > > > > > > > > > > > > > > > > > > imho.
> > > > > > > > > > > > > > > > > > > In
> > > > > > > > > > > > > > > > > > > your
> > > > > > > > > > > > > > > > > > > specific case, it might
> > > > > > > > > > > > > > > > > > > be more save to cast to "Number"
> > > > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > than
> > > > > > > > > > > > > > > > > > > invoke
> > > > > > > > > > > > > > > > > > > "longValue()" instead.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Nevertheless, we might being able
> > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > restore
> > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > old
> > > > > > > > > > > > > > > > > > > behaviour by
> > > > > > > > > > > > > > > > > > > tuning the JSONParser
> > > > > > > > > > > > > > > > > > > configuration.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Thoughts?
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Am Freitag, dem 03.11.2023 um
> > > > > > > > > > > > > > > > > > > 18:32
> > > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > > schrieb
> > > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > > > > > On a side note, the Long value
> > > > > > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > > > > "MyKOBolt" is
> > > > > > > > > > > > > > > > > > > > set by a
> > > > > > > > > > > > > > > > > > > > DefaultTopologyConfig to a
> > > > > > > > > > > > > > > > > > > > value with
> > > > > > > > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > > java.lang.Long instance...
> > > > > > > > > > > > > > > > > > > > hence it's surprising to see it
> > > > > > > > > > > > > > > > > > > > internally
> > > > > > > > > > > > > > > > > > > > converted into a
> > > > > > > > > > > > > > > > > > > > java.lang.Integer in the Map
> > > > > > > > > > > > > > > > > > > > passed
> > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > prepare()
> > > > > > > > > > > > > > > > > > > > method...
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 18:27,
> > > > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > > > > > > > <av...@gmail.com> a
> > > > > > > > > > > > > > > > > > > > écrit :
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Hello,
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > I found a breaking change
> > > > > > > > > > > > > > > > > > > > > using
> > > > > > > > > > > > > > > > > > > > > Apache
> > > > > > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > > > > > 2.6.0 snapshot build
> > > > > > > > > > > > > > > > > > > > > dated 20231103 and without
> > > > > > > > > > > > > > > > > > > > > changing
> > > > > > > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > > > > > > topologies' code (keeping
> > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > same big jars).
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > This seems to be tied to a
> > > > > > > > > > > > > > > > > > > > > datatype
> > > > > > > > > > > > > > > > > > > > > change in
> > > > > > > > > > > > > > > > > > > > > Map
> > > > > > > > > > > > > > > > > > > > > argument of
> > > > > > > > > > > > > > > > > > > > > prepare() method of Bolt.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > For example, this Bolt is KO
> > > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > > runtime:
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > public class MyKOBolt extends
> > > > > > > > > > > > > > > > > > > > > BaseRichBolt {
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > > > > > > > public void prepare(Map
> > > > > > > > > > > > > > > > > > > > > stormConf,
> > > > > > > > > > > > > > > > > > > > > TopologyContext context,
> > > > > > > > > > > > > > > > > > > > > OutputCollector collector) {
> > > > > > > > > > > > > > > > > > > > > cacheMaxSize = (Long)
> > > > > > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DE
> > > > > > > > > > > > > > > > > > > > > F_CACH
> > > > > > > > > > > > > > > > > > > > > E_SI
> > > > > > > > > > > > > > > > > > > > > ZE
> > > > > > > > > > > > > > > > > > > > > );
> > > > > > > > > > > > > > > > > > > > > cacheTtlMin = (Long)
> > > > > > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DE
> > > > > > > > > > > > > > > > > > > > > F_CACH
> > > > > > > > > > > > > > > > > > > > > E_TT
> > > > > > > > > > > > > > > > > > > > > L_
> > > > > > > > > > > > > > > > > > > > > MIN);
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > as it throws this error:
> > > > > > > > > > > > > > > > > > > > > 2023-11-03 16:42:20.230
> > > > > > > > > > > > > > > > > > > > > o.a.s.e.b.BoltExecutor
> > > > > > > > > > > > > > > > > > > > > Thread-14-__system-executor[-
> > > > > > > > > > > > > > > > > > > > > 1, -1]
> > > > > > > > > > > > > > > > > > > > > [INFO]
> > > > > > > > > > > > > > > > > > > > > Preparing bolt
> > > > > > > > > > > > > > > > > > > > > __system:[-1]
> > > > > > > > > > > > > > > > > > > > > 2023-11-03 16:42:20.193
> > > > > > > > > > > > > > > > > > > > > o.a.s.u.Utils
> > > > > > > > > > > > > > > > > > > > > Thread-15-evaluateTriggers-
> > > > > > > > > > > > > > > > > > > > > executor[11,
> > > > > > > > > > > > > > > > > > > > > 11]
> > > > > > > > > > > > > > > > > > > > > [ERROR] Async loop
> > > > > > > > > > > > > > > > > > > > > died!
> > > > > > > > > > > > > > > > > > > > > java.lang.ClassCastException:
> > > > > > > > > > > > > > > > > > > > > java.lang.Integer
> > > > > > > > > > > > > > > > > > > > > incompatible with
> > > > > > > > > > > > > > > > > > > > > java.lang.Long
> > > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > > com.acme.storm.alerting.MyKOB
> > > > > > > > > > > > > > > > > > > > > olt
> > > > > > > > > > > > > > > > > > > > > .prepare(EvaluationBolt.java:
> > > > > > > > > > > > > > > > > > > > > 71)
> > > > > > > > > > > > > > > > > > > > > ~[stormjar.jar:?]
> > > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > > org.apache.storm.executor.bol
> > > > > > > > > > > > > > > > > > > > > t.Bolt
> > > > > > > > > > > > > > > > > > > > > Exec
> > > > > > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > > > > > or.ini
> > > > > > > > > > > > > > > > > > > > > t(
> > > > > > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > > > > > 128)
> > > > > > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > > org.apache.storm.executor.bol
> > > > > > > > > > > > > > > > > > > > > t.Bolt
> > > > > > > > > > > > > > > > > > > > > Exec
> > > > > > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > > > > > or.cal
> > > > > > > > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > > > > > 138)
> > > > > > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > > org.apache.storm.executor.bol
> > > > > > > > > > > > > > > > > > > > > t.Bolt
> > > > > > > > > > > > > > > > > > > > > Exec
> > > > > > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > > > > > or.cal
> > > > > > > > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > > > > > 54)
> > > > > > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > > org.apache.storm.utils.Utils$
> > > > > > > > > > > > > > > > > > > > > 1.run(
> > > > > > > > > > > > > > > > > > > > > Util
> > > > > > > > > > > > > > > > > > > > > s.
> > > > > > > > > > > > > > > > > > > > > java:3
> > > > > > > > > > > > > > > > > > > > > 93
> > > > > > > > > > > > > > > > > > > > > )
> > > > > > > > > > > > > > > > > > > > > [storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > > java.lang.Thread.run(Thread.j
> > > > > > > > > > > > > > > > > > > > > ava:85
> > > > > > > > > > > > > > > > > > > > > 7)
> > > > > > > > > > > > > > > > > > > > > [?:?]
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > whereas this bolt is fine
> > > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > > > > > 2.6.0:
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > public class MyOKBolt extends
> > > > > > > > > > > > > > > > > > > > > BaseBasicBolt {
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > > > > > > > public void prepare(final
> > > > > > > > > > > > > > > > > > > > > Map
> > > > > > > > > > > > > > > > > > > > > stormConf,
> > > > > > > > > > > > > > > > > > > > > final TopologyContext
> > > > > > > > > > > > > > > > > > > > > context) {
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > super.prepare(stormConf,
> > > > > > > > > > > > > > > > > > > > > context);
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > this.redisPort
> > > > > > > > > > > > > > > > > > > > > =
> > > > > > > > > > > > > > > > > > > > > Integer.parseInt((String)
> > > > > > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.REDIS_
> > > > > > > > > > > > > > > > > > > > > PORT))
> > > > > > > > > > > > > > > > > > > > > ;
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > That said, I can modify the
> > > > > > > > > > > > > > > > > > > > > code of
> > > > > > > > > > > > > > > > > > > > > "MyKOBolt"
> > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > use same
> > > > > > > > > > > > > > > > > > > > > Integer.parseInt or
> > > > > > > > > > > > > > > > > > > > > Integer.parseLong
> > > > > > > > > > > > > > > > > > > > > trick,
> > > > > > > > > > > > > > > > > > > > > but
> > > > > > > > > > > > > > > > > > > > > it's the first
> > > > > > > > > > > > > > > > > > > > > time
> > > > > > > > > > > > > > > > > > > > > in my long history of
> > > > > > > > > > > > > > > > > > > > > upgrades that
> > > > > > > > > > > > > > > > > > > > > I
> > > > > > > > > > > > > > > > > > > > > have seen
> > > > > > > > > > > > > > > > > > > > > such runtime
> > > > > > > > > > > > > > > > > > > > > incompatibility.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Maybe "MyKOBolt" was badly
> > > > > > > > > > > > > > > > > > > > > written
> > > > > > > > > > > > > > > > > > > > > since
> > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > beginning and I have
> > > > > > > > > > > > > > > > > > > > > just
> > > > > > > > > > > > > > > > > > > > > hit the punishment for it: is
> > > > > > > > > > > > > > > > > > > > > there
> > > > > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > > > documentation which clarifies
> > > > > > > > > > > > > > > > > > > > > datatypes of the Map argument
> > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > prepare()
> > > > > > > > > > > > > > > > > > > > > method?
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Also worth noting: "MyKOBolt"
> > > > > > > > > > > > > > > > > > > > > derives
> > > > > > > > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > > > > > > BaseRichBolt , while
> > > > > > > > > > > > > > > > > > > > > "MyOKBolt" derives from
> > > > > > > > > > > > > > > > > > > > > BaseBasicBolt
> > > > > > > > > > > > > > > > > > > > > =>
> > > > > > > > > > > > > > > > > > > > > could
> > > > > > > > > > > > > > > > > > > > > this have any impact
> > > > > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > > > this finding ?
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Last, I'm running this on
> > > > > > > > > > > > > > > > > > > > > Redhat
> > > > > > > > > > > > > > > > > > > > > Linux
> > > > > > > > > > > > > > > > > > > > > 8
> > > > > > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > > > IBM
> > > > > > > > > > > > > > > > > > > > > Semeru JDK
> > > > > > > > > > > > > > > > > > > > > 17.0.8.1.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 15:16,
> > > > > > > > > > > > > > > > > > > > > Julien
> > > > > > > > > > > > > > > > > > > > > Nioche
> > > > > > > > > > > > > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > > > lists.digitalpebble@gmail.com>
> > > > > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > > > écrit
> > > > > > > > > > > > > > > > > > > > > :
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > Thanks Richard.
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > Tried the latest snapshot
> > > > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > > > StormCrawler
> > > > > > > > > > > > > > > > > > > > > > both in local and
> > > > > > > > > > > > > > > > > > > > > > deployed mode
> > > > > > > > > > > > > > > > > > > > > > and did not find any
> > > > > > > > > > > > > > > > > > > > > > issues.
> > > > > > > > > > > > > > > > > > > > > > Will try it on a topology
> > > > > > > > > > > > > > > > > > > > > > generating
> > > > > > > > > > > > > > > > > > > > > > WARC
> > > > > > > > > > > > > > > > > > > > > > files
> > > > > > > > > > > > > > > > > > > > > > next week to
> > > > > > > > > > > > > > > > > > > > > > check that the
> > > > > > > > > > > > > > > > > > > > > > dependency changes on
> > > > > > > > > > > > > > > > > > > > > > Hadoop have
> > > > > > > > > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > > > > > > > broken
> > > > > > > > > > > > > > > > > > > > > > anything.
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > Have a good week end
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > Julien
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > On Thu, 2 Nov 2023 at
> > > > > > > > > > > > > > > > > > > > > > 19:25,
> > > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > > > > > <rz...@apache.org>
> > > > > > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > Short update. Sorted out
> > > > > > > > > > > > > > > > > > > > > > > an
> > > > > > > > > > > > > > > > > > > > > > > issue
> > > > > > > > > > > > > > > > > > > > > > > with the
> > > > > > > > > > > > > > > > > > > > > > > tar.gz/zip with
> > > > > > > > > > > > > > > > > > > > > > > Julien today
> > > > > > > > > > > > > > > > > > > > > > > and re-uploaded them to
> > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > nightlies
> > > > > > > > > > > > > > > > > > > > > > > area.
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > This new bundle works as
> > > > > > > > > > > > > > > > > > > > > > > expected
> > > > > > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > > > > > > > > deployment but happy to
> > > > > > > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > > > > > > additional feedback
> > > > > > > > > > > > > > > > > > > > > > > before
> > > > > > > > > > > > > > > > > > > > > > > getting
> > > > > > > > > > > > > > > > > > > > > > > up
> > > > > > > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > > > > > first
> > > > > > > > > > > > > > > > > > > > > > > release candidate
> > > > > > > > > > > > > > > > > > > > > > > ;-)
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023
> > > > > > > > > > > > > > > > > > > > > > > um
> > > > > > > > > > > > > > > > > > > > > > > 08:21
> > > > > > > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > > > > > > schrieb Richard
> > > > > > > > > > > > > > > > > > > > > > > Zowalla:
> > > > > > > > > > > > > > > > > > > > > > > > Hi Alexandre,
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > we are not in a hurry
> > > > > > > > > > > > > > > > > > > > > > > > here :)
> > > > > > > > > > > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > > > > > > > > > > take as
> > > > > > > > > > > > > > > > > > > > > > > > much as time you
> > > > > > > > > > > > > > > > > > > > > > > > need.
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > Am Montag, dem
> > > > > > > > > > > > > > > > > > > > > > > > 30.10.2023 um
> > > > > > > > > > > > > > > > > > > > > > > > 07:50
> > > > > > > > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > > > > > > > schrieb Alexandre
> > > > > > > > > > > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > > > > > > > > > > Hello Richard,
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > Okay, I'm more than
> > > > > > > > > > > > > > > > > > > > > > > > > happy
> > > > > > > > > > > > > > > > > > > > > > > > > to do
> > > > > > > > > > > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > > > > > > my pre-production
> > > > > > > > > > > > > > > > > > > > > > > > > cluster
> > > > > > > > > > > > > > > > > > > > > > > > > (~10 topologies)
> > > > > > > > > > > > > > > > > > > > > > > > > using the
> > > > > > > > > > > > > > > > > > > > > > > > > binary
> > > > > > > > > > > > > > > > > > > > > > > > > artifacts.
> > > > > > > > > > > > > > > > > > > > > > > > > Would it be OK if I
> > > > > > > > > > > > > > > > > > > > > > > > > can use
> > > > > > > > > > > > > > > > > > > > > > > > > up
> > > > > > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > > > > end
> > > > > > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > > > > this week so that
> > > > > > > > > > > > > > > > > > > > > > > > > I'll be
> > > > > > > > > > > > > > > > > > > > > > > > > able to have enough
> > > > > > > > > > > > > > > > > > > > > > > > > time to
> > > > > > > > > > > > > > > > > > > > > > > > > check
> > > > > > > > > > > > > > > > > > > > > > > > > all
> > > > > > > > > > > > > > > > > > > > > > > > > potential issues that
> > > > > > > > > > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > > > > > > > > > upgrade bring?
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > Le lun. 30 oct. 2023
> > > > > > > > > > > > > > > > > > > > > > > > > à
> > > > > > > > > > > > > > > > > > > > > > > > > 07:35,
> > > > > > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > > > > > > > > <rz...@apache.org> a
> > > > > > > > > > > > > > > > > > > > > > > > > écrit :
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > before starting a
> > > > > > > > > > > > > > > > > > > > > > > > > > release
> > > > > > > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > > > > > 2.6.0-
> > > > > > > > > > > > > > > > > > > > > > > > > > SNAPSHOT, I would
> > > > > > > > > > > > > > > > > > > > > > > > > > like
> > > > > > > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > > > > > > > > > some feedback on
> > > > > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > > > > current
> > > > > > > > > > > > > > > > > > > > > > > > > > SNAPSHOT
> > > > > > > > > > > > > > > > > > > > > > > > > > build.
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > I just uploaded a
> > > > > > > > > > > > > > > > > > > > > > > > > > 2.6.0-
> > > > > > > > > > > > > > > > > > > > > > > > > > SNAPSHOT of
> > > > > > > > > > > > > > > > > > > > > > > > > > Storm build from
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > It is available via
> > > > > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > > > > ASF
> > > > > > > > > > > > > > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > > > > > > > > > > > > > repository:
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > You can consume by
> > > > > > > > > > > > > > > > > > > > > > > > > > adding
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > <repository>
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > <id>apache.snapshot
> > > > > > > > > > > > > > > > > > > > > > > > > > s</id>
> > > > > > > > > > > > > > > > > > > > > > > > > > <name>Apache
> > > > > > > > > > > > > > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > > > > > > > > > > > > > Repository</name>
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > <url>
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > https://repository.apache.org/snap
> > > > > > > > > > > > > > > > > > > > > > > > > > shots</url>
> > > > > > > > > > > > > > > > > > > > > > > > > > <releases>
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > <enabled>false</ena
> > > > > > > > > > > > > > > > > > > > > > > > > > bled>
> > > > > > > > > > > > > > > > > > > > > > > > > > </releases>
> > > > > > > > > > > > > > > > > > > > > > > > > > </repository>
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > to your project
> > > > > > > > > > > > > > > > > > > > > > > > > > pom. As
> > > > > > > > > > > > > > > > > > > > > > > > > > we do
> > > > > > > > > > > > > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > > > > > > > > > > > deploy SNAPSHOTS
> > > > > > > > > > > > > > > > > > > > > > > > > > automatically,
> > > > > > > > > > > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > > > > > > > > > should be easy to
> > > > > > > > > > > > > > > > > > > > > > > > > > just
> > > > > > > > > > > > > > > > > > > > > > > > > > consume
> > > > > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > > > > latest SNAPSHOT.
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > The packaged
> > > > > > > > > > > > > > > > > > > > > > > > > > binaries are
> > > > > > > > > > > > > > > > > > > > > > > > > > available
> > > > > > > > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > > > > > > > nightlies.apache.or
> > > > > > > > > > > > > > > > > > > > > > > > > > g:
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > If you have some
> > > > > > > > > > > > > > > > > > > > > > > > > > minutes
> > > > > > > > > > > > > > > > > > > > > > > > > > left:
> > > > > > > > > > > > > > > > > > > > > > > > > > Please
> > > > > > > > > > > > > > > > > > > > > > > > > > test and report any
> > > > > > > > > > > > > > > > > > > > > > > > > > issues
> > > > > > > > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > > > > > > > this binaries, so
> > > > > > > > > > > > > > > > > > > > > > > > > > we can
> > > > > > > > > > > > > > > > > > > > > > > > > > fix
> > > > > > > > > > > > > > > > > > > > > > > > > > before
> > > > > > > > > > > > > > > > > > > > > > > > > > attempting to
> > > > > > > > > > > > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > The most
> > > > > > > > > > > > > > > > > > > > > > > > > > significant
> > > > > > > > > > > > > > > > > > > > > > > > > > changes
> > > > > > > > > > > > > > > > > > > > > > > > > > are in
> > > > > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > > > > are of
> > > > > > > > > > > > > > > > > > > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > > > > > > > > upgraded from 2.x
> > > > > > > > > > > > > > > > > > > > > > > > > > to 3.x
> > > > > > > > > > > > > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > > > > > > > > > > > > our
> > > > > > > > > > > > > > > > > > > > > > > > > > own
> > > > > > > > > > > > > > > > > > > > > > > > > > test
> > > > > > > > > > > > > > > > > > > > > > > > > > coverage within
> > > > > > > > > > > > > > > > > > > > > > > > > > the build
> > > > > > > > > > > > > > > > > > > > > > > > > > looks
> > > > > > > > > > > > > > > > > > > > > > > > > > good but would be
> > > > > > > > > > > > > > > > > > > > > > > > > > nice
> > > > > > > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > > > > > get
> > > > > > > > > > > > > > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > > > > > > > > > > real world use-case
> > > > > > > > > > > > > > > > > > > > > > > > > > feedback
> > > > > > > > > > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > > > > > > > > this. In addition,
> > > > > > > > > > > > > > > > > > > > > > > > > > we had
> > > > > > > > > > > > > > > > > > > > > > > > > > quite
> > > > > > > > > > > > > > > > > > > > > > > > > > a lof
> > > > > > > > > > > > > > > > > > > > > > > > > > of 3rd party
> > > > > > > > > > > > > > > > > > > > > > > > > > dependency
> > > > > > > > > > > > > > > > > > > > > > > > > > upgrades.
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > In addition, it
> > > > > > > > > > > > > > > > > > > > > > > > > > contains
> > > > > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > > > > pruning
> > > > > > > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > > > > > external modules
> > > > > > > > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > > > > > > > listed
> > > > > > > > > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > A summaryin Jira is
> > > > > > > > > > > > > > > > > > > > > > > > > > here:
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > *Open Source Solutions for
> > > > > > > > > > > > > > > > > > > > > > Text
> > > > > > > > > > > > > > > > > > > > > > Engineering*
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > http://www.digitalpebble.com
> > > > > > > > > > > > > > > > > > > > > > http://digitalpebble.blogspot.com/
> > > > > > > > > > > > > > > > > > > > > > #digitalpebble
> > > > > > > > > > > > > > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > > > > http://twitter.com/digitalpebble
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> >
> > --
> >
> > *Open Source Solutions for Text Engineering*
> >
> > http://www.digitalpebble.com
> > http://digitalpebble.blogspot.com/
> > #digitalpebble <http://twitter.com/digitalpebble>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Yet another regression on in-work 2.6.0 : the supervisor.html view is
broken, there's a Javascript syntax error (probably my fault again).
It' s bit late for me, I'll check that tomorrow...
Alexandre
Le lun. 13 nov. 2023 à 15:36, Julien Nioche
<li...@gmail.com> a écrit :
>
> I agree with Richard. The next release contains enough big changes and it
> would be good to release it ASAP before embarking on another big
> modification.
>
> Julien
>
> On Mon, 13 Nov 2023 at 12:08, Richard Zowalla <rz...@apache.org> wrote:
>
> > Hi,
> >
> > yes - in principal I agree, that switching to slf4j 2.x would be great
> > in the near future. However, it requires some more work due to our old
> > dependency stack ("external modules", ...) and most likely a lot of
> > exclusions.
> >
> > Therefore, personally, I wouldn't see it in a 2.6.0 release but as a
> > task on which we could work after having a new version in the pipeline
> >
> > wdyt?
> >
> > Gruß
> > Richard
> >
> > Am Sonntag, dem 12.11.2023 um 09:40 +0100 schrieb Alexandre
> > Vermeerbergen:
> > > Thank Richard for the great work to cleanup CVEs on in-work 2.6.0 !
> > > One last "regret" I have with current in-work 2.6.0 is that we still
> > > rely on very old slf4j 1.x.
> > > While it was not on slf4j itself, the log4shell CVE should remind us
> > > that we should keep loggers as up to date as possible.
> > >
> > > I see in a deployed Storm runtime the following hits:
> > >
> > > [root]# find . -name "slf*jar" -print
> > > ./lib/slf4j-api-1.7.36.jar
> > > ./lib-tools/storm-kafka-monitor/slf4j-api-1.7.36.jar
> > > ./lib-tools/submit-tools/slf4j-api-1.7.36.jar
> > > ./lib-tools/sql/core/slf4j-api-1.7.36.jar
> > > ./lib-tools/sql/runtime/slf4j-api-1.7.36.jar
> > > ./external/storm-autocreds/slf4j-api-1.7.36.jar
> > > ./lib-worker/slf4j-api-1.7.36.jar
> > > ./lib-webapp/slf4j-api-1.7.36.jar
> > >
> > > From https://www.slf4j.org/download.html it is said that "The older
> > > stable SLF4J version is 1.7.36. It is no longer actively developed."
> > >
> > > The most current version of slf4j is 2.0.9.
> > >
> > > But I think at same time we would need to update these ones too,
> > > otherwise we will have a mismatch between SLF4J v1 logger
> > > implementations vs new v2 APIs:
> > >
> > > [root@ip-172-31-43-21 storm-stable]# find . -name "log*jar" -print
> > > ./lib/log4j-over-slf4j-1.7.36.jar
> > > ./lib/log4j-api-2.20.0.jar
> > > ./lib/log4j-slf4j-impl-2.20.0.jar
> > > ./lib/log4j-core-2.20.0.jar
> > > ./external/storm-autocreds/log4j-api-2.20.0.jar
> > > ./external/storm-autocreds/log4j-web-2.20.0.jar
> > > ./external/storm-autocreds/log4j-1.2-api-2.20.0.jar
> > > ./external/storm-autocreds/log4j-core-2.20.0.jar
> > > ./lib-worker/log4j-over-slf4j-1.7.36.jar
> > > ./lib-worker/log4j-api-2.20.0.jar
> > > ./lib-worker/log4j-slf4j-impl-2.20.0.jar
> > > ./lib-worker/log4j-core-2.20.0.jar
> > > ./lib-webapp/log4j-over-slf4j-1.7.36.jar
> > > ./lib-webapp/log4j-api-2.20.0.jar
> > > ./lib-webapp/log4j-core-2.20.0.jar
> > >
> > > Disclaimer: I have no idea how hard this update would be.
> > >
> > > Anyone with good experience at updating SLF4J & related loggers could
> > > react on this proposal ?
> > >
> > > Thanks,
> > > Alexandre
> > >
> > >
> > > Le jeu. 9 nov. 2023 à 15:04, Richard Zowalla <rz...@apache.org> a
> > > écrit :
> > > >
> > > > Ok, most of the stuff reported by grype should be fixed now.
> > > > I might be able to push a new snapshot within the next week, so we
> > > > can
> > > > do some testing and than go for a release candiate / vote :)
> > > >
> > > > Gruß
> > > > Richard
> > > >
> > > >
> > > >
> > > > Am Donnerstag, dem 09.11.2023 um 08:53 +0100 schrieb Richard
> > > > Zowalla:
> > > > > Maybe just run grype [1] on the lib directories and post the
> > > > > output
> > > > > on
> > > > > the list / thread, so we can create tickets for it
> > > > >
> > > > > There might be some false positives but will give good insights.
> > > > >
> > > > > Gruß
> > > > > Richard
> > > > >
> > > > >
> > > > > [1] https://github.com/anchore/grype
> > > > > Am Donnerstag, dem 09.11.2023 um 08:49 +0100 schrieb Alexandre
> > > > > Vermeerbergen:
> > > > > > Yet another finding on my side to "not call yet for vote on a
> > > > > > Storm
> > > > > > 2.6.0", we have some CVEs and we shoudn't release a version
> > > > > > having
> > > > > > known CVEs.
> > > > > >
> > > > > > For example, we have
> > > > > >
> > > > > > find . -name "commons-text*jar" -print
> > > > > > ./lib-webapp/commons-text-1.9.jar
> > > > > > ./external/storm-autocreds/commons-text-1.10.0.jar
> > > > > >
> > > > > > The second one is okay from
> > > > > > https://www.cvedetails.com/cve/CVE-2022-42889/ perspective, but
> > > > > > not
> > > > > > the first one.
> > > > > >
> > > > > > I will continue checking if there are other CVEs...
> > > > > >
> > > > > > Thanks,
> > > > > > Alexandre
> > > > > >
> > > > > > Le mar. 7 nov. 2023 à 17:43, Alexandre Vermeerbergen
> > > > > > <av...@gmail.com> a écrit :
> > > > > > >
> > > > > > > Oh thank you very much for this cherry-pinking, I was too
> > > > > > > busy
> > > > > > > doing
> > > > > > > other unrelated stuff..
> > > > > > >
> > > > > > > Le mar. 7 nov. 2023 à 17:35, Richard Zowalla
> > > > > > > <rz...@apache.org> a
> > > > > > > écrit :
> > > > > > > >
> > > > > > > > I cherry-picked your commit and put that on master
> > > > > > > >
> > > > > > > >
> > > > > > > > Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard
> > > > > > > > Zowalla:
> > > > > > > > > You pushed on your own branch. You would need to push to
> > > > > > > > > apache/storm
> > > > > > > > > main branch or open a PR ;-)
> > > > > > > > >
> > > > > > > > > Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb
> > > > > > > > > Alexandre
> > > > > > > > > Vermeerbergen:
> > > > > > > > > > Just did a 'git push' with that trivial fix... is that
> > > > > > > > > > enough?
> > > > > > > > > >
> > > > > > > > > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla
> > > > > > > > > > <ri...@zowalla.com>
> > > > > > > > > > a
> > > > > > > > > > écrit :
> > > > > > > > > > >
> > > > > > > > > > > You can do as you like ;-)
> > > > > > > > > > >
> > > > > > > > > > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre
> > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > <av...@gmail.com>:
> > > > > > > > > > > > I mean :
> > > > > > > > > > > >
> > > > > > > > > > > > Can I keep using my
> > > > > > > > > > > > 'STORM-3958-Capacity-to-set-St
> > > > > > > > > > > > orm-UI's-title-in-conf/storm.yaml'
> > > > > > > > > > > > branch to push this fix, or should I create a new
> > > > > > > > > > > > one?
> > > > > > > > > > > >
> > > > > > > > > > > > Le sam. 4 nov. 2023 à 08:53, Alexandre
> > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > >
> > > > > > > > > > > > > Can you keep using my
> > > > > > > > > > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > > > > > > > > > > conf/storm.yaml'
> > > > > > > > > > > > > branch to push this fix, or should I create a new
> > > > > > > > > > > > > one?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > > > > > > > > > > <ri...@zowalla.com> a écrit :
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Just push a fix for it (referencing the
> > > > > > > > > > > > > > original
> > > > > > > > > > > > > > issue) ;-)
> > > > > > > > > > > > > > -
> > > > > > > > > > > > > > we hadn't a release in between, so not a big
> > > > > > > > > > > > > > deal.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Am 4. November 2023 08:01:54 MEZ schrieb
> > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > Vermeerbergen <av...@gmail.com>:
> > > > > > > > > > > > > > > Temporary workaround for the Storm UI title's
> > > > > > > > > > > > > > > regression:
> > > > > > > > > > > > > > > run this
> > > > > > > > > > > > > > > command on the title-template.html file (in
> > > > > > > > > > > > > > > storm-stable/public/templates directory)
> > > > > > > > > > > > > > > after
> > > > > > > > > > > > > > > having
> > > > > > > > > > > > > > > extracted the
> > > > > > > > > > > > > > > binary archive:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-
> > > > > > > > > > > > > > > template.html
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Again sorry for having introduced this
> > > > > > > > > > > > > > > regression.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre
> > > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Updates regarding my test with Storm 2.6.0
> > > > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > > > 2023/11/02:
> > > > > > > > > > > > > > > > 1. I have modified my KO bolts using this
> > > > > > > > > > > > > > > > pattern in
> > > > > > > > > > > > > > > > their prepare() method:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > long cacheMaxSize =
> > > > > > > > > > > > > > > > Long.parseLong(stormConf.get(ConfKeys.SVC_D
> > > > > > > > > > > > > > > > EF_C
> > > > > > > > > > > > > > > > AC
> > > > > > > > > > > > > > > > HE_SIZ
> > > > > > > > > > > > > > > > E)
> > > > > > > > > > > > > > > > .toString());
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > instead of:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > long cacheMaxSize = (Long)
> > > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > 2. I have found the cause of the regression
> > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > loss of
> > > > > > > > > > > > > > > > 'Storm UI'
> > > > > > > > > > > > > > > > title at the top left-hand side of all
> > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > UI's
> > > > > > > > > > > > > > > > pages.
> > > > > > > > > > > > > > > > This is a mistake of mine in this PR:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > > > > > > > > > > > > > More precisely, at line 19 of this
> > > > > > > > > > > > > > > > file:
> > > > > > > > > > > > > > > >
> > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > > > > > > > > > > > > > > we must have:
> > > > > > > > > > > > > > > > <h1><a href="/">{{title}}</a></h1>
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > instead of:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > <h1><a href="/">{{ui.title}}</a></h1>
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I think I missed this during my tests
> > > > > > > > > > > > > > > > because
> > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > browser's cache
> > > > > > > > > > > > > > > > (bad excuse, I know...): probably I tested
> > > > > > > > > > > > > > > > by-
> > > > > > > > > > > > > > > > patching
> > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > deployed
> > > > > > > > > > > > > > > > Storm cluster, and I finally forgot to
> > > > > > > > > > > > > > > > checkout
> > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > latest change of
> > > > > > > > > > > > > > > > my delivery.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I had to use "private navigation" to
> > > > > > > > > > > > > > > > make
> > > > > > > > > > > > > > > > sure
> > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > time it fixes this issue
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > What would be the quickest way for this
> > > > > > > > > > > > > > > > very
> > > > > > > > > > > > > > > > simple fix
> > > > > > > > > > > > > > > > to be delivered ?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 20:57, Alexandre
> > > > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Thanks Richard for the analysis.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > My thought on this are very simple: let
> > > > > > > > > > > > > > > > > me
> > > > > > > > > > > > > > > > > fix
> > > > > > > > > > > > > > > > > my own
> > > > > > > > > > > > > > > > > old code, after
> > > > > > > > > > > > > > > > > all this was impacting only one of my
> > > > > > > > > > > > > > > > > topologies, the
> > > > > > > > > > > > > > > > > other one were
> > > > > > > > > > > > > > > > > using more careful Integer.parseInt or
> > > > > > > > > > > > > > > > > Integer.parseLong to avoid
> > > > > > > > > > > > > > > > > making assumption on the way the values
> > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > config
> > > > > > > > > > > > > > > > > Map were
> > > > > > > > > > > > > > > > > serialized.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > My other concern with 2.6.0 Snapshot is
> > > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > UI, the "Storm
> > > > > > > > > > > > > > > > > UI" title at the top left hand side of
> > > > > > > > > > > > > > > > > each
> > > > > > > > > > > > > > > > > page is
> > > > > > > > > > > > > > > > > missing. This may
> > > > > > > > > > > > > > > > > be the result of one of my own PR, I'll
> > > > > > > > > > > > > > > > > have
> > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > check
> > > > > > > > > > > > > > > > > why...
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Alex
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 20:23, Richard
> > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > <rz...@apache.org> a écrit :
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > thanks for the feedback.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > The main difference to 2.5.0 is, that
> > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > switched
> > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > JSON
> > > > > > > > > > > > > > > > > > implementation in Utils, which relied
> > > > > > > > > > > > > > > > > > on a
> > > > > > > > > > > > > > > > > > super old JSON library. What is
> > > > > > > > > > > > > > > > > > happening
> > > > > > > > > > > > > > > > > > here is,
> > > > > > > > > > > > > > > > > > that the new JSON
> > > > > > > > > > > > > > > > > > library (net.minidev) behaves
> > > > > > > > > > > > > > > > > > differently.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > We can see it by writing some simple
> > > > > > > > > > > > > > > > > > code
> > > > > > > > > > > > > > > > > > without
> > > > > > > > > > > > > > > > > > Storm (The related
> > > > > > > > > > > > > > > > > > code line with the parsing is taken
> > > > > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > > Utils)
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Map<String, Long> conf = new
> > > > > > > > > > > > > > > > > > HashMap<>();
> > > > > > > > > > > > > > > > > > conf.put("test", 100000000L);
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Map<String, Object> deserTopoConf =
> > > > > > > > > > > > > > > > > > (Map<String,
> > > > > > > > > > > > > > > > > > Object>)
> > > > > > > > > > > > > > > > > > JSONValue.parseWithException(JSONValue.
> > > > > > > > > > > > > > > > > > toJS
> > > > > > > > > > > > > > > > > > ON
> > > > > > > > > > > > > > > > > > String
> > > > > > > > > > > > > > > > > > (c
> > > > > > > > > > > > > > > > > > onf));
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > System.out.println(e.getValue().getClas
> > > > > > > > > > > > > > > > > > s().
> > > > > > > > > > > > > > > > > > ge
> > > > > > > > > > > > > > > > > > tSimpl
> > > > > > > > > > > > > > > > > > eN
> > > > > > > > > > > > > > > > > > ame());
> > > > > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > This will print "Integer". Using
> > > > > > > > > > > > > > > > > > "10000000000000000L"
> > > > > > > > > > > > > > > > > > as a value will
> > > > > > > > > > > > > > > > > > result in "Long".
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > The main issue derives from the fact,
> > > > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > serialize the config map
> > > > > > > > > > > > > > > > > > into a JSON String and parse the
> > > > > > > > > > > > > > > > > > result.
> > > > > > > > > > > > > > > > > > Based on the String representation,
> > > > > > > > > > > > > > > > > > "100000000L"
> > > > > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > > still fit into an
> > > > > > > > > > > > > > > > > > Integer, so the parser will not use
> > > > > > > > > > > > > > > > > > Long ;-
> > > > > > > > > > > > > > > > > > )
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > If the number is big enough, the parser
> > > > > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > > switch
> > > > > > > > > > > > > > > > > > to a Long> >> > >> > > The old library
> > > > > > > > > > > > > > > > > > (>
> > > > > > > > > > > > > > > > > > 10
> > > > > > > > > > > > > > > > > > years
> > > > > > > > > > > > > > > > > > old now):
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > deserTopoConf = (Map<String,
> > > > > > > > > > > > > > > > > > Object>)
> > > > > > > > > > > > > > > > > > org.json.simple.JSONValue.parseWithExce
> > > > > > > > > > > > > > > > > > ptio
> > > > > > > > > > > > > > > > > > n(
> > > > > > > > > > > > > > > > > > org.js
> > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > .simple.JSONValue.
> > > > > > > > > > > > > > > > > > toJSONString(conf));
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > System.out.println(e.getValue().getClas
> > > > > > > > > > > > > > > > > > s().
> > > > > > > > > > > > > > > > > > ge
> > > > > > > > > > > > > > > > > > tSimpl
> > > > > > > > > > > > > > > > > > eN
> > > > > > > > > > > > > > > > > > ame());
> > > > > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > This will print "Long".
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > The difference between these libraries
> > > > > > > > > > > > > > > > > > is,
> > > > > > > > > > > > > > > > > > that the
> > > > > > > > > > > > > > > > > > super old
> > > > > > > > > > > > > > > > > > org.json.simple just always returns
> > > > > > > > > > > > > > > > > > Long as
> > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > type.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > If can eventually tune the new JSON
> > > > > > > > > > > > > > > > > > parser
> > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > something like:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > deserTopoConf = (Map<String, Object>)
> > > > > > > > > > > > > > > > > > new
> > > > > > > > > > > > > > > > > > JSONParser(JSONParser.BIG_DIGIT_UNRESTR
> > > > > > > > > > > > > > > > > > ICTE
> > > > > > > > > > > > > > > > > > D)
> > > > > > > > > > > > > > > > > > .parse
> > > > > > > > > > > > > > > > > > (J
> > > > > > > > > > > > > > > > > > SONValue= toJSONStr
> > > > > > > > > > > > > > > > > > ing(conf),
> > > > > > > > > > > > > > > > > > JSONValue.defaultReader.DEFAULT);
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > This will result in "Long" but might
> > > > > > > > > > > > > > > > > > have
> > > > > > > > > > > > > > > > > > other
> > > > > > > > > > > > > > > > > > side-
> > > > > > > > > > > > > > > > > > effects.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > From my point of view, the newer JSON
> > > > > > > > > > > > > > > > > > library
> > > > > > > > > > > > > > > > > > works
> > > > > > > > > > > > > > > > > > as expected
> > > > > > > > > > > > > > > > > > (although it might break backward
> > > > > > > > > > > > > > > > > > compatibility).
> > > > > > > > > > > > > > > > > > Unchecked casts are always evil, imho.
> > > > > > > > > > > > > > > > > > In
> > > > > > > > > > > > > > > > > > your
> > > > > > > > > > > > > > > > > > specific case, it might
> > > > > > > > > > > > > > > > > > be more save to cast to "Number" and
> > > > > > > > > > > > > > > > > > than
> > > > > > > > > > > > > > > > > > invoke
> > > > > > > > > > > > > > > > > > "longValue()" instead.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Nevertheless, we might being able to
> > > > > > > > > > > > > > > > > > restore
> > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > old
> > > > > > > > > > > > > > > > > > behaviour by
> > > > > > > > > > > > > > > > > > tuning the JSONParser configuration.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Thoughts?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Am Freitag, dem 03.11.2023 um 18:32
> > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > schrieb
> > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > > > > On a side note, the Long value that
> > > > > > > > > > > > > > > > > > > "MyKOBolt" is
> > > > > > > > > > > > > > > > > > > set by a
> > > > > > > > > > > > > > > > > > > DefaultTopologyConfig to a value with
> > > > > > > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > java.lang.Long instance...
> > > > > > > > > > > > > > > > > > > hence it's surprising to see it
> > > > > > > > > > > > > > > > > > > internally
> > > > > > > > > > > > > > > > > > > converted into a
> > > > > > > > > > > > > > > > > > > java.lang.Integer in the Map passed
> > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > prepare()
> > > > > > > > > > > > > > > > > > > method...
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 18:27,
> > > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Hello,
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > I found a breaking change using
> > > > > > > > > > > > > > > > > > > > Apache
> > > > > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > > > > 2.6.0 snapshot build
> > > > > > > > > > > > > > > > > > > > dated 20231103 and without changing
> > > > > > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > > > > > topologies' code (keeping
> > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > same big jars).
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > This seems to be tied to a datatype
> > > > > > > > > > > > > > > > > > > > change in
> > > > > > > > > > > > > > > > > > > > Map
> > > > > > > > > > > > > > > > > > > > argument of
> > > > > > > > > > > > > > > > > > > > prepare() method of Bolt.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > For example, this Bolt is KO at
> > > > > > > > > > > > > > > > > > > > runtime:
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > public class MyKOBolt extends
> > > > > > > > > > > > > > > > > > > > BaseRichBolt {
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > > > > > > public void prepare(Map
> > > > > > > > > > > > > > > > > > > > stormConf,
> > > > > > > > > > > > > > > > > > > > TopologyContext context,
> > > > > > > > > > > > > > > > > > > > OutputCollector collector) {
> > > > > > > > > > > > > > > > > > > > cacheMaxSize = (Long)
> > > > > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACH
> > > > > > > > > > > > > > > > > > > > E_SI
> > > > > > > > > > > > > > > > > > > > ZE
> > > > > > > > > > > > > > > > > > > > );
> > > > > > > > > > > > > > > > > > > > cacheTtlMin = (Long)
> > > > > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACH
> > > > > > > > > > > > > > > > > > > > E_TT
> > > > > > > > > > > > > > > > > > > > L_
> > > > > > > > > > > > > > > > > > > > MIN);
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > as it throws this error:
> > > > > > > > > > > > > > > > > > > > 2023-11-03 16:42:20.230
> > > > > > > > > > > > > > > > > > > > o.a.s.e.b.BoltExecutor
> > > > > > > > > > > > > > > > > > > > Thread-14-__system-executor[-1, -1]
> > > > > > > > > > > > > > > > > > > > [INFO]
> > > > > > > > > > > > > > > > > > > > Preparing bolt
> > > > > > > > > > > > > > > > > > > > __system:[-1]
> > > > > > > > > > > > > > > > > > > > 2023-11-03 16:42:20.193
> > > > > > > > > > > > > > > > > > > > o.a.s.u.Utils
> > > > > > > > > > > > > > > > > > > > Thread-15-evaluateTriggers-
> > > > > > > > > > > > > > > > > > > > executor[11,
> > > > > > > > > > > > > > > > > > > > 11]
> > > > > > > > > > > > > > > > > > > > [ERROR] Async loop
> > > > > > > > > > > > > > > > > > > > died!
> > > > > > > > > > > > > > > > > > > > java.lang.ClassCastException:
> > > > > > > > > > > > > > > > > > > > java.lang.Integer
> > > > > > > > > > > > > > > > > > > > incompatible with
> > > > > > > > > > > > > > > > > > > > java.lang.Long
> > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > com.acme.storm.alerting.MyKOBolt
> > > > > > > > > > > > > > > > > > > > .prepare(EvaluationBolt.java:71)
> > > > > > > > > > > > > > > > > > > > ~[stormjar.jar:?]
> > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.Bolt
> > > > > > > > > > > > > > > > > > > > Exec
> > > > > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > > > > or.ini
> > > > > > > > > > > > > > > > > > > > t(
> > > > > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > > > > 128)
> > > > > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.Bolt
> > > > > > > > > > > > > > > > > > > > Exec
> > > > > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > > > > or.cal
> > > > > > > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > > > > 138)
> > > > > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.Bolt
> > > > > > > > > > > > > > > > > > > > Exec
> > > > > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > > > > or.cal
> > > > > > > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > > > > 54)
> > > > > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > org.apache.storm.utils.Utils$1.run(
> > > > > > > > > > > > > > > > > > > > Util
> > > > > > > > > > > > > > > > > > > > s.
> > > > > > > > > > > > > > > > > > > > java:3
> > > > > > > > > > > > > > > > > > > > 93
> > > > > > > > > > > > > > > > > > > > )
> > > > > > > > > > > > > > > > > > > > [storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > java.lang.Thread.run(Thread.java:85
> > > > > > > > > > > > > > > > > > > > 7)
> > > > > > > > > > > > > > > > > > > > [?:?]
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > whereas this bolt is fine with
> > > > > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > > > > 2.6.0:
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > public class MyOKBolt extends
> > > > > > > > > > > > > > > > > > > > BaseBasicBolt {
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > > > > > > public void prepare(final Map
> > > > > > > > > > > > > > > > > > > > stormConf,
> > > > > > > > > > > > > > > > > > > > final TopologyContext
> > > > > > > > > > > > > > > > > > > > context) {
> > > > > > > > > > > > > > > > > > > > super.prepare(stormConf,
> > > > > > > > > > > > > > > > > > > > context);
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > this.redisPort
> > > > > > > > > > > > > > > > > > > > =
> > > > > > > > > > > > > > > > > > > > Integer.parseInt((String)
> > > > > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.REDIS_PORT))
> > > > > > > > > > > > > > > > > > > > ;
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > That said, I can modify the code of
> > > > > > > > > > > > > > > > > > > > "MyKOBolt"
> > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > use same
> > > > > > > > > > > > > > > > > > > > Integer.parseInt or
> > > > > > > > > > > > > > > > > > > > Integer.parseLong
> > > > > > > > > > > > > > > > > > > > trick,
> > > > > > > > > > > > > > > > > > > > but
> > > > > > > > > > > > > > > > > > > > it's the first
> > > > > > > > > > > > > > > > > > > > time
> > > > > > > > > > > > > > > > > > > > in my long history of upgrades that
> > > > > > > > > > > > > > > > > > > > I
> > > > > > > > > > > > > > > > > > > > have seen
> > > > > > > > > > > > > > > > > > > > such runtime
> > > > > > > > > > > > > > > > > > > > incompatibility.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Maybe "MyKOBolt" was badly written
> > > > > > > > > > > > > > > > > > > > since
> > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > beginning and I have
> > > > > > > > > > > > > > > > > > > > just
> > > > > > > > > > > > > > > > > > > > hit the punishment for it: is there
> > > > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > > documentation which clarifies
> > > > > > > > > > > > > > > > > > > > datatypes of the Map argument of
> > > > > > > > > > > > > > > > > > > > prepare()
> > > > > > > > > > > > > > > > > > > > method?
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Also worth noting: "MyKOBolt"
> > > > > > > > > > > > > > > > > > > > derives
> > > > > > > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > > > > > BaseRichBolt , while
> > > > > > > > > > > > > > > > > > > > "MyOKBolt" derives from
> > > > > > > > > > > > > > > > > > > > BaseBasicBolt
> > > > > > > > > > > > > > > > > > > > =>
> > > > > > > > > > > > > > > > > > > > could
> > > > > > > > > > > > > > > > > > > > this have any impact
> > > > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > > this finding ?
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Last, I'm running this on Redhat
> > > > > > > > > > > > > > > > > > > > Linux
> > > > > > > > > > > > > > > > > > > > 8
> > > > > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > > IBM
> > > > > > > > > > > > > > > > > > > > Semeru JDK
> > > > > > > > > > > > > > > > > > > > 17.0.8.1.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 15:16, Julien
> > > > > > > > > > > > > > > > > > > > Nioche
> > > > > > > > > > > > > > > > > > > > <li...@gmail.com> a
> > > > > > > > > > > > > > > > > > > > écrit
> > > > > > > > > > > > > > > > > > > > :
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Thanks Richard.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Tried the latest snapshot with
> > > > > > > > > > > > > > > > > > > > > StormCrawler
> > > > > > > > > > > > > > > > > > > > > both in local and
> > > > > > > > > > > > > > > > > > > > > deployed mode
> > > > > > > > > > > > > > > > > > > > > and did not find any issues.
> > > > > > > > > > > > > > > > > > > > > Will try it on a topology
> > > > > > > > > > > > > > > > > > > > > generating
> > > > > > > > > > > > > > > > > > > > > WARC
> > > > > > > > > > > > > > > > > > > > > files
> > > > > > > > > > > > > > > > > > > > > next week to
> > > > > > > > > > > > > > > > > > > > > check that the
> > > > > > > > > > > > > > > > > > > > > dependency changes on Hadoop have
> > > > > > > > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > > > > > > broken
> > > > > > > > > > > > > > > > > > > > > anything.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Have a good week end
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Julien
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > On Thu, 2 Nov 2023 at 19:25,
> > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > > > > <rz...@apache.org>
> > > > > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > Short update. Sorted out an
> > > > > > > > > > > > > > > > > > > > > > issue
> > > > > > > > > > > > > > > > > > > > > > with the
> > > > > > > > > > > > > > > > > > > > > > tar.gz/zip with
> > > > > > > > > > > > > > > > > > > > > > Julien today
> > > > > > > > > > > > > > > > > > > > > > and re-uploaded them to the
> > > > > > > > > > > > > > > > > > > > > > nightlies
> > > > > > > > > > > > > > > > > > > > > > area.
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > This new bundle works as
> > > > > > > > > > > > > > > > > > > > > > expected
> > > > > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > > > > > > > deployment but happy to
> > > > > > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > > > > > additional feedback before
> > > > > > > > > > > > > > > > > > > > > > getting
> > > > > > > > > > > > > > > > > > > > > > up
> > > > > > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > > > > first
> > > > > > > > > > > > > > > > > > > > > > release candidate
> > > > > > > > > > > > > > > > > > > > > > ;-)
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um
> > > > > > > > > > > > > > > > > > > > > > 08:21
> > > > > > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > > > > > schrieb Richard
> > > > > > > > > > > > > > > > > > > > > > Zowalla:
> > > > > > > > > > > > > > > > > > > > > > > Hi Alexandre,
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > we are not in a hurry here :)
> > > > > > > > > > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > > > > > > > > > take as
> > > > > > > > > > > > > > > > > > > > > > > much as time you
> > > > > > > > > > > > > > > > > > > > > > > need.
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um
> > > > > > > > > > > > > > > > > > > > > > > 07:50
> > > > > > > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > > > > > > schrieb Alexandre
> > > > > > > > > > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > > > > > > > > > Hello Richard,
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > Okay, I'm more than happy
> > > > > > > > > > > > > > > > > > > > > > > > to do
> > > > > > > > > > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > > > > > my pre-production
> > > > > > > > > > > > > > > > > > > > > > > > cluster
> > > > > > > > > > > > > > > > > > > > > > > > (~10 topologies) using the
> > > > > > > > > > > > > > > > > > > > > > > > binary
> > > > > > > > > > > > > > > > > > > > > > > > artifacts.
> > > > > > > > > > > > > > > > > > > > > > > > Would it be OK if I can use
> > > > > > > > > > > > > > > > > > > > > > > > up
> > > > > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > > > end
> > > > > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > > > this week so that
> > > > > > > > > > > > > > > > > > > > > > > > I'll be
> > > > > > > > > > > > > > > > > > > > > > > > able to have enough time to
> > > > > > > > > > > > > > > > > > > > > > > > check
> > > > > > > > > > > > > > > > > > > > > > > > all
> > > > > > > > > > > > > > > > > > > > > > > > potential issues that
> > > > > > > > > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > > > > > > > > upgrade bring?
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > Le lun. 30 oct. 2023 à
> > > > > > > > > > > > > > > > > > > > > > > > 07:35,
> > > > > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > > > > > > > <rz...@apache.org> a
> > > > > > > > > > > > > > > > > > > > > > > > écrit :
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > before starting a release
> > > > > > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > > > > 2.6.0-
> > > > > > > > > > > > > > > > > > > > > > > > > SNAPSHOT, I would like
> > > > > > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > > > > > > > > some feedback on the
> > > > > > > > > > > > > > > > > > > > > > > > > current
> > > > > > > > > > > > > > > > > > > > > > > > > SNAPSHOT
> > > > > > > > > > > > > > > > > > > > > > > > > build.
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > I just uploaded a 2.6.0-
> > > > > > > > > > > > > > > > > > > > > > > > > SNAPSHOT of
> > > > > > > > > > > > > > > > > > > > > > > > > Storm build from
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > >
> > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > It is available via the
> > > > > > > > > > > > > > > > > > > > > > > > > ASF
> > > > > > > > > > > > > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > > > > > > > > > > > > repository:
> > > > > > > > > > > > > > > > > > > > > > > > >
> > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > You can consume by adding
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > <repository>
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > <id>apache.snapshots</id>
> > > > > > > > > > > > > > > > > > > > > > > > > <name>Apache
> > > > > > > > > > > > > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > > > > > > > > > > > > Repository</name>
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > <url>
> > > > > > > > > > > > > > > > > > > > > > > > >
> > https://repository.apache.org/snap
> > > > > > > > > > > > > > > > > > > > > > > > > shots</url>
> > > > > > > > > > > > > > > > > > > > > > > > > <releases>
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > <enabled>false</enabled>
> > > > > > > > > > > > > > > > > > > > > > > > > </releases>
> > > > > > > > > > > > > > > > > > > > > > > > > </repository>
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > to your project pom. As
> > > > > > > > > > > > > > > > > > > > > > > > > we do
> > > > > > > > > > > > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > > > > > > > > > > deploy SNAPSHOTS
> > > > > > > > > > > > > > > > > > > > > > > > > automatically,
> > > > > > > > > > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > > > > > > > > should be easy to just
> > > > > > > > > > > > > > > > > > > > > > > > > consume
> > > > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > > > latest SNAPSHOT.
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > The packaged binaries are
> > > > > > > > > > > > > > > > > > > > > > > > > available
> > > > > > > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > > > > > > nightlies.apache.org:
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > >
> > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > If you have some minutes
> > > > > > > > > > > > > > > > > > > > > > > > > left:
> > > > > > > > > > > > > > > > > > > > > > > > > Please
> > > > > > > > > > > > > > > > > > > > > > > > > test and report any
> > > > > > > > > > > > > > > > > > > > > > > > > issues
> > > > > > > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > > > > > > this binaries, so we can
> > > > > > > > > > > > > > > > > > > > > > > > > fix
> > > > > > > > > > > > > > > > > > > > > > > > > before
> > > > > > > > > > > > > > > > > > > > > > > > > attempting to
> > > > > > > > > > > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > The most significant
> > > > > > > > > > > > > > > > > > > > > > > > > changes
> > > > > > > > > > > > > > > > > > > > > > > > > are in
> > > > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > > > are of
> > > > > > > > > > > > > > > > > > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > > > > > > > upgraded from 2.x to 3.x
> > > > > > > > > > > > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > > > > > > > > > > > our
> > > > > > > > > > > > > > > > > > > > > > > > > own
> > > > > > > > > > > > > > > > > > > > > > > > > test
> > > > > > > > > > > > > > > > > > > > > > > > > coverage within
> > > > > > > > > > > > > > > > > > > > > > > > > the build
> > > > > > > > > > > > > > > > > > > > > > > > > looks
> > > > > > > > > > > > > > > > > > > > > > > > > good but would be nice
> > > > > > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > > > > get
> > > > > > > > > > > > > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > > > > > > > > > real world use-case
> > > > > > > > > > > > > > > > > > > > > > > > > feedback
> > > > > > > > > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > > > > > > > this. In addition, we had
> > > > > > > > > > > > > > > > > > > > > > > > > quite
> > > > > > > > > > > > > > > > > > > > > > > > > a lof
> > > > > > > > > > > > > > > > > > > > > > > > > of 3rd party
> > > > > > > > > > > > > > > > > > > > > > > > > dependency
> > > > > > > > > > > > > > > > > > > > > > > > > upgrades.
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > In addition, it contains
> > > > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > > > pruning
> > > > > > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > > > > external modules
> > > > > > > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > > > > > > listed
> > > > > > > > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > >
> > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > A summaryin Jira is here:
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > *Open Source Solutions for Text
> > > > > > > > > > > > > > > > > > > > > Engineering*
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > http://www.digitalpebble.com
> > > > > > > > > > > > > > > > > > > > > http://digitalpebble.blogspot.com/
> > > > > > > > > > > > > > > > > > > > > #digitalpebble
> > > > > > > > > > > > > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > > > http://twitter.com/digitalpebble>
> > > > > > > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > >
> > > >
> > >
> >
> >
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble <http://twitter.com/digitalpebble>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Julien Nioche <li...@gmail.com>.
I agree with Richard. The next release contains enough big changes and it
would be good to release it ASAP before embarking on another big
modification.
Julien
On Mon, 13 Nov 2023 at 12:08, Richard Zowalla <rz...@apache.org> wrote:
> Hi,
>
> yes - in principal I agree, that switching to slf4j 2.x would be great
> in the near future. However, it requires some more work due to our old
> dependency stack ("external modules", ...) and most likely a lot of
> exclusions.
>
> Therefore, personally, I wouldn't see it in a 2.6.0 release but as a
> task on which we could work after having a new version in the pipeline
>
> wdyt?
>
> Gruß
> Richard
>
> Am Sonntag, dem 12.11.2023 um 09:40 +0100 schrieb Alexandre
> Vermeerbergen:
> > Thank Richard for the great work to cleanup CVEs on in-work 2.6.0 !
> > One last "regret" I have with current in-work 2.6.0 is that we still
> > rely on very old slf4j 1.x.
> > While it was not on slf4j itself, the log4shell CVE should remind us
> > that we should keep loggers as up to date as possible.
> >
> > I see in a deployed Storm runtime the following hits:
> >
> > [root]# find . -name "slf*jar" -print
> > ./lib/slf4j-api-1.7.36.jar
> > ./lib-tools/storm-kafka-monitor/slf4j-api-1.7.36.jar
> > ./lib-tools/submit-tools/slf4j-api-1.7.36.jar
> > ./lib-tools/sql/core/slf4j-api-1.7.36.jar
> > ./lib-tools/sql/runtime/slf4j-api-1.7.36.jar
> > ./external/storm-autocreds/slf4j-api-1.7.36.jar
> > ./lib-worker/slf4j-api-1.7.36.jar
> > ./lib-webapp/slf4j-api-1.7.36.jar
> >
> > From https://www.slf4j.org/download.html it is said that "The older
> > stable SLF4J version is 1.7.36. It is no longer actively developed."
> >
> > The most current version of slf4j is 2.0.9.
> >
> > But I think at same time we would need to update these ones too,
> > otherwise we will have a mismatch between SLF4J v1 logger
> > implementations vs new v2 APIs:
> >
> > [root@ip-172-31-43-21 storm-stable]# find . -name "log*jar" -print
> > ./lib/log4j-over-slf4j-1.7.36.jar
> > ./lib/log4j-api-2.20.0.jar
> > ./lib/log4j-slf4j-impl-2.20.0.jar
> > ./lib/log4j-core-2.20.0.jar
> > ./external/storm-autocreds/log4j-api-2.20.0.jar
> > ./external/storm-autocreds/log4j-web-2.20.0.jar
> > ./external/storm-autocreds/log4j-1.2-api-2.20.0.jar
> > ./external/storm-autocreds/log4j-core-2.20.0.jar
> > ./lib-worker/log4j-over-slf4j-1.7.36.jar
> > ./lib-worker/log4j-api-2.20.0.jar
> > ./lib-worker/log4j-slf4j-impl-2.20.0.jar
> > ./lib-worker/log4j-core-2.20.0.jar
> > ./lib-webapp/log4j-over-slf4j-1.7.36.jar
> > ./lib-webapp/log4j-api-2.20.0.jar
> > ./lib-webapp/log4j-core-2.20.0.jar
> >
> > Disclaimer: I have no idea how hard this update would be.
> >
> > Anyone with good experience at updating SLF4J & related loggers could
> > react on this proposal ?
> >
> > Thanks,
> > Alexandre
> >
> >
> > Le jeu. 9 nov. 2023 à 15:04, Richard Zowalla <rz...@apache.org> a
> > écrit :
> > >
> > > Ok, most of the stuff reported by grype should be fixed now.
> > > I might be able to push a new snapshot within the next week, so we
> > > can
> > > do some testing and than go for a release candiate / vote :)
> > >
> > > Gruß
> > > Richard
> > >
> > >
> > >
> > > Am Donnerstag, dem 09.11.2023 um 08:53 +0100 schrieb Richard
> > > Zowalla:
> > > > Maybe just run grype [1] on the lib directories and post the
> > > > output
> > > > on
> > > > the list / thread, so we can create tickets for it
> > > >
> > > > There might be some false positives but will give good insights.
> > > >
> > > > Gruß
> > > > Richard
> > > >
> > > >
> > > > [1] https://github.com/anchore/grype
> > > > Am Donnerstag, dem 09.11.2023 um 08:49 +0100 schrieb Alexandre
> > > > Vermeerbergen:
> > > > > Yet another finding on my side to "not call yet for vote on a
> > > > > Storm
> > > > > 2.6.0", we have some CVEs and we shoudn't release a version
> > > > > having
> > > > > known CVEs.
> > > > >
> > > > > For example, we have
> > > > >
> > > > > find . -name "commons-text*jar" -print
> > > > > ./lib-webapp/commons-text-1.9.jar
> > > > > ./external/storm-autocreds/commons-text-1.10.0.jar
> > > > >
> > > > > The second one is okay from
> > > > > https://www.cvedetails.com/cve/CVE-2022-42889/ perspective, but
> > > > > not
> > > > > the first one.
> > > > >
> > > > > I will continue checking if there are other CVEs...
> > > > >
> > > > > Thanks,
> > > > > Alexandre
> > > > >
> > > > > Le mar. 7 nov. 2023 à 17:43, Alexandre Vermeerbergen
> > > > > <av...@gmail.com> a écrit :
> > > > > >
> > > > > > Oh thank you very much for this cherry-pinking, I was too
> > > > > > busy
> > > > > > doing
> > > > > > other unrelated stuff..
> > > > > >
> > > > > > Le mar. 7 nov. 2023 à 17:35, Richard Zowalla
> > > > > > <rz...@apache.org> a
> > > > > > écrit :
> > > > > > >
> > > > > > > I cherry-picked your commit and put that on master
> > > > > > >
> > > > > > >
> > > > > > > Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard
> > > > > > > Zowalla:
> > > > > > > > You pushed on your own branch. You would need to push to
> > > > > > > > apache/storm
> > > > > > > > main branch or open a PR ;-)
> > > > > > > >
> > > > > > > > Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb
> > > > > > > > Alexandre
> > > > > > > > Vermeerbergen:
> > > > > > > > > Just did a 'git push' with that trivial fix... is that
> > > > > > > > > enough?
> > > > > > > > >
> > > > > > > > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla
> > > > > > > > > <ri...@zowalla.com>
> > > > > > > > > a
> > > > > > > > > écrit :
> > > > > > > > > >
> > > > > > > > > > You can do as you like ;-)
> > > > > > > > > >
> > > > > > > > > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre
> > > > > > > > > > Vermeerbergen
> > > > > > > > > > <av...@gmail.com>:
> > > > > > > > > > > I mean :
> > > > > > > > > > >
> > > > > > > > > > > Can I keep using my
> > > > > > > > > > > 'STORM-3958-Capacity-to-set-St
> > > > > > > > > > > orm-UI's-title-in-conf/storm.yaml'
> > > > > > > > > > > branch to push this fix, or should I create a new
> > > > > > > > > > > one?
> > > > > > > > > > >
> > > > > > > > > > > Le sam. 4 nov. 2023 à 08:53, Alexandre
> > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > >
> > > > > > > > > > > > Can you keep using my
> > > > > > > > > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > > > > > > > > > conf/storm.yaml'
> > > > > > > > > > > > branch to push this fix, or should I create a new
> > > > > > > > > > > > one?
> > > > > > > > > > > >
> > > > > > > > > > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > > > > > > > > > <ri...@zowalla.com> a écrit :
> > > > > > > > > > > > >
> > > > > > > > > > > > > Just push a fix for it (referencing the
> > > > > > > > > > > > > original
> > > > > > > > > > > > > issue) ;-)
> > > > > > > > > > > > > -
> > > > > > > > > > > > > we hadn't a release in between, so not a big
> > > > > > > > > > > > > deal.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Am 4. November 2023 08:01:54 MEZ schrieb
> > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > Vermeerbergen <av...@gmail.com>:
> > > > > > > > > > > > > > Temporary workaround for the Storm UI title's
> > > > > > > > > > > > > > regression:
> > > > > > > > > > > > > > run this
> > > > > > > > > > > > > > command on the title-template.html file (in
> > > > > > > > > > > > > > storm-stable/public/templates directory)
> > > > > > > > > > > > > > after
> > > > > > > > > > > > > > having
> > > > > > > > > > > > > > extracted the
> > > > > > > > > > > > > > binary archive:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-
> > > > > > > > > > > > > > template.html
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Again sorry for having introduced this
> > > > > > > > > > > > > > regression.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre
> > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Updates regarding my test with Storm 2.6.0
> > > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > > 2023/11/02:
> > > > > > > > > > > > > > > 1. I have modified my KO bolts using this
> > > > > > > > > > > > > > > pattern in
> > > > > > > > > > > > > > > their prepare() method:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > long cacheMaxSize =
> > > > > > > > > > > > > > > Long.parseLong(stormConf.get(ConfKeys.SVC_D
> > > > > > > > > > > > > > > EF_C
> > > > > > > > > > > > > > > AC
> > > > > > > > > > > > > > > HE_SIZ
> > > > > > > > > > > > > > > E)
> > > > > > > > > > > > > > > .toString());
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > instead of:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > long cacheMaxSize = (Long)
> > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > 2. I have found the cause of the regression
> > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > loss of
> > > > > > > > > > > > > > > 'Storm UI'
> > > > > > > > > > > > > > > title at the top left-hand side of all
> > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > UI's
> > > > > > > > > > > > > > > pages.
> > > > > > > > > > > > > > > This is a mistake of mine in this PR:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > > > > > > > > > > > > More precisely, at line 19 of this
> > > > > > > > > > > > > > > file:
> > > > > > > > > > > > > > >
> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > > > > > > > > > > > > > we must have:
> > > > > > > > > > > > > > > <h1><a href="/">{{title}}</a></h1>
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > instead of:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > <h1><a href="/">{{ui.title}}</a></h1>
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I think I missed this during my tests
> > > > > > > > > > > > > > > because
> > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > browser's cache
> > > > > > > > > > > > > > > (bad excuse, I know...): probably I tested
> > > > > > > > > > > > > > > by-
> > > > > > > > > > > > > > > patching
> > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > deployed
> > > > > > > > > > > > > > > Storm cluster, and I finally forgot to
> > > > > > > > > > > > > > > checkout
> > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > latest change of
> > > > > > > > > > > > > > > my delivery.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I had to use "private navigation" to
> > > > > > > > > > > > > > > make
> > > > > > > > > > > > > > > sure
> > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > time it fixes this issue
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > What would be the quickest way for this
> > > > > > > > > > > > > > > very
> > > > > > > > > > > > > > > simple fix
> > > > > > > > > > > > > > > to be delivered ?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 20:57, Alexandre
> > > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Thanks Richard for the analysis.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > My thought on this are very simple: let
> > > > > > > > > > > > > > > > me
> > > > > > > > > > > > > > > > fix
> > > > > > > > > > > > > > > > my own
> > > > > > > > > > > > > > > > old code, after
> > > > > > > > > > > > > > > > all this was impacting only one of my
> > > > > > > > > > > > > > > > topologies, the
> > > > > > > > > > > > > > > > other one were
> > > > > > > > > > > > > > > > using more careful Integer.parseInt or
> > > > > > > > > > > > > > > > Integer.parseLong to avoid
> > > > > > > > > > > > > > > > making assumption on the way the values
> > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > config
> > > > > > > > > > > > > > > > Map were
> > > > > > > > > > > > > > > > serialized.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > My other concern with 2.6.0 Snapshot is
> > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > UI, the "Storm
> > > > > > > > > > > > > > > > UI" title at the top left hand side of
> > > > > > > > > > > > > > > > each
> > > > > > > > > > > > > > > > page is
> > > > > > > > > > > > > > > > missing. This may
> > > > > > > > > > > > > > > > be the result of one of my own PR, I'll
> > > > > > > > > > > > > > > > have
> > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > check
> > > > > > > > > > > > > > > > why...
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Alex
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 20:23, Richard
> > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > <rz...@apache.org> a écrit :
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > thanks for the feedback.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > The main difference to 2.5.0 is, that
> > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > switched
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > JSON
> > > > > > > > > > > > > > > > > implementation in Utils, which relied
> > > > > > > > > > > > > > > > > on a
> > > > > > > > > > > > > > > > > super old JSON library. What is
> > > > > > > > > > > > > > > > > happening
> > > > > > > > > > > > > > > > > here is,
> > > > > > > > > > > > > > > > > that the new JSON
> > > > > > > > > > > > > > > > > library (net.minidev) behaves
> > > > > > > > > > > > > > > > > differently.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > We can see it by writing some simple
> > > > > > > > > > > > > > > > > code
> > > > > > > > > > > > > > > > > without
> > > > > > > > > > > > > > > > > Storm (The related
> > > > > > > > > > > > > > > > > code line with the parsing is taken
> > > > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > Utils)
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Map<String, Long> conf = new
> > > > > > > > > > > > > > > > > HashMap<>();
> > > > > > > > > > > > > > > > > conf.put("test", 100000000L);
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Map<String, Object> deserTopoConf =
> > > > > > > > > > > > > > > > > (Map<String,
> > > > > > > > > > > > > > > > > Object>)
> > > > > > > > > > > > > > > > > JSONValue.parseWithException(JSONValue.
> > > > > > > > > > > > > > > > > toJS
> > > > > > > > > > > > > > > > > ON
> > > > > > > > > > > > > > > > > String
> > > > > > > > > > > > > > > > > (c
> > > > > > > > > > > > > > > > > onf));
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > System.out.println(e.getValue().getClas
> > > > > > > > > > > > > > > > > s().
> > > > > > > > > > > > > > > > > ge
> > > > > > > > > > > > > > > > > tSimpl
> > > > > > > > > > > > > > > > > eN
> > > > > > > > > > > > > > > > > ame());
> > > > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > This will print "Integer". Using
> > > > > > > > > > > > > > > > > "10000000000000000L"
> > > > > > > > > > > > > > > > > as a value will
> > > > > > > > > > > > > > > > > result in "Long".
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > The main issue derives from the fact,
> > > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > serialize the config map
> > > > > > > > > > > > > > > > > into a JSON String and parse the
> > > > > > > > > > > > > > > > > result.
> > > > > > > > > > > > > > > > > Based on the String representation,
> > > > > > > > > > > > > > > > > "100000000L"
> > > > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > still fit into an
> > > > > > > > > > > > > > > > > Integer, so the parser will not use
> > > > > > > > > > > > > > > > > Long ;-
> > > > > > > > > > > > > > > > > )
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > If the number is big enough, the parser
> > > > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > switch
> > > > > > > > > > > > > > > > > to a Long> >> > >> > > The old library
> > > > > > > > > > > > > > > > > (>
> > > > > > > > > > > > > > > > > 10
> > > > > > > > > > > > > > > > > years
> > > > > > > > > > > > > > > > > old now):
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > deserTopoConf = (Map<String,
> > > > > > > > > > > > > > > > > Object>)
> > > > > > > > > > > > > > > > > org.json.simple.JSONValue.parseWithExce
> > > > > > > > > > > > > > > > > ptio
> > > > > > > > > > > > > > > > > n(
> > > > > > > > > > > > > > > > > org.js
> > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > .simple.JSONValue.
> > > > > > > > > > > > > > > > > toJSONString(conf));
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > System.out.println(e.getValue().getClas
> > > > > > > > > > > > > > > > > s().
> > > > > > > > > > > > > > > > > ge
> > > > > > > > > > > > > > > > > tSimpl
> > > > > > > > > > > > > > > > > eN
> > > > > > > > > > > > > > > > > ame());
> > > > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > This will print "Long".
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > The difference between these libraries
> > > > > > > > > > > > > > > > > is,
> > > > > > > > > > > > > > > > > that the
> > > > > > > > > > > > > > > > > super old
> > > > > > > > > > > > > > > > > org.json.simple just always returns
> > > > > > > > > > > > > > > > > Long as
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > type.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > If can eventually tune the new JSON
> > > > > > > > > > > > > > > > > parser
> > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > something like:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > deserTopoConf = (Map<String, Object>)
> > > > > > > > > > > > > > > > > new
> > > > > > > > > > > > > > > > > JSONParser(JSONParser.BIG_DIGIT_UNRESTR
> > > > > > > > > > > > > > > > > ICTE
> > > > > > > > > > > > > > > > > D)
> > > > > > > > > > > > > > > > > .parse
> > > > > > > > > > > > > > > > > (J
> > > > > > > > > > > > > > > > > SONValue= toJSONStr
> > > > > > > > > > > > > > > > > ing(conf),
> > > > > > > > > > > > > > > > > JSONValue.defaultReader.DEFAULT);
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > This will result in "Long" but might
> > > > > > > > > > > > > > > > > have
> > > > > > > > > > > > > > > > > other
> > > > > > > > > > > > > > > > > side-
> > > > > > > > > > > > > > > > > effects.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > From my point of view, the newer JSON
> > > > > > > > > > > > > > > > > library
> > > > > > > > > > > > > > > > > works
> > > > > > > > > > > > > > > > > as expected
> > > > > > > > > > > > > > > > > (although it might break backward
> > > > > > > > > > > > > > > > > compatibility).
> > > > > > > > > > > > > > > > > Unchecked casts are always evil, imho.
> > > > > > > > > > > > > > > > > In
> > > > > > > > > > > > > > > > > your
> > > > > > > > > > > > > > > > > specific case, it might
> > > > > > > > > > > > > > > > > be more save to cast to "Number" and
> > > > > > > > > > > > > > > > > than
> > > > > > > > > > > > > > > > > invoke
> > > > > > > > > > > > > > > > > "longValue()" instead.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Nevertheless, we might being able to
> > > > > > > > > > > > > > > > > restore
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > old
> > > > > > > > > > > > > > > > > behaviour by
> > > > > > > > > > > > > > > > > tuning the JSONParser configuration.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Thoughts?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Am Freitag, dem 03.11.2023 um 18:32
> > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > schrieb
> > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > > > On a side note, the Long value that
> > > > > > > > > > > > > > > > > > "MyKOBolt" is
> > > > > > > > > > > > > > > > > > set by a
> > > > > > > > > > > > > > > > > > DefaultTopologyConfig to a value with
> > > > > > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > java.lang.Long instance...
> > > > > > > > > > > > > > > > > > hence it's surprising to see it
> > > > > > > > > > > > > > > > > > internally
> > > > > > > > > > > > > > > > > > converted into a
> > > > > > > > > > > > > > > > > > java.lang.Integer in the Map passed
> > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > prepare()
> > > > > > > > > > > > > > > > > > method...
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 18:27,
> > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Hello,
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > I found a breaking change using
> > > > > > > > > > > > > > > > > > > Apache
> > > > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > > > 2.6.0 snapshot build
> > > > > > > > > > > > > > > > > > > dated 20231103 and without changing
> > > > > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > > > > topologies' code (keeping
> > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > same big jars).
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > This seems to be tied to a datatype
> > > > > > > > > > > > > > > > > > > change in
> > > > > > > > > > > > > > > > > > > Map
> > > > > > > > > > > > > > > > > > > argument of
> > > > > > > > > > > > > > > > > > > prepare() method of Bolt.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > For example, this Bolt is KO at
> > > > > > > > > > > > > > > > > > > runtime:
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > public class MyKOBolt extends
> > > > > > > > > > > > > > > > > > > BaseRichBolt {
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > > > > > public void prepare(Map
> > > > > > > > > > > > > > > > > > > stormConf,
> > > > > > > > > > > > > > > > > > > TopologyContext context,
> > > > > > > > > > > > > > > > > > > OutputCollector collector) {
> > > > > > > > > > > > > > > > > > > cacheMaxSize = (Long)
> > > > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACH
> > > > > > > > > > > > > > > > > > > E_SI
> > > > > > > > > > > > > > > > > > > ZE
> > > > > > > > > > > > > > > > > > > );
> > > > > > > > > > > > > > > > > > > cacheTtlMin = (Long)
> > > > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACH
> > > > > > > > > > > > > > > > > > > E_TT
> > > > > > > > > > > > > > > > > > > L_
> > > > > > > > > > > > > > > > > > > MIN);
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > as it throws this error:
> > > > > > > > > > > > > > > > > > > 2023-11-03 16:42:20.230
> > > > > > > > > > > > > > > > > > > o.a.s.e.b.BoltExecutor
> > > > > > > > > > > > > > > > > > > Thread-14-__system-executor[-1, -1]
> > > > > > > > > > > > > > > > > > > [INFO]
> > > > > > > > > > > > > > > > > > > Preparing bolt
> > > > > > > > > > > > > > > > > > > __system:[-1]
> > > > > > > > > > > > > > > > > > > 2023-11-03 16:42:20.193
> > > > > > > > > > > > > > > > > > > o.a.s.u.Utils
> > > > > > > > > > > > > > > > > > > Thread-15-evaluateTriggers-
> > > > > > > > > > > > > > > > > > > executor[11,
> > > > > > > > > > > > > > > > > > > 11]
> > > > > > > > > > > > > > > > > > > [ERROR] Async loop
> > > > > > > > > > > > > > > > > > > died!
> > > > > > > > > > > > > > > > > > > java.lang.ClassCastException:
> > > > > > > > > > > > > > > > > > > java.lang.Integer
> > > > > > > > > > > > > > > > > > > incompatible with
> > > > > > > > > > > > > > > > > > > java.lang.Long
> > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > com.acme.storm.alerting.MyKOBolt
> > > > > > > > > > > > > > > > > > > .prepare(EvaluationBolt.java:71)
> > > > > > > > > > > > > > > > > > > ~[stormjar.jar:?]
> > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.Bolt
> > > > > > > > > > > > > > > > > > > Exec
> > > > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > > > or.ini
> > > > > > > > > > > > > > > > > > > t(
> > > > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > > > 128)
> > > > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.Bolt
> > > > > > > > > > > > > > > > > > > Exec
> > > > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > > > or.cal
> > > > > > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > > > 138)
> > > > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.Bolt
> > > > > > > > > > > > > > > > > > > Exec
> > > > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > > > or.cal
> > > > > > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > > > 54)
> > > > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > org.apache.storm.utils.Utils$1.run(
> > > > > > > > > > > > > > > > > > > Util
> > > > > > > > > > > > > > > > > > > s.
> > > > > > > > > > > > > > > > > > > java:3
> > > > > > > > > > > > > > > > > > > 93
> > > > > > > > > > > > > > > > > > > )
> > > > > > > > > > > > > > > > > > > [storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > java.lang.Thread.run(Thread.java:85
> > > > > > > > > > > > > > > > > > > 7)
> > > > > > > > > > > > > > > > > > > [?:?]
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > whereas this bolt is fine with
> > > > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > > > 2.6.0:
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > public class MyOKBolt extends
> > > > > > > > > > > > > > > > > > > BaseBasicBolt {
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > > > > > public void prepare(final Map
> > > > > > > > > > > > > > > > > > > stormConf,
> > > > > > > > > > > > > > > > > > > final TopologyContext
> > > > > > > > > > > > > > > > > > > context) {
> > > > > > > > > > > > > > > > > > > super.prepare(stormConf,
> > > > > > > > > > > > > > > > > > > context);
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > this.redisPort
> > > > > > > > > > > > > > > > > > > =
> > > > > > > > > > > > > > > > > > > Integer.parseInt((String)
> > > > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.REDIS_PORT))
> > > > > > > > > > > > > > > > > > > ;
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > That said, I can modify the code of
> > > > > > > > > > > > > > > > > > > "MyKOBolt"
> > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > use same
> > > > > > > > > > > > > > > > > > > Integer.parseInt or
> > > > > > > > > > > > > > > > > > > Integer.parseLong
> > > > > > > > > > > > > > > > > > > trick,
> > > > > > > > > > > > > > > > > > > but
> > > > > > > > > > > > > > > > > > > it's the first
> > > > > > > > > > > > > > > > > > > time
> > > > > > > > > > > > > > > > > > > in my long history of upgrades that
> > > > > > > > > > > > > > > > > > > I
> > > > > > > > > > > > > > > > > > > have seen
> > > > > > > > > > > > > > > > > > > such runtime
> > > > > > > > > > > > > > > > > > > incompatibility.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Maybe "MyKOBolt" was badly written
> > > > > > > > > > > > > > > > > > > since
> > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > beginning and I have
> > > > > > > > > > > > > > > > > > > just
> > > > > > > > > > > > > > > > > > > hit the punishment for it: is there
> > > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > documentation which clarifies
> > > > > > > > > > > > > > > > > > > datatypes of the Map argument of
> > > > > > > > > > > > > > > > > > > prepare()
> > > > > > > > > > > > > > > > > > > method?
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Also worth noting: "MyKOBolt"
> > > > > > > > > > > > > > > > > > > derives
> > > > > > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > > > > BaseRichBolt , while
> > > > > > > > > > > > > > > > > > > "MyOKBolt" derives from
> > > > > > > > > > > > > > > > > > > BaseBasicBolt
> > > > > > > > > > > > > > > > > > > =>
> > > > > > > > > > > > > > > > > > > could
> > > > > > > > > > > > > > > > > > > this have any impact
> > > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > this finding ?
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Last, I'm running this on Redhat
> > > > > > > > > > > > > > > > > > > Linux
> > > > > > > > > > > > > > > > > > > 8
> > > > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > IBM
> > > > > > > > > > > > > > > > > > > Semeru JDK
> > > > > > > > > > > > > > > > > > > 17.0.8.1.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 15:16, Julien
> > > > > > > > > > > > > > > > > > > Nioche
> > > > > > > > > > > > > > > > > > > <li...@gmail.com> a
> > > > > > > > > > > > > > > > > > > écrit
> > > > > > > > > > > > > > > > > > > :
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Thanks Richard.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Tried the latest snapshot with
> > > > > > > > > > > > > > > > > > > > StormCrawler
> > > > > > > > > > > > > > > > > > > > both in local and
> > > > > > > > > > > > > > > > > > > > deployed mode
> > > > > > > > > > > > > > > > > > > > and did not find any issues.
> > > > > > > > > > > > > > > > > > > > Will try it on a topology
> > > > > > > > > > > > > > > > > > > > generating
> > > > > > > > > > > > > > > > > > > > WARC
> > > > > > > > > > > > > > > > > > > > files
> > > > > > > > > > > > > > > > > > > > next week to
> > > > > > > > > > > > > > > > > > > > check that the
> > > > > > > > > > > > > > > > > > > > dependency changes on Hadoop have
> > > > > > > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > > > > > broken
> > > > > > > > > > > > > > > > > > > > anything.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Have a good week end
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Julien
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > On Thu, 2 Nov 2023 at 19:25,
> > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > > > <rz...@apache.org>
> > > > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Short update. Sorted out an
> > > > > > > > > > > > > > > > > > > > > issue
> > > > > > > > > > > > > > > > > > > > > with the
> > > > > > > > > > > > > > > > > > > > > tar.gz/zip with
> > > > > > > > > > > > > > > > > > > > > Julien today
> > > > > > > > > > > > > > > > > > > > > and re-uploaded them to the
> > > > > > > > > > > > > > > > > > > > > nightlies
> > > > > > > > > > > > > > > > > > > > > area.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > This new bundle works as
> > > > > > > > > > > > > > > > > > > > > expected
> > > > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > > > > > > deployment but happy to
> > > > > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > > > > additional feedback before
> > > > > > > > > > > > > > > > > > > > > getting
> > > > > > > > > > > > > > > > > > > > > up
> > > > > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > > > first
> > > > > > > > > > > > > > > > > > > > > release candidate
> > > > > > > > > > > > > > > > > > > > > ;-)
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um
> > > > > > > > > > > > > > > > > > > > > 08:21
> > > > > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > > > > schrieb Richard
> > > > > > > > > > > > > > > > > > > > > Zowalla:
> > > > > > > > > > > > > > > > > > > > > > Hi Alexandre,
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > we are not in a hurry here :)
> > > > > > > > > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > > > > > > > > take as
> > > > > > > > > > > > > > > > > > > > > > much as time you
> > > > > > > > > > > > > > > > > > > > > > need.
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um
> > > > > > > > > > > > > > > > > > > > > > 07:50
> > > > > > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > > > > > schrieb Alexandre
> > > > > > > > > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > > > > > > > > Hello Richard,
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > Okay, I'm more than happy
> > > > > > > > > > > > > > > > > > > > > > > to do
> > > > > > > > > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > > > > my pre-production
> > > > > > > > > > > > > > > > > > > > > > > cluster
> > > > > > > > > > > > > > > > > > > > > > > (~10 topologies) using the
> > > > > > > > > > > > > > > > > > > > > > > binary
> > > > > > > > > > > > > > > > > > > > > > > artifacts.
> > > > > > > > > > > > > > > > > > > > > > > Would it be OK if I can use
> > > > > > > > > > > > > > > > > > > > > > > up
> > > > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > > end
> > > > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > > this week so that
> > > > > > > > > > > > > > > > > > > > > > > I'll be
> > > > > > > > > > > > > > > > > > > > > > > able to have enough time to
> > > > > > > > > > > > > > > > > > > > > > > check
> > > > > > > > > > > > > > > > > > > > > > > all
> > > > > > > > > > > > > > > > > > > > > > > potential issues that
> > > > > > > > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > > > > > > > upgrade bring?
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > Le lun. 30 oct. 2023 à
> > > > > > > > > > > > > > > > > > > > > > > 07:35,
> > > > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > > > > > > <rz...@apache.org> a
> > > > > > > > > > > > > > > > > > > > > > > écrit :
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > before starting a release
> > > > > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > > > 2.6.0-
> > > > > > > > > > > > > > > > > > > > > > > > SNAPSHOT, I would like
> > > > > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > > > > > > > some feedback on the
> > > > > > > > > > > > > > > > > > > > > > > > current
> > > > > > > > > > > > > > > > > > > > > > > > SNAPSHOT
> > > > > > > > > > > > > > > > > > > > > > > > build.
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > I just uploaded a 2.6.0-
> > > > > > > > > > > > > > > > > > > > > > > > SNAPSHOT of
> > > > > > > > > > > > > > > > > > > > > > > > Storm build from
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > It is available via the
> > > > > > > > > > > > > > > > > > > > > > > > ASF
> > > > > > > > > > > > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > > > > > > > > > > > repository:
> > > > > > > > > > > > > > > > > > > > > > > >
> https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > You can consume by adding
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > <repository>
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > <id>apache.snapshots</id>
> > > > > > > > > > > > > > > > > > > > > > > > <name>Apache
> > > > > > > > > > > > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > > > > > > > > > > > Repository</name>
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > <url>
> > > > > > > > > > > > > > > > > > > > > > > >
> https://repository.apache.org/snap
> > > > > > > > > > > > > > > > > > > > > > > > shots</url>
> > > > > > > > > > > > > > > > > > > > > > > > <releases>
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > <enabled>false</enabled>
> > > > > > > > > > > > > > > > > > > > > > > > </releases>
> > > > > > > > > > > > > > > > > > > > > > > > </repository>
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > to your project pom. As
> > > > > > > > > > > > > > > > > > > > > > > > we do
> > > > > > > > > > > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > > > > > > > > > deploy SNAPSHOTS
> > > > > > > > > > > > > > > > > > > > > > > > automatically,
> > > > > > > > > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > > > > > > > should be easy to just
> > > > > > > > > > > > > > > > > > > > > > > > consume
> > > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > > latest SNAPSHOT.
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > The packaged binaries are
> > > > > > > > > > > > > > > > > > > > > > > > available
> > > > > > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > > > > > nightlies.apache.org:
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > If you have some minutes
> > > > > > > > > > > > > > > > > > > > > > > > left:
> > > > > > > > > > > > > > > > > > > > > > > > Please
> > > > > > > > > > > > > > > > > > > > > > > > test and report any
> > > > > > > > > > > > > > > > > > > > > > > > issues
> > > > > > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > > > > > this binaries, so we can
> > > > > > > > > > > > > > > > > > > > > > > > fix
> > > > > > > > > > > > > > > > > > > > > > > > before
> > > > > > > > > > > > > > > > > > > > > > > > attempting to
> > > > > > > > > > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > The most significant
> > > > > > > > > > > > > > > > > > > > > > > > changes
> > > > > > > > > > > > > > > > > > > > > > > > are in
> > > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > > are of
> > > > > > > > > > > > > > > > > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > > > > > > upgraded from 2.x to 3.x
> > > > > > > > > > > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > > > > > > > > > > our
> > > > > > > > > > > > > > > > > > > > > > > > own
> > > > > > > > > > > > > > > > > > > > > > > > test
> > > > > > > > > > > > > > > > > > > > > > > > coverage within
> > > > > > > > > > > > > > > > > > > > > > > > the build
> > > > > > > > > > > > > > > > > > > > > > > > looks
> > > > > > > > > > > > > > > > > > > > > > > > good but would be nice
> > > > > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > > > get
> > > > > > > > > > > > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > > > > > > > > real world use-case
> > > > > > > > > > > > > > > > > > > > > > > > feedback
> > > > > > > > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > > > > > > this. In addition, we had
> > > > > > > > > > > > > > > > > > > > > > > > quite
> > > > > > > > > > > > > > > > > > > > > > > > a lof
> > > > > > > > > > > > > > > > > > > > > > > > of 3rd party
> > > > > > > > > > > > > > > > > > > > > > > > dependency
> > > > > > > > > > > > > > > > > > > > > > > > upgrades.
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > In addition, it contains
> > > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > > pruning
> > > > > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > > > external modules
> > > > > > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > > > > > listed
> > > > > > > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > >
> https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > A summaryin Jira is here:
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > *Open Source Solutions for Text
> > > > > > > > > > > > > > > > > > > > Engineering*
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > http://www.digitalpebble.com
> > > > > > > > > > > > > > > > > > > > http://digitalpebble.blogspot.com/
> > > > > > > > > > > > > > > > > > > > #digitalpebble
> > > > > > > > > > > > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > > http://twitter.com/digitalpebble>
> > > > > > > > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > >
> > >
> >
>
>
--
*Open Source Solutions for Text Engineering*
http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble <http://twitter.com/digitalpebble>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Richard Zowalla <rz...@apache.org>.
Hi,
yes - in principal I agree, that switching to slf4j 2.x would be great
in the near future. However, it requires some more work due to our old
dependency stack ("external modules", ...) and most likely a lot of
exclusions.
Therefore, personally, I wouldn't see it in a 2.6.0 release but as a
task on which we could work after having a new version in the pipeline
wdyt?
Gruß
Richard
Am Sonntag, dem 12.11.2023 um 09:40 +0100 schrieb Alexandre
Vermeerbergen:
> Thank Richard for the great work to cleanup CVEs on in-work 2.6.0 !
> One last "regret" I have with current in-work 2.6.0 is that we still
> rely on very old slf4j 1.x.
> While it was not on slf4j itself, the log4shell CVE should remind us
> that we should keep loggers as up to date as possible.
>
> I see in a deployed Storm runtime the following hits:
>
> [root]# find . -name "slf*jar" -print
> ./lib/slf4j-api-1.7.36.jar
> ./lib-tools/storm-kafka-monitor/slf4j-api-1.7.36.jar
> ./lib-tools/submit-tools/slf4j-api-1.7.36.jar
> ./lib-tools/sql/core/slf4j-api-1.7.36.jar
> ./lib-tools/sql/runtime/slf4j-api-1.7.36.jar
> ./external/storm-autocreds/slf4j-api-1.7.36.jar
> ./lib-worker/slf4j-api-1.7.36.jar
> ./lib-webapp/slf4j-api-1.7.36.jar
>
> From https://www.slf4j.org/download.html it is said that "The older
> stable SLF4J version is 1.7.36. It is no longer actively developed."
>
> The most current version of slf4j is 2.0.9.
>
> But I think at same time we would need to update these ones too,
> otherwise we will have a mismatch between SLF4J v1 logger
> implementations vs new v2 APIs:
>
> [root@ip-172-31-43-21 storm-stable]# find . -name "log*jar" -print
> ./lib/log4j-over-slf4j-1.7.36.jar
> ./lib/log4j-api-2.20.0.jar
> ./lib/log4j-slf4j-impl-2.20.0.jar
> ./lib/log4j-core-2.20.0.jar
> ./external/storm-autocreds/log4j-api-2.20.0.jar
> ./external/storm-autocreds/log4j-web-2.20.0.jar
> ./external/storm-autocreds/log4j-1.2-api-2.20.0.jar
> ./external/storm-autocreds/log4j-core-2.20.0.jar
> ./lib-worker/log4j-over-slf4j-1.7.36.jar
> ./lib-worker/log4j-api-2.20.0.jar
> ./lib-worker/log4j-slf4j-impl-2.20.0.jar
> ./lib-worker/log4j-core-2.20.0.jar
> ./lib-webapp/log4j-over-slf4j-1.7.36.jar
> ./lib-webapp/log4j-api-2.20.0.jar
> ./lib-webapp/log4j-core-2.20.0.jar
>
> Disclaimer: I have no idea how hard this update would be.
>
> Anyone with good experience at updating SLF4J & related loggers could
> react on this proposal ?
>
> Thanks,
> Alexandre
>
>
> Le jeu. 9 nov. 2023 à 15:04, Richard Zowalla <rz...@apache.org> a
> écrit :
> >
> > Ok, most of the stuff reported by grype should be fixed now.
> > I might be able to push a new snapshot within the next week, so we
> > can
> > do some testing and than go for a release candiate / vote :)
> >
> > Gruß
> > Richard
> >
> >
> >
> > Am Donnerstag, dem 09.11.2023 um 08:53 +0100 schrieb Richard
> > Zowalla:
> > > Maybe just run grype [1] on the lib directories and post the
> > > output
> > > on
> > > the list / thread, so we can create tickets for it
> > >
> > > There might be some false positives but will give good insights.
> > >
> > > Gruß
> > > Richard
> > >
> > >
> > > [1] https://github.com/anchore/grype
> > > Am Donnerstag, dem 09.11.2023 um 08:49 +0100 schrieb Alexandre
> > > Vermeerbergen:
> > > > Yet another finding on my side to "not call yet for vote on a
> > > > Storm
> > > > 2.6.0", we have some CVEs and we shoudn't release a version
> > > > having
> > > > known CVEs.
> > > >
> > > > For example, we have
> > > >
> > > > find . -name "commons-text*jar" -print
> > > > ./lib-webapp/commons-text-1.9.jar
> > > > ./external/storm-autocreds/commons-text-1.10.0.jar
> > > >
> > > > The second one is okay from
> > > > https://www.cvedetails.com/cve/CVE-2022-42889/ perspective, but
> > > > not
> > > > the first one.
> > > >
> > > > I will continue checking if there are other CVEs...
> > > >
> > > > Thanks,
> > > > Alexandre
> > > >
> > > > Le mar. 7 nov. 2023 à 17:43, Alexandre Vermeerbergen
> > > > <av...@gmail.com> a écrit :
> > > > >
> > > > > Oh thank you very much for this cherry-pinking, I was too
> > > > > busy
> > > > > doing
> > > > > other unrelated stuff..
> > > > >
> > > > > Le mar. 7 nov. 2023 à 17:35, Richard Zowalla
> > > > > <rz...@apache.org> a
> > > > > écrit :
> > > > > >
> > > > > > I cherry-picked your commit and put that on master
> > > > > >
> > > > > >
> > > > > > Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard
> > > > > > Zowalla:
> > > > > > > You pushed on your own branch. You would need to push to
> > > > > > > apache/storm
> > > > > > > main branch or open a PR ;-)
> > > > > > >
> > > > > > > Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb
> > > > > > > Alexandre
> > > > > > > Vermeerbergen:
> > > > > > > > Just did a 'git push' with that trivial fix... is that
> > > > > > > > enough?
> > > > > > > >
> > > > > > > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla
> > > > > > > > <ri...@zowalla.com>
> > > > > > > > a
> > > > > > > > écrit :
> > > > > > > > >
> > > > > > > > > You can do as you like ;-)
> > > > > > > > >
> > > > > > > > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre
> > > > > > > > > Vermeerbergen
> > > > > > > > > <av...@gmail.com>:
> > > > > > > > > > I mean :
> > > > > > > > > >
> > > > > > > > > > Can I keep using my
> > > > > > > > > > 'STORM-3958-Capacity-to-set-St
> > > > > > > > > > orm-UI's-title-in-conf/storm.yaml'
> > > > > > > > > > branch to push this fix, or should I create a new
> > > > > > > > > > one?
> > > > > > > > > >
> > > > > > > > > > Le sam. 4 nov. 2023 à 08:53, Alexandre
> > > > > > > > > > Vermeerbergen
> > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > >
> > > > > > > > > > > Can you keep using my
> > > > > > > > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > > > > > > > > conf/storm.yaml'
> > > > > > > > > > > branch to push this fix, or should I create a new
> > > > > > > > > > > one?
> > > > > > > > > > >
> > > > > > > > > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > > > > > > > > <ri...@zowalla.com> a écrit :
> > > > > > > > > > > >
> > > > > > > > > > > > Just push a fix for it (referencing the
> > > > > > > > > > > > original
> > > > > > > > > > > > issue) ;-)
> > > > > > > > > > > > -
> > > > > > > > > > > > we hadn't a release in between, so not a big
> > > > > > > > > > > > deal.
> > > > > > > > > > > >
> > > > > > > > > > > > Am 4. November 2023 08:01:54 MEZ schrieb
> > > > > > > > > > > > Alexandre
> > > > > > > > > > > > Vermeerbergen <av...@gmail.com>:
> > > > > > > > > > > > > Temporary workaround for the Storm UI title's
> > > > > > > > > > > > > regression:
> > > > > > > > > > > > > run this
> > > > > > > > > > > > > command on the title-template.html file (in
> > > > > > > > > > > > > storm-stable/public/templates directory)
> > > > > > > > > > > > > after
> > > > > > > > > > > > > having
> > > > > > > > > > > > > extracted the
> > > > > > > > > > > > > binary archive:
> > > > > > > > > > > > >
> > > > > > > > > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-
> > > > > > > > > > > > > template.html
> > > > > > > > > > > > >
> > > > > > > > > > > > > Again sorry for having introduced this
> > > > > > > > > > > > > regression.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > >
> > > > > > > > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre
> > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Updates regarding my test with Storm 2.6.0
> > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > 2023/11/02:
> > > > > > > > > > > > > > 1. I have modified my KO bolts using this
> > > > > > > > > > > > > > pattern in
> > > > > > > > > > > > > > their prepare() method:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > long cacheMaxSize =
> > > > > > > > > > > > > > Long.parseLong(stormConf.get(ConfKeys.SVC_D
> > > > > > > > > > > > > > EF_C
> > > > > > > > > > > > > > AC
> > > > > > > > > > > > > > HE_SIZ
> > > > > > > > > > > > > > E)
> > > > > > > > > > > > > > .toString());
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > instead of:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > long cacheMaxSize = (Long)
> > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 2. I have found the cause of the regression
> > > > > > > > > > > > > > in
> > > > > > > > > > > > > > loss of
> > > > > > > > > > > > > > 'Storm UI'
> > > > > > > > > > > > > > title at the top left-hand side of all
> > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > UI's
> > > > > > > > > > > > > > pages.
> > > > > > > > > > > > > > This is a mistake of mine in this PR:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > > > > > > > > > > > More precisely, at line 19 of this
> > > > > > > > > > > > > > file:
> > > > > > > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > > > > > > > > > > > > we must have:
> > > > > > > > > > > > > > <h1><a href="/">{{title}}</a></h1>
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > instead of:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > <h1><a href="/">{{ui.title}}</a></h1>
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I think I missed this during my tests
> > > > > > > > > > > > > > because
> > > > > > > > > > > > > > of
> > > > > > > > > > > > > > browser's cache
> > > > > > > > > > > > > > (bad excuse, I know...): probably I tested
> > > > > > > > > > > > > > by-
> > > > > > > > > > > > > > patching
> > > > > > > > > > > > > > my
> > > > > > > > > > > > > > deployed
> > > > > > > > > > > > > > Storm cluster, and I finally forgot to
> > > > > > > > > > > > > > checkout
> > > > > > > > > > > > > > this
> > > > > > > > > > > > > > latest change of
> > > > > > > > > > > > > > my delivery.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I had to use "private navigation" to
> > > > > > > > > > > > > > make
> > > > > > > > > > > > > > sure
> > > > > > > > > > > > > > this
> > > > > > > > > > > > > > time it fixes this issue
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > What would be the quickest way for this
> > > > > > > > > > > > > > very
> > > > > > > > > > > > > > simple fix
> > > > > > > > > > > > > > to be delivered ?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 20:57, Alexandre
> > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thanks Richard for the analysis.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > My thought on this are very simple: let
> > > > > > > > > > > > > > > me
> > > > > > > > > > > > > > > fix
> > > > > > > > > > > > > > > my own
> > > > > > > > > > > > > > > old code, after
> > > > > > > > > > > > > > > all this was impacting only one of my
> > > > > > > > > > > > > > > topologies, the
> > > > > > > > > > > > > > > other one were
> > > > > > > > > > > > > > > using more careful Integer.parseInt or
> > > > > > > > > > > > > > > Integer.parseLong to avoid
> > > > > > > > > > > > > > > making assumption on the way the values
> > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > config
> > > > > > > > > > > > > > > Map were
> > > > > > > > > > > > > > > serialized.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > My other concern with 2.6.0 Snapshot is
> > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > UI, the "Storm
> > > > > > > > > > > > > > > UI" title at the top left hand side of
> > > > > > > > > > > > > > > each
> > > > > > > > > > > > > > > page is
> > > > > > > > > > > > > > > missing. This may
> > > > > > > > > > > > > > > be the result of one of my own PR, I'll
> > > > > > > > > > > > > > > have
> > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > check
> > > > > > > > > > > > > > > why...
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Alex
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 20:23, Richard
> > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > <rz...@apache.org> a écrit :
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > thanks for the feedback.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The main difference to 2.5.0 is, that
> > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > switched
> > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > JSON
> > > > > > > > > > > > > > > > implementation in Utils, which relied
> > > > > > > > > > > > > > > > on a
> > > > > > > > > > > > > > > > super old JSON library. What is
> > > > > > > > > > > > > > > > happening
> > > > > > > > > > > > > > > > here is,
> > > > > > > > > > > > > > > > that the new JSON
> > > > > > > > > > > > > > > > library (net.minidev) behaves
> > > > > > > > > > > > > > > > differently.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > We can see it by writing some simple
> > > > > > > > > > > > > > > > code
> > > > > > > > > > > > > > > > without
> > > > > > > > > > > > > > > > Storm (The related
> > > > > > > > > > > > > > > > code line with the parsing is taken
> > > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > Utils)
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Map<String, Long> conf = new
> > > > > > > > > > > > > > > > HashMap<>();
> > > > > > > > > > > > > > > > conf.put("test", 100000000L);
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Map<String, Object> deserTopoConf =
> > > > > > > > > > > > > > > > (Map<String,
> > > > > > > > > > > > > > > > Object>)
> > > > > > > > > > > > > > > > JSONValue.parseWithException(JSONValue.
> > > > > > > > > > > > > > > > toJS
> > > > > > > > > > > > > > > > ON
> > > > > > > > > > > > > > > > String
> > > > > > > > > > > > > > > > (c
> > > > > > > > > > > > > > > > onf));
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > System.out.println(e.getValue().getClas
> > > > > > > > > > > > > > > > s().
> > > > > > > > > > > > > > > > ge
> > > > > > > > > > > > > > > > tSimpl
> > > > > > > > > > > > > > > > eN
> > > > > > > > > > > > > > > > ame());
> > > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > This will print "Integer". Using
> > > > > > > > > > > > > > > > "10000000000000000L"
> > > > > > > > > > > > > > > > as a value will
> > > > > > > > > > > > > > > > result in "Long".
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The main issue derives from the fact,
> > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > serialize the config map
> > > > > > > > > > > > > > > > into a JSON String and parse the
> > > > > > > > > > > > > > > > result.
> > > > > > > > > > > > > > > > Based on the String representation,
> > > > > > > > > > > > > > > > "100000000L"
> > > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > still fit into an
> > > > > > > > > > > > > > > > Integer, so the parser will not use
> > > > > > > > > > > > > > > > Long ;-
> > > > > > > > > > > > > > > > )
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > If the number is big enough, the parser
> > > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > switch
> > > > > > > > > > > > > > > > to a Long> >> > >> > > The old library
> > > > > > > > > > > > > > > > (>
> > > > > > > > > > > > > > > > 10
> > > > > > > > > > > > > > > > years
> > > > > > > > > > > > > > > > old now):
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > deserTopoConf = (Map<String,
> > > > > > > > > > > > > > > > Object>)
> > > > > > > > > > > > > > > > org.json.simple.JSONValue.parseWithExce
> > > > > > > > > > > > > > > > ptio
> > > > > > > > > > > > > > > > n(
> > > > > > > > > > > > > > > > org.js
> > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > .simple.JSONValue.
> > > > > > > > > > > > > > > > toJSONString(conf));
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > System.out.println(e.getValue().getClas
> > > > > > > > > > > > > > > > s().
> > > > > > > > > > > > > > > > ge
> > > > > > > > > > > > > > > > tSimpl
> > > > > > > > > > > > > > > > eN
> > > > > > > > > > > > > > > > ame());
> > > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > This will print "Long".
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The difference between these libraries
> > > > > > > > > > > > > > > > is,
> > > > > > > > > > > > > > > > that the
> > > > > > > > > > > > > > > > super old
> > > > > > > > > > > > > > > > org.json.simple just always returns
> > > > > > > > > > > > > > > > Long as
> > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > type.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > If can eventually tune the new JSON
> > > > > > > > > > > > > > > > parser
> > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > something like:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > deserTopoConf = (Map<String, Object>)
> > > > > > > > > > > > > > > > new
> > > > > > > > > > > > > > > > JSONParser(JSONParser.BIG_DIGIT_UNRESTR
> > > > > > > > > > > > > > > > ICTE
> > > > > > > > > > > > > > > > D)
> > > > > > > > > > > > > > > > .parse
> > > > > > > > > > > > > > > > (J
> > > > > > > > > > > > > > > > SONValue= toJSONStr
> > > > > > > > > > > > > > > > ing(conf),
> > > > > > > > > > > > > > > > JSONValue.defaultReader.DEFAULT);
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > This will result in "Long" but might
> > > > > > > > > > > > > > > > have
> > > > > > > > > > > > > > > > other
> > > > > > > > > > > > > > > > side-
> > > > > > > > > > > > > > > > effects.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > From my point of view, the newer JSON
> > > > > > > > > > > > > > > > library
> > > > > > > > > > > > > > > > works
> > > > > > > > > > > > > > > > as expected
> > > > > > > > > > > > > > > > (although it might break backward
> > > > > > > > > > > > > > > > compatibility).
> > > > > > > > > > > > > > > > Unchecked casts are always evil, imho.
> > > > > > > > > > > > > > > > In
> > > > > > > > > > > > > > > > your
> > > > > > > > > > > > > > > > specific case, it might
> > > > > > > > > > > > > > > > be more save to cast to "Number" and
> > > > > > > > > > > > > > > > than
> > > > > > > > > > > > > > > > invoke
> > > > > > > > > > > > > > > > "longValue()" instead.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Nevertheless, we might being able to
> > > > > > > > > > > > > > > > restore
> > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > old
> > > > > > > > > > > > > > > > behaviour by
> > > > > > > > > > > > > > > > tuning the JSONParser configuration.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Thoughts?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Am Freitag, dem 03.11.2023 um 18:32
> > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > schrieb
> > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > > On a side note, the Long value that
> > > > > > > > > > > > > > > > > "MyKOBolt" is
> > > > > > > > > > > > > > > > > set by a
> > > > > > > > > > > > > > > > > DefaultTopologyConfig to a value with
> > > > > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > java.lang.Long instance...
> > > > > > > > > > > > > > > > > hence it's surprising to see it
> > > > > > > > > > > > > > > > > internally
> > > > > > > > > > > > > > > > > converted into a
> > > > > > > > > > > > > > > > > java.lang.Integer in the Map passed
> > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > prepare()
> > > > > > > > > > > > > > > > > method...
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 18:27,
> > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Hello,
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > I found a breaking change using
> > > > > > > > > > > > > > > > > > Apache
> > > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > > 2.6.0 snapshot build
> > > > > > > > > > > > > > > > > > dated 20231103 and without changing
> > > > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > > > topologies' code (keeping
> > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > same big jars).
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > This seems to be tied to a datatype
> > > > > > > > > > > > > > > > > > change in
> > > > > > > > > > > > > > > > > > Map
> > > > > > > > > > > > > > > > > > argument of
> > > > > > > > > > > > > > > > > > prepare() method of Bolt.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > For example, this Bolt is KO at
> > > > > > > > > > > > > > > > > > runtime:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > public class MyKOBolt extends
> > > > > > > > > > > > > > > > > > BaseRichBolt {
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > > > > public void prepare(Map
> > > > > > > > > > > > > > > > > > stormConf,
> > > > > > > > > > > > > > > > > > TopologyContext context,
> > > > > > > > > > > > > > > > > > OutputCollector collector) {
> > > > > > > > > > > > > > > > > > cacheMaxSize = (Long)
> > > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACH
> > > > > > > > > > > > > > > > > > E_SI
> > > > > > > > > > > > > > > > > > ZE
> > > > > > > > > > > > > > > > > > );
> > > > > > > > > > > > > > > > > > cacheTtlMin = (Long)
> > > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACH
> > > > > > > > > > > > > > > > > > E_TT
> > > > > > > > > > > > > > > > > > L_
> > > > > > > > > > > > > > > > > > MIN);
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > as it throws this error:
> > > > > > > > > > > > > > > > > > 2023-11-03 16:42:20.230
> > > > > > > > > > > > > > > > > > o.a.s.e.b.BoltExecutor
> > > > > > > > > > > > > > > > > > Thread-14-__system-executor[-1, -1]
> > > > > > > > > > > > > > > > > > [INFO]
> > > > > > > > > > > > > > > > > > Preparing bolt
> > > > > > > > > > > > > > > > > > __system:[-1]
> > > > > > > > > > > > > > > > > > 2023-11-03 16:42:20.193
> > > > > > > > > > > > > > > > > > o.a.s.u.Utils
> > > > > > > > > > > > > > > > > > Thread-15-evaluateTriggers-
> > > > > > > > > > > > > > > > > > executor[11,
> > > > > > > > > > > > > > > > > > 11]
> > > > > > > > > > > > > > > > > > [ERROR] Async loop
> > > > > > > > > > > > > > > > > > died!
> > > > > > > > > > > > > > > > > > java.lang.ClassCastException:
> > > > > > > > > > > > > > > > > > java.lang.Integer
> > > > > > > > > > > > > > > > > > incompatible with
> > > > > > > > > > > > > > > > > > java.lang.Long
> > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > com.acme.storm.alerting.MyKOBolt
> > > > > > > > > > > > > > > > > > .prepare(EvaluationBolt.java:71)
> > > > > > > > > > > > > > > > > > ~[stormjar.jar:?]
> > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.Bolt
> > > > > > > > > > > > > > > > > > Exec
> > > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > > or.ini
> > > > > > > > > > > > > > > > > > t(
> > > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > > 128)
> > > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.Bolt
> > > > > > > > > > > > > > > > > > Exec
> > > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > > or.cal
> > > > > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > > 138)
> > > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.Bolt
> > > > > > > > > > > > > > > > > > Exec
> > > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > > or.cal
> > > > > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > > 54)
> > > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > org.apache.storm.utils.Utils$1.run(
> > > > > > > > > > > > > > > > > > Util
> > > > > > > > > > > > > > > > > > s.
> > > > > > > > > > > > > > > > > > java:3
> > > > > > > > > > > > > > > > > > 93
> > > > > > > > > > > > > > > > > > )
> > > > > > > > > > > > > > > > > > [storm-client-2.6.0-
> > > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > java.lang.Thread.run(Thread.java:85
> > > > > > > > > > > > > > > > > > 7)
> > > > > > > > > > > > > > > > > > [?:?]
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > whereas this bolt is fine with
> > > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > > 2.6.0:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > public class MyOKBolt extends
> > > > > > > > > > > > > > > > > > BaseBasicBolt {
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > > > > public void prepare(final Map
> > > > > > > > > > > > > > > > > > stormConf,
> > > > > > > > > > > > > > > > > > final TopologyContext
> > > > > > > > > > > > > > > > > > context) {
> > > > > > > > > > > > > > > > > > super.prepare(stormConf,
> > > > > > > > > > > > > > > > > > context);
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > this.redisPort
> > > > > > > > > > > > > > > > > > =
> > > > > > > > > > > > > > > > > > Integer.parseInt((String)
> > > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.REDIS_PORT))
> > > > > > > > > > > > > > > > > > ;
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > That said, I can modify the code of
> > > > > > > > > > > > > > > > > > "MyKOBolt"
> > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > use same
> > > > > > > > > > > > > > > > > > Integer.parseInt or
> > > > > > > > > > > > > > > > > > Integer.parseLong
> > > > > > > > > > > > > > > > > > trick,
> > > > > > > > > > > > > > > > > > but
> > > > > > > > > > > > > > > > > > it's the first
> > > > > > > > > > > > > > > > > > time
> > > > > > > > > > > > > > > > > > in my long history of upgrades that
> > > > > > > > > > > > > > > > > > I
> > > > > > > > > > > > > > > > > > have seen
> > > > > > > > > > > > > > > > > > such runtime
> > > > > > > > > > > > > > > > > > incompatibility.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Maybe "MyKOBolt" was badly written
> > > > > > > > > > > > > > > > > > since
> > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > beginning and I have
> > > > > > > > > > > > > > > > > > just
> > > > > > > > > > > > > > > > > > hit the punishment for it: is there
> > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > documentation which clarifies
> > > > > > > > > > > > > > > > > > datatypes of the Map argument of
> > > > > > > > > > > > > > > > > > prepare()
> > > > > > > > > > > > > > > > > > method?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Also worth noting: "MyKOBolt"
> > > > > > > > > > > > > > > > > > derives
> > > > > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > > > BaseRichBolt , while
> > > > > > > > > > > > > > > > > > "MyOKBolt" derives from
> > > > > > > > > > > > > > > > > > BaseBasicBolt
> > > > > > > > > > > > > > > > > > =>
> > > > > > > > > > > > > > > > > > could
> > > > > > > > > > > > > > > > > > this have any impact
> > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > this finding ?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Last, I'm running this on Redhat
> > > > > > > > > > > > > > > > > > Linux
> > > > > > > > > > > > > > > > > > 8
> > > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > IBM
> > > > > > > > > > > > > > > > > > Semeru JDK
> > > > > > > > > > > > > > > > > > 17.0.8.1.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 15:16, Julien
> > > > > > > > > > > > > > > > > > Nioche
> > > > > > > > > > > > > > > > > > <li...@gmail.com> a
> > > > > > > > > > > > > > > > > > écrit
> > > > > > > > > > > > > > > > > > :
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Thanks Richard.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Tried the latest snapshot with
> > > > > > > > > > > > > > > > > > > StormCrawler
> > > > > > > > > > > > > > > > > > > both in local and
> > > > > > > > > > > > > > > > > > > deployed mode
> > > > > > > > > > > > > > > > > > > and did not find any issues.
> > > > > > > > > > > > > > > > > > > Will try it on a topology
> > > > > > > > > > > > > > > > > > > generating
> > > > > > > > > > > > > > > > > > > WARC
> > > > > > > > > > > > > > > > > > > files
> > > > > > > > > > > > > > > > > > > next week to
> > > > > > > > > > > > > > > > > > > check that the
> > > > > > > > > > > > > > > > > > > dependency changes on Hadoop have
> > > > > > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > > > > broken
> > > > > > > > > > > > > > > > > > > anything.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Have a good week end
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Julien
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > On Thu, 2 Nov 2023 at 19:25,
> > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > > <rz...@apache.org>
> > > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Short update. Sorted out an
> > > > > > > > > > > > > > > > > > > > issue
> > > > > > > > > > > > > > > > > > > > with the
> > > > > > > > > > > > > > > > > > > > tar.gz/zip with
> > > > > > > > > > > > > > > > > > > > Julien today
> > > > > > > > > > > > > > > > > > > > and re-uploaded them to the
> > > > > > > > > > > > > > > > > > > > nightlies
> > > > > > > > > > > > > > > > > > > > area.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > This new bundle works as
> > > > > > > > > > > > > > > > > > > > expected
> > > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > > > > > deployment but happy to
> > > > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > > > additional feedback before
> > > > > > > > > > > > > > > > > > > > getting
> > > > > > > > > > > > > > > > > > > > up
> > > > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > > first
> > > > > > > > > > > > > > > > > > > > release candidate
> > > > > > > > > > > > > > > > > > > > ;-)
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um
> > > > > > > > > > > > > > > > > > > > 08:21
> > > > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > > > schrieb Richard
> > > > > > > > > > > > > > > > > > > > Zowalla:
> > > > > > > > > > > > > > > > > > > > > Hi Alexandre,
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > we are not in a hurry here :)
> > > > > > > > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > > > > > > > take as
> > > > > > > > > > > > > > > > > > > > > much as time you
> > > > > > > > > > > > > > > > > > > > > need.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um
> > > > > > > > > > > > > > > > > > > > > 07:50
> > > > > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > > > > schrieb Alexandre
> > > > > > > > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > > > > > > > Hello Richard,
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > Okay, I'm more than happy
> > > > > > > > > > > > > > > > > > > > > > to do
> > > > > > > > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > > > my pre-production
> > > > > > > > > > > > > > > > > > > > > > cluster
> > > > > > > > > > > > > > > > > > > > > > (~10 topologies) using the
> > > > > > > > > > > > > > > > > > > > > > binary
> > > > > > > > > > > > > > > > > > > > > > artifacts.
> > > > > > > > > > > > > > > > > > > > > > Would it be OK if I can use
> > > > > > > > > > > > > > > > > > > > > > up
> > > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > end
> > > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > this week so that
> > > > > > > > > > > > > > > > > > > > > > I'll be
> > > > > > > > > > > > > > > > > > > > > > able to have enough time to
> > > > > > > > > > > > > > > > > > > > > > check
> > > > > > > > > > > > > > > > > > > > > > all
> > > > > > > > > > > > > > > > > > > > > > potential issues that
> > > > > > > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > > > > > > upgrade bring?
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > Le lun. 30 oct. 2023 à
> > > > > > > > > > > > > > > > > > > > > > 07:35,
> > > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > > > > > <rz...@apache.org> a
> > > > > > > > > > > > > > > > > > > > > > écrit :
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > before starting a release
> > > > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > > 2.6.0-
> > > > > > > > > > > > > > > > > > > > > > > SNAPSHOT, I would like
> > > > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > > > > > > some feedback on the
> > > > > > > > > > > > > > > > > > > > > > > current
> > > > > > > > > > > > > > > > > > > > > > > SNAPSHOT
> > > > > > > > > > > > > > > > > > > > > > > build.
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > I just uploaded a 2.6.0-
> > > > > > > > > > > > > > > > > > > > > > > SNAPSHOT of
> > > > > > > > > > > > > > > > > > > > > > > Storm build from
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > It is available via the
> > > > > > > > > > > > > > > > > > > > > > > ASF
> > > > > > > > > > > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > > > > > > > > > > repository:
> > > > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > You can consume by adding
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > <repository>
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > <id>apache.snapshots</id>
> > > > > > > > > > > > > > > > > > > > > > > <name>Apache
> > > > > > > > > > > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > > > > > > > > > > Repository</name>
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > <url>
> > > > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/snap
> > > > > > > > > > > > > > > > > > > > > > > shots</url>
> > > > > > > > > > > > > > > > > > > > > > > <releases>
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > <enabled>false</enabled>
> > > > > > > > > > > > > > > > > > > > > > > </releases>
> > > > > > > > > > > > > > > > > > > > > > > </repository>
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > to your project pom. As
> > > > > > > > > > > > > > > > > > > > > > > we do
> > > > > > > > > > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > > > > > > > > deploy SNAPSHOTS
> > > > > > > > > > > > > > > > > > > > > > > automatically,
> > > > > > > > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > > > > > > should be easy to just
> > > > > > > > > > > > > > > > > > > > > > > consume
> > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > latest SNAPSHOT.
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > The packaged binaries are
> > > > > > > > > > > > > > > > > > > > > > > available
> > > > > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > > > > nightlies.apache.org:
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > If you have some minutes
> > > > > > > > > > > > > > > > > > > > > > > left:
> > > > > > > > > > > > > > > > > > > > > > > Please
> > > > > > > > > > > > > > > > > > > > > > > test and report any
> > > > > > > > > > > > > > > > > > > > > > > issues
> > > > > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > > > > this binaries, so we can
> > > > > > > > > > > > > > > > > > > > > > > fix
> > > > > > > > > > > > > > > > > > > > > > > before
> > > > > > > > > > > > > > > > > > > > > > > attempting to
> > > > > > > > > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > The most significant
> > > > > > > > > > > > > > > > > > > > > > > changes
> > > > > > > > > > > > > > > > > > > > > > > are in
> > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > are of
> > > > > > > > > > > > > > > > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > > > > > upgraded from 2.x to 3.x
> > > > > > > > > > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > > > > > > > > > our
> > > > > > > > > > > > > > > > > > > > > > > own
> > > > > > > > > > > > > > > > > > > > > > > test
> > > > > > > > > > > > > > > > > > > > > > > coverage within
> > > > > > > > > > > > > > > > > > > > > > > the build
> > > > > > > > > > > > > > > > > > > > > > > looks
> > > > > > > > > > > > > > > > > > > > > > > good but would be nice
> > > > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > > get
> > > > > > > > > > > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > > > > > > > real world use-case
> > > > > > > > > > > > > > > > > > > > > > > feedback
> > > > > > > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > > > > > this. In addition, we had
> > > > > > > > > > > > > > > > > > > > > > > quite
> > > > > > > > > > > > > > > > > > > > > > > a lof
> > > > > > > > > > > > > > > > > > > > > > > of 3rd party
> > > > > > > > > > > > > > > > > > > > > > > dependency
> > > > > > > > > > > > > > > > > > > > > > > upgrades.
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > In addition, it contains
> > > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > pruning
> > > > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > > external modules
> > > > > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > > > > listed
> > > > > > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > A summaryin Jira is here:
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > *Open Source Solutions for Text
> > > > > > > > > > > > > > > > > > > Engineering*
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > http://www.digitalpebble.com
> > > > > > > > > > > > > > > > > > > http://digitalpebble.blogspot.com/
> > > > > > > > > > > > > > > > > > > #digitalpebble
> > > > > > > > > > > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > http://twitter.com/digitalpebble>
> > > > > > > > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > >
> >
>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Thank Richard for the great work to cleanup CVEs on in-work 2.6.0 !
One last "regret" I have with current in-work 2.6.0 is that we still
rely on very old slf4j 1.x.
While it was not on slf4j itself, the log4shell CVE should remind us
that we should keep loggers as up to date as possible.
I see in a deployed Storm runtime the following hits:
[root]# find . -name "slf*jar" -print
./lib/slf4j-api-1.7.36.jar
./lib-tools/storm-kafka-monitor/slf4j-api-1.7.36.jar
./lib-tools/submit-tools/slf4j-api-1.7.36.jar
./lib-tools/sql/core/slf4j-api-1.7.36.jar
./lib-tools/sql/runtime/slf4j-api-1.7.36.jar
./external/storm-autocreds/slf4j-api-1.7.36.jar
./lib-worker/slf4j-api-1.7.36.jar
./lib-webapp/slf4j-api-1.7.36.jar
From https://www.slf4j.org/download.html it is said that "The older
stable SLF4J version is 1.7.36. It is no longer actively developed."
The most current version of slf4j is 2.0.9.
But I think at same time we would need to update these ones too,
otherwise we will have a mismatch between SLF4J v1 logger
implementations vs new v2 APIs:
[root@ip-172-31-43-21 storm-stable]# find . -name "log*jar" -print
./lib/log4j-over-slf4j-1.7.36.jar
./lib/log4j-api-2.20.0.jar
./lib/log4j-slf4j-impl-2.20.0.jar
./lib/log4j-core-2.20.0.jar
./external/storm-autocreds/log4j-api-2.20.0.jar
./external/storm-autocreds/log4j-web-2.20.0.jar
./external/storm-autocreds/log4j-1.2-api-2.20.0.jar
./external/storm-autocreds/log4j-core-2.20.0.jar
./lib-worker/log4j-over-slf4j-1.7.36.jar
./lib-worker/log4j-api-2.20.0.jar
./lib-worker/log4j-slf4j-impl-2.20.0.jar
./lib-worker/log4j-core-2.20.0.jar
./lib-webapp/log4j-over-slf4j-1.7.36.jar
./lib-webapp/log4j-api-2.20.0.jar
./lib-webapp/log4j-core-2.20.0.jar
Disclaimer: I have no idea how hard this update would be.
Anyone with good experience at updating SLF4J & related loggers could
react on this proposal ?
Thanks,
Alexandre
Le jeu. 9 nov. 2023 à 15:04, Richard Zowalla <rz...@apache.org> a écrit :
>
> Ok, most of the stuff reported by grype should be fixed now.
> I might be able to push a new snapshot within the next week, so we can
> do some testing and than go for a release candiate / vote :)
>
> Gruß
> Richard
>
>
>
> Am Donnerstag, dem 09.11.2023 um 08:53 +0100 schrieb Richard Zowalla:
> > Maybe just run grype [1] on the lib directories and post the output
> > on
> > the list / thread, so we can create tickets for it
> >
> > There might be some false positives but will give good insights.
> >
> > Gruß
> > Richard
> >
> >
> > [1] https://github.com/anchore/grype
> > Am Donnerstag, dem 09.11.2023 um 08:49 +0100 schrieb Alexandre
> > Vermeerbergen:
> > > Yet another finding on my side to "not call yet for vote on a Storm
> > > 2.6.0", we have some CVEs and we shoudn't release a version having
> > > known CVEs.
> > >
> > > For example, we have
> > >
> > > find . -name "commons-text*jar" -print
> > > ./lib-webapp/commons-text-1.9.jar
> > > ./external/storm-autocreds/commons-text-1.10.0.jar
> > >
> > > The second one is okay from
> > > https://www.cvedetails.com/cve/CVE-2022-42889/ perspective, but not
> > > the first one.
> > >
> > > I will continue checking if there are other CVEs...
> > >
> > > Thanks,
> > > Alexandre
> > >
> > > Le mar. 7 nov. 2023 à 17:43, Alexandre Vermeerbergen
> > > <av...@gmail.com> a écrit :
> > > >
> > > > Oh thank you very much for this cherry-pinking, I was too busy
> > > > doing
> > > > other unrelated stuff..
> > > >
> > > > Le mar. 7 nov. 2023 à 17:35, Richard Zowalla <rz...@apache.org> a
> > > > écrit :
> > > > >
> > > > > I cherry-picked your commit and put that on master
> > > > >
> > > > >
> > > > > Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard
> > > > > Zowalla:
> > > > > > You pushed on your own branch. You would need to push to
> > > > > > apache/storm
> > > > > > main branch or open a PR ;-)
> > > > > >
> > > > > > Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb Alexandre
> > > > > > Vermeerbergen:
> > > > > > > Just did a 'git push' with that trivial fix... is that
> > > > > > > enough?
> > > > > > >
> > > > > > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla
> > > > > > > <ri...@zowalla.com>
> > > > > > > a
> > > > > > > écrit :
> > > > > > > >
> > > > > > > > You can do as you like ;-)
> > > > > > > >
> > > > > > > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre
> > > > > > > > Vermeerbergen
> > > > > > > > <av...@gmail.com>:
> > > > > > > > > I mean :
> > > > > > > > >
> > > > > > > > > Can I keep using my
> > > > > > > > > 'STORM-3958-Capacity-to-set-St
> > > > > > > > > orm-UI's-title-in-conf/storm.yaml'
> > > > > > > > > branch to push this fix, or should I create a new one?
> > > > > > > > >
> > > > > > > > > Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > >
> > > > > > > > > > Can you keep using my
> > > > > > > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > > > > > > > conf/storm.yaml'
> > > > > > > > > > branch to push this fix, or should I create a new
> > > > > > > > > > one?
> > > > > > > > > >
> > > > > > > > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > > > > > > > <ri...@zowalla.com> a écrit :
> > > > > > > > > > >
> > > > > > > > > > > Just push a fix for it (referencing the original
> > > > > > > > > > > issue) ;-)
> > > > > > > > > > > -
> > > > > > > > > > > we hadn't a release in between, so not a big deal.
> > > > > > > > > > >
> > > > > > > > > > > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre
> > > > > > > > > > > Vermeerbergen <av...@gmail.com>:
> > > > > > > > > > > > Temporary workaround for the Storm UI title's
> > > > > > > > > > > > regression:
> > > > > > > > > > > > run this
> > > > > > > > > > > > command on the title-template.html file (in
> > > > > > > > > > > > storm-stable/public/templates directory) after
> > > > > > > > > > > > having
> > > > > > > > > > > > extracted the
> > > > > > > > > > > > binary archive:
> > > > > > > > > > > >
> > > > > > > > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-
> > > > > > > > > > > > template.html
> > > > > > > > > > > >
> > > > > > > > > > > > Again sorry for having introduced this
> > > > > > > > > > > > regression.
> > > > > > > > > > > >
> > > > > > > > > > > > Alexandre
> > > > > > > > > > > >
> > > > > > > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre
> > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > >
> > > > > > > > > > > > > Updates regarding my test with Storm 2.6.0
> > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > 2023/11/02:
> > > > > > > > > > > > > 1. I have modified my KO bolts using this
> > > > > > > > > > > > > pattern in
> > > > > > > > > > > > > their prepare() method:
> > > > > > > > > > > > >
> > > > > > > > > > > > > long cacheMaxSize =
> > > > > > > > > > > > > Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_C
> > > > > > > > > > > > > AC
> > > > > > > > > > > > > HE_SIZ
> > > > > > > > > > > > > E)
> > > > > > > > > > > > > .toString());
> > > > > > > > > > > > >
> > > > > > > > > > > > > instead of:
> > > > > > > > > > > > >
> > > > > > > > > > > > > long cacheMaxSize = (Long)
> > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > > > > > >
> > > > > > > > > > > > > 2. I have found the cause of the regression in
> > > > > > > > > > > > > loss of
> > > > > > > > > > > > > 'Storm UI'
> > > > > > > > > > > > > title at the top left-hand side of all Storm
> > > > > > > > > > > > > UI's
> > > > > > > > > > > > > pages.
> > > > > > > > > > > > > This is a mistake of mine in this PR:
> > > > > > > > > > > > >
> > > > > > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > > > > > > > > > > More precisely, at line 19 of this file:
> > > > > > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > > > > > > > > > > > we must have:
> > > > > > > > > > > > > <h1><a href="/">{{title}}</a></h1>
> > > > > > > > > > > > >
> > > > > > > > > > > > > instead of:
> > > > > > > > > > > > >
> > > > > > > > > > > > > <h1><a href="/">{{ui.title}}</a></h1>
> > > > > > > > > > > > >
> > > > > > > > > > > > > I think I missed this during my tests
> > > > > > > > > > > > > because
> > > > > > > > > > > > > of
> > > > > > > > > > > > > browser's cache
> > > > > > > > > > > > > (bad excuse, I know...): probably I tested by-
> > > > > > > > > > > > > patching
> > > > > > > > > > > > > my
> > > > > > > > > > > > > deployed
> > > > > > > > > > > > > Storm cluster, and I finally forgot to checkout
> > > > > > > > > > > > > this
> > > > > > > > > > > > > latest change of
> > > > > > > > > > > > > my delivery.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I had to use "private navigation" to make
> > > > > > > > > > > > > sure
> > > > > > > > > > > > > this
> > > > > > > > > > > > > time it fixes this issue
> > > > > > > > > > > > >
> > > > > > > > > > > > > What would be the quickest way for this very
> > > > > > > > > > > > > simple fix
> > > > > > > > > > > > > to be delivered ?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > >
> > > > > > > > > > > > > Le ven. 3 nov. 2023 à 20:57, Alexandre
> > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Thanks Richard for the analysis.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > My thought on this are very simple: let me
> > > > > > > > > > > > > > fix
> > > > > > > > > > > > > > my own
> > > > > > > > > > > > > > old code, after
> > > > > > > > > > > > > > all this was impacting only one of my
> > > > > > > > > > > > > > topologies, the
> > > > > > > > > > > > > > other one were
> > > > > > > > > > > > > > using more careful Integer.parseInt or
> > > > > > > > > > > > > > Integer.parseLong to avoid
> > > > > > > > > > > > > > making assumption on the way the values in
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > config
> > > > > > > > > > > > > > Map were
> > > > > > > > > > > > > > serialized.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > My other concern with 2.6.0 Snapshot is that
> > > > > > > > > > > > > > in
> > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > UI, the "Storm
> > > > > > > > > > > > > > UI" title at the top left hand side of each
> > > > > > > > > > > > > > page is
> > > > > > > > > > > > > > missing. This may
> > > > > > > > > > > > > > be the result of one of my own PR, I'll have
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > check
> > > > > > > > > > > > > > why...
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Alex
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla
> > > > > > > > > > > > > > <rz...@apache.org> a écrit :
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > thanks for the feedback.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The main difference to 2.5.0 is, that we
> > > > > > > > > > > > > > > switched
> > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > JSON
> > > > > > > > > > > > > > > implementation in Utils, which relied on a
> > > > > > > > > > > > > > > super old JSON library. What is happening
> > > > > > > > > > > > > > > here is,
> > > > > > > > > > > > > > > that the new JSON
> > > > > > > > > > > > > > > library (net.minidev) behaves differently.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > We can see it by writing some simple code
> > > > > > > > > > > > > > > without
> > > > > > > > > > > > > > > Storm (The related
> > > > > > > > > > > > > > > code line with the parsing is taken from
> > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > Utils)
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Map<String, Long> conf = new
> > > > > > > > > > > > > > > HashMap<>();
> > > > > > > > > > > > > > > conf.put("test", 100000000L);
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Map<String, Object> deserTopoConf =
> > > > > > > > > > > > > > > (Map<String,
> > > > > > > > > > > > > > > Object>)
> > > > > > > > > > > > > > > JSONValue.parseWithException(JSONValue.toJS
> > > > > > > > > > > > > > > ON
> > > > > > > > > > > > > > > String
> > > > > > > > > > > > > > > (c
> > > > > > > > > > > > > > > onf));
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > System.out.println(e.getValue().getClass().
> > > > > > > > > > > > > > > ge
> > > > > > > > > > > > > > > tSimpl
> > > > > > > > > > > > > > > eN
> > > > > > > > > > > > > > > ame());
> > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > This will print "Integer". Using
> > > > > > > > > > > > > > > "10000000000000000L"
> > > > > > > > > > > > > > > as a value will
> > > > > > > > > > > > > > > result in "Long".
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The main issue derives from the fact, that
> > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > serialize the config map
> > > > > > > > > > > > > > > into a JSON String and parse the result.
> > > > > > > > > > > > > > > Based on the String representation,
> > > > > > > > > > > > > > > "100000000L"
> > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > still fit into an
> > > > > > > > > > > > > > > Integer, so the parser will not use Long ;-
> > > > > > > > > > > > > > > )
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > If the number is big enough, the parser
> > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > switch
> > > > > > > > > > > > > > > to a Long> >> > >> > > The old library (>
> > > > > > > > > > > > > > > 10
> > > > > > > > > > > > > > > years
> > > > > > > > > > > > > > > old now):
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > deserTopoConf = (Map<String, Object>)
> > > > > > > > > > > > > > > org.json.simple.JSONValue.parseWithExceptio
> > > > > > > > > > > > > > > n(
> > > > > > > > > > > > > > > org.js
> > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > .simple.JSONValue.
> > > > > > > > > > > > > > > toJSONString(conf));
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > System.out.println(e.getValue().getClass().
> > > > > > > > > > > > > > > ge
> > > > > > > > > > > > > > > tSimpl
> > > > > > > > > > > > > > > eN
> > > > > > > > > > > > > > > ame());
> > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > This will print "Long".
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The difference between these libraries is,
> > > > > > > > > > > > > > > that the
> > > > > > > > > > > > > > > super old
> > > > > > > > > > > > > > > org.json.simple just always returns Long as
> > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > type.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > If can eventually tune the new JSON parser
> > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > something like:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > deserTopoConf = (Map<String, Object>) new
> > > > > > > > > > > > > > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTE
> > > > > > > > > > > > > > > D)
> > > > > > > > > > > > > > > .parse
> > > > > > > > > > > > > > > (J
> > > > > > > > > > > > > > > SONValue= toJSONStr
> > > > > > > > > > > > > > > ing(conf),
> > > > > > > > > > > > > > > JSONValue.defaultReader.DEFAULT);
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > This will result in "Long" but might have
> > > > > > > > > > > > > > > other
> > > > > > > > > > > > > > > side-
> > > > > > > > > > > > > > > effects.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > From my point of view, the newer JSON
> > > > > > > > > > > > > > > library
> > > > > > > > > > > > > > > works
> > > > > > > > > > > > > > > as expected
> > > > > > > > > > > > > > > (although it might break backward
> > > > > > > > > > > > > > > compatibility).
> > > > > > > > > > > > > > > Unchecked casts are always evil, imho. In
> > > > > > > > > > > > > > > your
> > > > > > > > > > > > > > > specific case, it might
> > > > > > > > > > > > > > > be more save to cast to "Number" and than
> > > > > > > > > > > > > > > invoke
> > > > > > > > > > > > > > > "longValue()" instead.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Nevertheless, we might being able to
> > > > > > > > > > > > > > > restore
> > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > old
> > > > > > > > > > > > > > > behaviour by
> > > > > > > > > > > > > > > tuning the JSONParser configuration.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thoughts?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Am Freitag, dem 03.11.2023 um 18:32 +0100
> > > > > > > > > > > > > > > schrieb
> > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > On a side note, the Long value that
> > > > > > > > > > > > > > > > "MyKOBolt" is
> > > > > > > > > > > > > > > > set by a
> > > > > > > > > > > > > > > > DefaultTopologyConfig to a value with is
> > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > java.lang.Long instance...
> > > > > > > > > > > > > > > > hence it's surprising to see it
> > > > > > > > > > > > > > > > internally
> > > > > > > > > > > > > > > > converted into a
> > > > > > > > > > > > > > > > java.lang.Integer in the Map passed to
> > > > > > > > > > > > > > > > prepare()
> > > > > > > > > > > > > > > > method...
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 18:27, Alexandre
> > > > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Hello,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I found a breaking change using Apache
> > > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > > 2.6.0 snapshot build
> > > > > > > > > > > > > > > > > dated 20231103 and without changing my
> > > > > > > > > > > > > > > > > topologies' code (keeping
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > same big jars).
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > This seems to be tied to a datatype
> > > > > > > > > > > > > > > > > change in
> > > > > > > > > > > > > > > > > Map
> > > > > > > > > > > > > > > > > argument of
> > > > > > > > > > > > > > > > > prepare() method of Bolt.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > For example, this Bolt is KO at
> > > > > > > > > > > > > > > > > runtime:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > public class MyKOBolt extends
> > > > > > > > > > > > > > > > > BaseRichBolt {
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > > > public void prepare(Map stormConf,
> > > > > > > > > > > > > > > > > TopologyContext context,
> > > > > > > > > > > > > > > > > OutputCollector collector) {
> > > > > > > > > > > > > > > > > cacheMaxSize = (Long)
> > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SI
> > > > > > > > > > > > > > > > > ZE
> > > > > > > > > > > > > > > > > );
> > > > > > > > > > > > > > > > > cacheTtlMin = (Long)
> > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TT
> > > > > > > > > > > > > > > > > L_
> > > > > > > > > > > > > > > > > MIN);
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > as it throws this error:
> > > > > > > > > > > > > > > > > 2023-11-03 16:42:20.230
> > > > > > > > > > > > > > > > > o.a.s.e.b.BoltExecutor
> > > > > > > > > > > > > > > > > Thread-14-__system-executor[-1, -1]
> > > > > > > > > > > > > > > > > [INFO]
> > > > > > > > > > > > > > > > > Preparing bolt
> > > > > > > > > > > > > > > > > __system:[-1]
> > > > > > > > > > > > > > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > > > > > > > > > > > > > > > Thread-15-evaluateTriggers-executor[11,
> > > > > > > > > > > > > > > > > 11]
> > > > > > > > > > > > > > > > > [ERROR] Async loop
> > > > > > > > > > > > > > > > > died!
> > > > > > > > > > > > > > > > > java.lang.ClassCastException:
> > > > > > > > > > > > > > > > > java.lang.Integer
> > > > > > > > > > > > > > > > > incompatible with
> > > > > > > > > > > > > > > > > java.lang.Long
> > > > > > > > > > > > > > > > > at com.acme.storm.alerting.MyKOBolt
> > > > > > > > > > > > > > > > > .prepare(EvaluationBolt.java:71)
> > > > > > > > > > > > > > > > > ~[stormjar.jar:?]
> > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExec
> > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > or.ini
> > > > > > > > > > > > > > > > > t(
> > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > 128)
> > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExec
> > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > or.cal
> > > > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > 138)
> > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExec
> > > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > > or.cal
> > > > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > > 54)
> > > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > org.apache.storm.utils.Utils$1.run(Util
> > > > > > > > > > > > > > > > > s.
> > > > > > > > > > > > > > > > > java:3
> > > > > > > > > > > > > > > > > 93
> > > > > > > > > > > > > > > > > )
> > > > > > > > > > > > > > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > java.lang.Thread.run(Thread.java:857)
> > > > > > > > > > > > > > > > > [?:?]
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > whereas this bolt is fine with Storm
> > > > > > > > > > > > > > > > > 2.6.0:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > public class MyOKBolt extends
> > > > > > > > > > > > > > > > > BaseBasicBolt {
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > > > public void prepare(final Map
> > > > > > > > > > > > > > > > > stormConf,
> > > > > > > > > > > > > > > > > final TopologyContext
> > > > > > > > > > > > > > > > > context) {
> > > > > > > > > > > > > > > > > super.prepare(stormConf,
> > > > > > > > > > > > > > > > > context);
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > this.redisPort
> > > > > > > > > > > > > > > > > =
> > > > > > > > > > > > > > > > > Integer.parseInt((String)
> > > > > > > > > > > > > > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > That said, I can modify the code of
> > > > > > > > > > > > > > > > > "MyKOBolt"
> > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > use same
> > > > > > > > > > > > > > > > > Integer.parseInt or Integer.parseLong
> > > > > > > > > > > > > > > > > trick,
> > > > > > > > > > > > > > > > > but
> > > > > > > > > > > > > > > > > it's the first
> > > > > > > > > > > > > > > > > time
> > > > > > > > > > > > > > > > > in my long history of upgrades that I
> > > > > > > > > > > > > > > > > have seen
> > > > > > > > > > > > > > > > > such runtime
> > > > > > > > > > > > > > > > > incompatibility.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Maybe "MyKOBolt" was badly written
> > > > > > > > > > > > > > > > > since
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > beginning and I have
> > > > > > > > > > > > > > > > > just
> > > > > > > > > > > > > > > > > hit the punishment for it: is there a
> > > > > > > > > > > > > > > > > documentation which clarifies
> > > > > > > > > > > > > > > > > datatypes of the Map argument of
> > > > > > > > > > > > > > > > > prepare()
> > > > > > > > > > > > > > > > > method?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Also worth noting: "MyKOBolt" derives
> > > > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > > BaseRichBolt , while
> > > > > > > > > > > > > > > > > "MyOKBolt" derives from BaseBasicBolt
> > > > > > > > > > > > > > > > > =>
> > > > > > > > > > > > > > > > > could
> > > > > > > > > > > > > > > > > this have any impact
> > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > this finding ?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Last, I'm running this on Redhat Linux
> > > > > > > > > > > > > > > > > 8
> > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > IBM
> > > > > > > > > > > > > > > > > Semeru JDK
> > > > > > > > > > > > > > > > > 17.0.8.1.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 15:16, Julien
> > > > > > > > > > > > > > > > > Nioche
> > > > > > > > > > > > > > > > > <li...@gmail.com> a écrit
> > > > > > > > > > > > > > > > > :
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Thanks Richard.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Tried the latest snapshot with
> > > > > > > > > > > > > > > > > > StormCrawler
> > > > > > > > > > > > > > > > > > both in local and
> > > > > > > > > > > > > > > > > > deployed mode
> > > > > > > > > > > > > > > > > > and did not find any issues.
> > > > > > > > > > > > > > > > > > Will try it on a topology generating
> > > > > > > > > > > > > > > > > > WARC
> > > > > > > > > > > > > > > > > > files
> > > > > > > > > > > > > > > > > > next week to
> > > > > > > > > > > > > > > > > > check that the
> > > > > > > > > > > > > > > > > > dependency changes on Hadoop have not
> > > > > > > > > > > > > > > > > > broken
> > > > > > > > > > > > > > > > > > anything.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Have a good week end
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Julien
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > On Thu, 2 Nov 2023 at 19:25, Richard
> > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > <rz...@apache.org>
> > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Short update. Sorted out an issue
> > > > > > > > > > > > > > > > > > > with the
> > > > > > > > > > > > > > > > > > > tar.gz/zip with
> > > > > > > > > > > > > > > > > > > Julien today
> > > > > > > > > > > > > > > > > > > and re-uploaded them to the
> > > > > > > > > > > > > > > > > > > nightlies
> > > > > > > > > > > > > > > > > > > area.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > This new bundle works as expected
> > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > > > > deployment but happy to
> > > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > > additional feedback before getting
> > > > > > > > > > > > > > > > > > > up
> > > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > first
> > > > > > > > > > > > > > > > > > > release candidate
> > > > > > > > > > > > > > > > > > > ;-)
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 08:21
> > > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > > schrieb Richard
> > > > > > > > > > > > > > > > > > > Zowalla:
> > > > > > > > > > > > > > > > > > > > Hi Alexandre,
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > we are not in a hurry here :) -
> > > > > > > > > > > > > > > > > > > > take as
> > > > > > > > > > > > > > > > > > > > much as time you
> > > > > > > > > > > > > > > > > > > > need.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um
> > > > > > > > > > > > > > > > > > > > 07:50
> > > > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > > > schrieb Alexandre
> > > > > > > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > > > > > > Hello Richard,
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Okay, I'm more than happy to do
> > > > > > > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > > my pre-production
> > > > > > > > > > > > > > > > > > > > > cluster
> > > > > > > > > > > > > > > > > > > > > (~10 topologies) using the
> > > > > > > > > > > > > > > > > > > > > binary
> > > > > > > > > > > > > > > > > > > > > artifacts.
> > > > > > > > > > > > > > > > > > > > > Would it be OK if I can use up
> > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > end
> > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > this week so that
> > > > > > > > > > > > > > > > > > > > > I'll be
> > > > > > > > > > > > > > > > > > > > > able to have enough time to
> > > > > > > > > > > > > > > > > > > > > check
> > > > > > > > > > > > > > > > > > > > > all
> > > > > > > > > > > > > > > > > > > > > potential issues that
> > > > > > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > > > > > upgrade bring?
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Le lun. 30 oct. 2023 à 07:35,
> > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > > > > <rz...@apache.org> a
> > > > > > > > > > > > > > > > > > > > > écrit :
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > before starting a release of
> > > > > > > > > > > > > > > > > > > > > > 2.6.0-
> > > > > > > > > > > > > > > > > > > > > > SNAPSHOT, I would like
> > > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > > > > > some feedback on the current
> > > > > > > > > > > > > > > > > > > > > > SNAPSHOT
> > > > > > > > > > > > > > > > > > > > > > build.
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > I just uploaded a 2.6.0-
> > > > > > > > > > > > > > > > > > > > > > SNAPSHOT of
> > > > > > > > > > > > > > > > > > > > > > Storm build from
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > It is available via the ASF
> > > > > > > > > > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > > > > > > > > > repository:
> > > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > You can consume by adding
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > <repository>
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > <id>apache.snapshots</id>
> > > > > > > > > > > > > > > > > > > > > > <name>Apache Snapshot
> > > > > > > > > > > > > > > > > > > > > > Repository</name>
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > <url>
> > > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/snap
> > > > > > > > > > > > > > > > > > > > > > shots</url>
> > > > > > > > > > > > > > > > > > > > > > <releases>
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > <enabled>false</enabled>
> > > > > > > > > > > > > > > > > > > > > > </releases>
> > > > > > > > > > > > > > > > > > > > > > </repository>
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > to your project pom. As we do
> > > > > > > > > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > > > > > > > deploy SNAPSHOTS
> > > > > > > > > > > > > > > > > > > > > > automatically,
> > > > > > > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > > > > > should be easy to just
> > > > > > > > > > > > > > > > > > > > > > consume
> > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > latest SNAPSHOT.
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > The packaged binaries are
> > > > > > > > > > > > > > > > > > > > > > available
> > > > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > > > nightlies.apache.org:
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > If you have some minutes
> > > > > > > > > > > > > > > > > > > > > > left:
> > > > > > > > > > > > > > > > > > > > > > Please
> > > > > > > > > > > > > > > > > > > > > > test and report any
> > > > > > > > > > > > > > > > > > > > > > issues
> > > > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > > > this binaries, so we can fix
> > > > > > > > > > > > > > > > > > > > > > before
> > > > > > > > > > > > > > > > > > > > > > attempting to
> > > > > > > > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > The most significant changes
> > > > > > > > > > > > > > > > > > > > > > are in
> > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > are of
> > > > > > > > > > > > > > > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > > > > upgraded from 2.x to 3.x -
> > > > > > > > > > > > > > > > > > > > > > our
> > > > > > > > > > > > > > > > > > > > > > own
> > > > > > > > > > > > > > > > > > > > > > test
> > > > > > > > > > > > > > > > > > > > > > coverage within
> > > > > > > > > > > > > > > > > > > > > > the build
> > > > > > > > > > > > > > > > > > > > > > looks
> > > > > > > > > > > > > > > > > > > > > > good but would be nice to
> > > > > > > > > > > > > > > > > > > > > > get
> > > > > > > > > > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > > > > > > real world use-case
> > > > > > > > > > > > > > > > > > > > > > feedback
> > > > > > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > > > > this. In addition, we had
> > > > > > > > > > > > > > > > > > > > > > quite
> > > > > > > > > > > > > > > > > > > > > > a lof
> > > > > > > > > > > > > > > > > > > > > > of 3rd party
> > > > > > > > > > > > > > > > > > > > > > dependency
> > > > > > > > > > > > > > > > > > > > > > upgrades.
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > In addition, it contains the
> > > > > > > > > > > > > > > > > > > > > > pruning
> > > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > external modules
> > > > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > > > listed
> > > > > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > A summaryin Jira is here:
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > *Open Source Solutions for Text
> > > > > > > > > > > > > > > > > > Engineering*
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > http://www.digitalpebble.com
> > > > > > > > > > > > > > > > > > http://digitalpebble.blogspot.com/
> > > > > > > > > > > > > > > > > > #digitalpebble
> > > > > > > > > > > > > > > > > > <http://twitter.com/digitalpebble>
> > > > > > > > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> >
>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Richard Zowalla <rz...@apache.org>.
Ok, most of the stuff reported by grype should be fixed now.
I might be able to push a new snapshot within the next week, so we can
do some testing and than go for a release candiate / vote :)
Gruß
Richard
Am Donnerstag, dem 09.11.2023 um 08:53 +0100 schrieb Richard Zowalla:
> Maybe just run grype [1] on the lib directories and post the output
> on
> the list / thread, so we can create tickets for it
>
> There might be some false positives but will give good insights.
>
> Gruß
> Richard
>
>
> [1] https://github.com/anchore/grype
> Am Donnerstag, dem 09.11.2023 um 08:49 +0100 schrieb Alexandre
> Vermeerbergen:
> > Yet another finding on my side to "not call yet for vote on a Storm
> > 2.6.0", we have some CVEs and we shoudn't release a version having
> > known CVEs.
> >
> > For example, we have
> >
> > find . -name "commons-text*jar" -print
> > ./lib-webapp/commons-text-1.9.jar
> > ./external/storm-autocreds/commons-text-1.10.0.jar
> >
> > The second one is okay from
> > https://www.cvedetails.com/cve/CVE-2022-42889/ perspective, but not
> > the first one.
> >
> > I will continue checking if there are other CVEs...
> >
> > Thanks,
> > Alexandre
> >
> > Le mar. 7 nov. 2023 à 17:43, Alexandre Vermeerbergen
> > <av...@gmail.com> a écrit :
> > >
> > > Oh thank you very much for this cherry-pinking, I was too busy
> > > doing
> > > other unrelated stuff..
> > >
> > > Le mar. 7 nov. 2023 à 17:35, Richard Zowalla <rz...@apache.org> a
> > > écrit :
> > > >
> > > > I cherry-picked your commit and put that on master
> > > >
> > > >
> > > > Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard
> > > > Zowalla:
> > > > > You pushed on your own branch. You would need to push to
> > > > > apache/storm
> > > > > main branch or open a PR ;-)
> > > > >
> > > > > Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb Alexandre
> > > > > Vermeerbergen:
> > > > > > Just did a 'git push' with that trivial fix... is that
> > > > > > enough?
> > > > > >
> > > > > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla
> > > > > > <ri...@zowalla.com>
> > > > > > a
> > > > > > écrit :
> > > > > > >
> > > > > > > You can do as you like ;-)
> > > > > > >
> > > > > > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre
> > > > > > > Vermeerbergen
> > > > > > > <av...@gmail.com>:
> > > > > > > > I mean :
> > > > > > > >
> > > > > > > > Can I keep using my
> > > > > > > > 'STORM-3958-Capacity-to-set-St
> > > > > > > > orm-UI's-title-in-conf/storm.yaml'
> > > > > > > > branch to push this fix, or should I create a new one?
> > > > > > > >
> > > > > > > > Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > >
> > > > > > > > > Can you keep using my
> > > > > > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > > > > > > conf/storm.yaml'
> > > > > > > > > branch to push this fix, or should I create a new
> > > > > > > > > one?
> > > > > > > > >
> > > > > > > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > > > > > > <ri...@zowalla.com> a écrit :
> > > > > > > > > >
> > > > > > > > > > Just push a fix for it (referencing the original
> > > > > > > > > > issue) ;-)
> > > > > > > > > > -
> > > > > > > > > > we hadn't a release in between, so not a big deal.
> > > > > > > > > >
> > > > > > > > > > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre
> > > > > > > > > > Vermeerbergen <av...@gmail.com>:
> > > > > > > > > > > Temporary workaround for the Storm UI title's
> > > > > > > > > > > regression:
> > > > > > > > > > > run this
> > > > > > > > > > > command on the title-template.html file (in
> > > > > > > > > > > storm-stable/public/templates directory) after
> > > > > > > > > > > having
> > > > > > > > > > > extracted the
> > > > > > > > > > > binary archive:
> > > > > > > > > > >
> > > > > > > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-
> > > > > > > > > > > template.html
> > > > > > > > > > >
> > > > > > > > > > > Again sorry for having introduced this
> > > > > > > > > > > regression.
> > > > > > > > > > >
> > > > > > > > > > > Alexandre
> > > > > > > > > > >
> > > > > > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre
> > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > >
> > > > > > > > > > > > Updates regarding my test with Storm 2.6.0
> > > > > > > > > > > > Snapshot
> > > > > > > > > > > > 2023/11/02:
> > > > > > > > > > > > 1. I have modified my KO bolts using this
> > > > > > > > > > > > pattern in
> > > > > > > > > > > > their prepare() method:
> > > > > > > > > > > >
> > > > > > > > > > > > long cacheMaxSize =
> > > > > > > > > > > > Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_C
> > > > > > > > > > > > AC
> > > > > > > > > > > > HE_SIZ
> > > > > > > > > > > > E)
> > > > > > > > > > > > .toString());
> > > > > > > > > > > >
> > > > > > > > > > > > instead of:
> > > > > > > > > > > >
> > > > > > > > > > > > long cacheMaxSize = (Long)
> > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > > > > >
> > > > > > > > > > > > 2. I have found the cause of the regression in
> > > > > > > > > > > > loss of
> > > > > > > > > > > > 'Storm UI'
> > > > > > > > > > > > title at the top left-hand side of all Storm
> > > > > > > > > > > > UI's
> > > > > > > > > > > > pages.
> > > > > > > > > > > > This is a mistake of mine in this PR:
> > > > > > > > > > > >
> > > > > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > > > > > > > > > More precisely, at line 19 of this file:
> > > > > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > > > > > > > > > > we must have:
> > > > > > > > > > > > <h1><a href="/">{{title}}</a></h1>
> > > > > > > > > > > >
> > > > > > > > > > > > instead of:
> > > > > > > > > > > >
> > > > > > > > > > > > <h1><a href="/">{{ui.title}}</a></h1>
> > > > > > > > > > > >
> > > > > > > > > > > > I think I missed this during my tests
> > > > > > > > > > > > because
> > > > > > > > > > > > of
> > > > > > > > > > > > browser's cache
> > > > > > > > > > > > (bad excuse, I know...): probably I tested by-
> > > > > > > > > > > > patching
> > > > > > > > > > > > my
> > > > > > > > > > > > deployed
> > > > > > > > > > > > Storm cluster, and I finally forgot to checkout
> > > > > > > > > > > > this
> > > > > > > > > > > > latest change of
> > > > > > > > > > > > my delivery.
> > > > > > > > > > > >
> > > > > > > > > > > > I had to use "private navigation" to make
> > > > > > > > > > > > sure
> > > > > > > > > > > > this
> > > > > > > > > > > > time it fixes this issue
> > > > > > > > > > > >
> > > > > > > > > > > > What would be the quickest way for this very
> > > > > > > > > > > > simple fix
> > > > > > > > > > > > to be delivered ?
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks,
> > > > > > > > > > > > Alexandre
> > > > > > > > > > > >
> > > > > > > > > > > > Le ven. 3 nov. 2023 à 20:57, Alexandre
> > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks Richard for the analysis.
> > > > > > > > > > > > >
> > > > > > > > > > > > > My thought on this are very simple: let me
> > > > > > > > > > > > > fix
> > > > > > > > > > > > > my own
> > > > > > > > > > > > > old code, after
> > > > > > > > > > > > > all this was impacting only one of my
> > > > > > > > > > > > > topologies, the
> > > > > > > > > > > > > other one were
> > > > > > > > > > > > > using more careful Integer.parseInt or
> > > > > > > > > > > > > Integer.parseLong to avoid
> > > > > > > > > > > > > making assumption on the way the values in
> > > > > > > > > > > > > the
> > > > > > > > > > > > > config
> > > > > > > > > > > > > Map were
> > > > > > > > > > > > > serialized.
> > > > > > > > > > > > >
> > > > > > > > > > > > > My other concern with 2.6.0 Snapshot is that
> > > > > > > > > > > > > in
> > > > > > > > > > > > > Storm
> > > > > > > > > > > > > UI, the "Storm
> > > > > > > > > > > > > UI" title at the top left hand side of each
> > > > > > > > > > > > > page is
> > > > > > > > > > > > > missing. This may
> > > > > > > > > > > > > be the result of one of my own PR, I'll have
> > > > > > > > > > > > > to
> > > > > > > > > > > > > check
> > > > > > > > > > > > > why...
> > > > > > > > > > > > >
> > > > > > > > > > > > > Alex
> > > > > > > > > > > > >
> > > > > > > > > > > > > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla
> > > > > > > > > > > > > <rz...@apache.org> a écrit :
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > thanks for the feedback.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The main difference to 2.5.0 is, that we
> > > > > > > > > > > > > > switched
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > JSON
> > > > > > > > > > > > > > implementation in Utils, which relied on a
> > > > > > > > > > > > > > super old JSON library. What is happening
> > > > > > > > > > > > > > here is,
> > > > > > > > > > > > > > that the new JSON
> > > > > > > > > > > > > > library (net.minidev) behaves differently.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > We can see it by writing some simple code
> > > > > > > > > > > > > > without
> > > > > > > > > > > > > > Storm (The related
> > > > > > > > > > > > > > code line with the parsing is taken from
> > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > Utils)
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Map<String, Long> conf = new
> > > > > > > > > > > > > > HashMap<>();
> > > > > > > > > > > > > > conf.put("test", 100000000L);
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Map<String, Object> deserTopoConf =
> > > > > > > > > > > > > > (Map<String,
> > > > > > > > > > > > > > Object>)
> > > > > > > > > > > > > > JSONValue.parseWithException(JSONValue.toJS
> > > > > > > > > > > > > > ON
> > > > > > > > > > > > > > String
> > > > > > > > > > > > > > (c
> > > > > > > > > > > > > > onf));
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > System.out.println(e.getValue().getClass().
> > > > > > > > > > > > > > ge
> > > > > > > > > > > > > > tSimpl
> > > > > > > > > > > > > > eN
> > > > > > > > > > > > > > ame());
> > > > > > > > > > > > > > }
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > This will print "Integer". Using
> > > > > > > > > > > > > > "10000000000000000L"
> > > > > > > > > > > > > > as a value will
> > > > > > > > > > > > > > result in "Long".
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The main issue derives from the fact, that
> > > > > > > > > > > > > > we
> > > > > > > > > > > > > > serialize the config map
> > > > > > > > > > > > > > into a JSON String and parse the result.
> > > > > > > > > > > > > > Based on the String representation,
> > > > > > > > > > > > > > "100000000L"
> > > > > > > > > > > > > > will
> > > > > > > > > > > > > > still fit into an
> > > > > > > > > > > > > > Integer, so the parser will not use Long ;-
> > > > > > > > > > > > > > )
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > If the number is big enough, the parser
> > > > > > > > > > > > > > will
> > > > > > > > > > > > > > switch
> > > > > > > > > > > > > > to a Long> >> > >> > > The old library (>
> > > > > > > > > > > > > > 10
> > > > > > > > > > > > > > years
> > > > > > > > > > > > > > old now):
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > deserTopoConf = (Map<String, Object>)
> > > > > > > > > > > > > > org.json.simple.JSONValue.parseWithExceptio
> > > > > > > > > > > > > > n(
> > > > > > > > > > > > > > org.js
> > > > > > > > > > > > > > on
> > > > > > > > > > > > > > .simple.JSONValue.
> > > > > > > > > > > > > > toJSONString(conf));
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > System.out.println(e.getValue().getClass().
> > > > > > > > > > > > > > ge
> > > > > > > > > > > > > > tSimpl
> > > > > > > > > > > > > > eN
> > > > > > > > > > > > > > ame());
> > > > > > > > > > > > > > }
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > This will print "Long".
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The difference between these libraries is,
> > > > > > > > > > > > > > that the
> > > > > > > > > > > > > > super old
> > > > > > > > > > > > > > org.json.simple just always returns Long as
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > type.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > If can eventually tune the new JSON parser
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > something like:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > deserTopoConf = (Map<String, Object>) new
> > > > > > > > > > > > > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTE
> > > > > > > > > > > > > > D)
> > > > > > > > > > > > > > .parse
> > > > > > > > > > > > > > (J
> > > > > > > > > > > > > > SONValue= toJSONStr
> > > > > > > > > > > > > > ing(conf),
> > > > > > > > > > > > > > JSONValue.defaultReader.DEFAULT);
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > This will result in "Long" but might have
> > > > > > > > > > > > > > other
> > > > > > > > > > > > > > side-
> > > > > > > > > > > > > > effects.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > From my point of view, the newer JSON
> > > > > > > > > > > > > > library
> > > > > > > > > > > > > > works
> > > > > > > > > > > > > > as expected
> > > > > > > > > > > > > > (although it might break backward
> > > > > > > > > > > > > > compatibility).
> > > > > > > > > > > > > > Unchecked casts are always evil, imho. In
> > > > > > > > > > > > > > your
> > > > > > > > > > > > > > specific case, it might
> > > > > > > > > > > > > > be more save to cast to "Number" and than
> > > > > > > > > > > > > > invoke
> > > > > > > > > > > > > > "longValue()" instead.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Nevertheless, we might being able to
> > > > > > > > > > > > > > restore
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > old
> > > > > > > > > > > > > > behaviour by
> > > > > > > > > > > > > > tuning the JSONParser configuration.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Thoughts?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Am Freitag, dem 03.11.2023 um 18:32 +0100
> > > > > > > > > > > > > > schrieb
> > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > On a side note, the Long value that
> > > > > > > > > > > > > > > "MyKOBolt" is
> > > > > > > > > > > > > > > set by a
> > > > > > > > > > > > > > > DefaultTopologyConfig to a value with is
> > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > java.lang.Long instance...
> > > > > > > > > > > > > > > hence it's surprising to see it
> > > > > > > > > > > > > > > internally
> > > > > > > > > > > > > > > converted into a
> > > > > > > > > > > > > > > java.lang.Integer in the Map passed to
> > > > > > > > > > > > > > > prepare()
> > > > > > > > > > > > > > > method...
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 18:27, Alexandre
> > > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Hello,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I found a breaking change using Apache
> > > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > > 2.6.0 snapshot build
> > > > > > > > > > > > > > > > dated 20231103 and without changing my
> > > > > > > > > > > > > > > > topologies' code (keeping
> > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > same big jars).
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > This seems to be tied to a datatype
> > > > > > > > > > > > > > > > change in
> > > > > > > > > > > > > > > > Map
> > > > > > > > > > > > > > > > argument of
> > > > > > > > > > > > > > > > prepare() method of Bolt.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > For example, this Bolt is KO at
> > > > > > > > > > > > > > > > runtime:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > public class MyKOBolt extends
> > > > > > > > > > > > > > > > BaseRichBolt {
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > > public void prepare(Map stormConf,
> > > > > > > > > > > > > > > > TopologyContext context,
> > > > > > > > > > > > > > > > OutputCollector collector) {
> > > > > > > > > > > > > > > > cacheMaxSize = (Long)
> > > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SI
> > > > > > > > > > > > > > > > ZE
> > > > > > > > > > > > > > > > );
> > > > > > > > > > > > > > > > cacheTtlMin = (Long)
> > > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TT
> > > > > > > > > > > > > > > > L_
> > > > > > > > > > > > > > > > MIN);
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > as it throws this error:
> > > > > > > > > > > > > > > > 2023-11-03 16:42:20.230
> > > > > > > > > > > > > > > > o.a.s.e.b.BoltExecutor
> > > > > > > > > > > > > > > > Thread-14-__system-executor[-1, -1]
> > > > > > > > > > > > > > > > [INFO]
> > > > > > > > > > > > > > > > Preparing bolt
> > > > > > > > > > > > > > > > __system:[-1]
> > > > > > > > > > > > > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > > > > > > > > > > > > > > Thread-15-evaluateTriggers-executor[11,
> > > > > > > > > > > > > > > > 11]
> > > > > > > > > > > > > > > > [ERROR] Async loop
> > > > > > > > > > > > > > > > died!
> > > > > > > > > > > > > > > > java.lang.ClassCastException:
> > > > > > > > > > > > > > > > java.lang.Integer
> > > > > > > > > > > > > > > > incompatible with
> > > > > > > > > > > > > > > > java.lang.Long
> > > > > > > > > > > > > > > > at com.acme.storm.alerting.MyKOBolt
> > > > > > > > > > > > > > > > .prepare(EvaluationBolt.java:71)
> > > > > > > > > > > > > > > > ~[stormjar.jar:?]
> > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExec
> > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > or.ini
> > > > > > > > > > > > > > > > t(
> > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > 128)
> > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExec
> > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > or.cal
> > > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > 138)
> > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExec
> > > > > > > > > > > > > > > > ut
> > > > > > > > > > > > > > > > or.cal
> > > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > > 54)
> > > > > > > > > > > > > > > > ~[storm-client-2.6.0-
> > > > > > > > > > > > > > > > SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > org.apache.storm.utils.Utils$1.run(Util
> > > > > > > > > > > > > > > > s.
> > > > > > > > > > > > > > > > java:3
> > > > > > > > > > > > > > > > 93
> > > > > > > > > > > > > > > > )
> > > > > > > > > > > > > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > java.lang.Thread.run(Thread.java:857)
> > > > > > > > > > > > > > > > [?:?]
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > whereas this bolt is fine with Storm
> > > > > > > > > > > > > > > > 2.6.0:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > public class MyOKBolt extends
> > > > > > > > > > > > > > > > BaseBasicBolt {
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > > public void prepare(final Map
> > > > > > > > > > > > > > > > stormConf,
> > > > > > > > > > > > > > > > final TopologyContext
> > > > > > > > > > > > > > > > context) {
> > > > > > > > > > > > > > > > super.prepare(stormConf,
> > > > > > > > > > > > > > > > context);
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > this.redisPort
> > > > > > > > > > > > > > > > =
> > > > > > > > > > > > > > > > Integer.parseInt((String)
> > > > > > > > > > > > > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > That said, I can modify the code of
> > > > > > > > > > > > > > > > "MyKOBolt"
> > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > use same
> > > > > > > > > > > > > > > > Integer.parseInt or Integer.parseLong
> > > > > > > > > > > > > > > > trick,
> > > > > > > > > > > > > > > > but
> > > > > > > > > > > > > > > > it's the first
> > > > > > > > > > > > > > > > time
> > > > > > > > > > > > > > > > in my long history of upgrades that I
> > > > > > > > > > > > > > > > have seen
> > > > > > > > > > > > > > > > such runtime
> > > > > > > > > > > > > > > > incompatibility.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Maybe "MyKOBolt" was badly written
> > > > > > > > > > > > > > > > since
> > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > beginning and I have
> > > > > > > > > > > > > > > > just
> > > > > > > > > > > > > > > > hit the punishment for it: is there a
> > > > > > > > > > > > > > > > documentation which clarifies
> > > > > > > > > > > > > > > > datatypes of the Map argument of
> > > > > > > > > > > > > > > > prepare()
> > > > > > > > > > > > > > > > method?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Also worth noting: "MyKOBolt" derives
> > > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > BaseRichBolt , while
> > > > > > > > > > > > > > > > "MyOKBolt" derives from BaseBasicBolt
> > > > > > > > > > > > > > > > =>
> > > > > > > > > > > > > > > > could
> > > > > > > > > > > > > > > > this have any impact
> > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > this finding ?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Last, I'm running this on Redhat Linux
> > > > > > > > > > > > > > > > 8
> > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > IBM
> > > > > > > > > > > > > > > > Semeru JDK
> > > > > > > > > > > > > > > > 17.0.8.1.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 15:16, Julien
> > > > > > > > > > > > > > > > Nioche
> > > > > > > > > > > > > > > > <li...@gmail.com> a écrit
> > > > > > > > > > > > > > > > :
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Thanks Richard.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Tried the latest snapshot with
> > > > > > > > > > > > > > > > > StormCrawler
> > > > > > > > > > > > > > > > > both in local and
> > > > > > > > > > > > > > > > > deployed mode
> > > > > > > > > > > > > > > > > and did not find any issues.
> > > > > > > > > > > > > > > > > Will try it on a topology generating
> > > > > > > > > > > > > > > > > WARC
> > > > > > > > > > > > > > > > > files
> > > > > > > > > > > > > > > > > next week to
> > > > > > > > > > > > > > > > > check that the
> > > > > > > > > > > > > > > > > dependency changes on Hadoop have not
> > > > > > > > > > > > > > > > > broken
> > > > > > > > > > > > > > > > > anything.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Have a good week end
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Julien
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > On Thu, 2 Nov 2023 at 19:25, Richard
> > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > <rz...@apache.org>
> > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Short update. Sorted out an issue
> > > > > > > > > > > > > > > > > > with the
> > > > > > > > > > > > > > > > > > tar.gz/zip with
> > > > > > > > > > > > > > > > > > Julien today
> > > > > > > > > > > > > > > > > > and re-uploaded them to the
> > > > > > > > > > > > > > > > > > nightlies
> > > > > > > > > > > > > > > > > > area.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > This new bundle works as expected
> > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > > > deployment but happy to
> > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > additional feedback before getting
> > > > > > > > > > > > > > > > > > up
> > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > first
> > > > > > > > > > > > > > > > > > release candidate
> > > > > > > > > > > > > > > > > > ;-)
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 08:21
> > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > schrieb Richard
> > > > > > > > > > > > > > > > > > Zowalla:
> > > > > > > > > > > > > > > > > > > Hi Alexandre,
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > we are not in a hurry here :) -
> > > > > > > > > > > > > > > > > > > take as
> > > > > > > > > > > > > > > > > > > much as time you
> > > > > > > > > > > > > > > > > > > need.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um
> > > > > > > > > > > > > > > > > > > 07:50
> > > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > > schrieb Alexandre
> > > > > > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > > > > > Hello Richard,
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Okay, I'm more than happy to do
> > > > > > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > my pre-production
> > > > > > > > > > > > > > > > > > > > cluster
> > > > > > > > > > > > > > > > > > > > (~10 topologies) using the
> > > > > > > > > > > > > > > > > > > > binary
> > > > > > > > > > > > > > > > > > > > artifacts.
> > > > > > > > > > > > > > > > > > > > Would it be OK if I can use up
> > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > end
> > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > this week so that
> > > > > > > > > > > > > > > > > > > > I'll be
> > > > > > > > > > > > > > > > > > > > able to have enough time to
> > > > > > > > > > > > > > > > > > > > check
> > > > > > > > > > > > > > > > > > > > all
> > > > > > > > > > > > > > > > > > > > potential issues that
> > > > > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > > > > upgrade bring?
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Le lun. 30 oct. 2023 à 07:35,
> > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > > > <rz...@apache.org> a
> > > > > > > > > > > > > > > > > > > > écrit :
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > before starting a release of
> > > > > > > > > > > > > > > > > > > > > 2.6.0-
> > > > > > > > > > > > > > > > > > > > > SNAPSHOT, I would like
> > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > > > > some feedback on the current
> > > > > > > > > > > > > > > > > > > > > SNAPSHOT
> > > > > > > > > > > > > > > > > > > > > build.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > I just uploaded a 2.6.0-
> > > > > > > > > > > > > > > > > > > > > SNAPSHOT of
> > > > > > > > > > > > > > > > > > > > > Storm build from
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > It is available via the ASF
> > > > > > > > > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > > > > > > > > repository:
> > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > You can consume by adding
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > <repository>
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > <id>apache.snapshots</id>
> > > > > > > > > > > > > > > > > > > > > <name>Apache Snapshot
> > > > > > > > > > > > > > > > > > > > > Repository</name>
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > <url>
> > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/snap
> > > > > > > > > > > > > > > > > > > > > shots</url>
> > > > > > > > > > > > > > > > > > > > > <releases>
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > <enabled>false</enabled>
> > > > > > > > > > > > > > > > > > > > > </releases>
> > > > > > > > > > > > > > > > > > > > > </repository>
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > to your project pom. As we do
> > > > > > > > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > > > > > > deploy SNAPSHOTS
> > > > > > > > > > > > > > > > > > > > > automatically,
> > > > > > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > > > > should be easy to just
> > > > > > > > > > > > > > > > > > > > > consume
> > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > latest SNAPSHOT.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > The packaged binaries are
> > > > > > > > > > > > > > > > > > > > > available
> > > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > > nightlies.apache.org:
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > If you have some minutes
> > > > > > > > > > > > > > > > > > > > > left:
> > > > > > > > > > > > > > > > > > > > > Please
> > > > > > > > > > > > > > > > > > > > > test and report any
> > > > > > > > > > > > > > > > > > > > > issues
> > > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > > this binaries, so we can fix
> > > > > > > > > > > > > > > > > > > > > before
> > > > > > > > > > > > > > > > > > > > > attempting to
> > > > > > > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > The most significant changes
> > > > > > > > > > > > > > > > > > > > > are in
> > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > are of
> > > > > > > > > > > > > > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > > > upgraded from 2.x to 3.x -
> > > > > > > > > > > > > > > > > > > > > our
> > > > > > > > > > > > > > > > > > > > > own
> > > > > > > > > > > > > > > > > > > > > test
> > > > > > > > > > > > > > > > > > > > > coverage within
> > > > > > > > > > > > > > > > > > > > > the build
> > > > > > > > > > > > > > > > > > > > > looks
> > > > > > > > > > > > > > > > > > > > > good but would be nice to
> > > > > > > > > > > > > > > > > > > > > get
> > > > > > > > > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > > > > > real world use-case
> > > > > > > > > > > > > > > > > > > > > feedback
> > > > > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > > > this. In addition, we had
> > > > > > > > > > > > > > > > > > > > > quite
> > > > > > > > > > > > > > > > > > > > > a lof
> > > > > > > > > > > > > > > > > > > > > of 3rd party
> > > > > > > > > > > > > > > > > > > > > dependency
> > > > > > > > > > > > > > > > > > > > > upgrades.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > In addition, it contains the
> > > > > > > > > > > > > > > > > > > > > pruning
> > > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > external modules
> > > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > > listed
> > > > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > A summaryin Jira is here:
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > *Open Source Solutions for Text
> > > > > > > > > > > > > > > > > Engineering*
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > http://www.digitalpebble.com
> > > > > > > > > > > > > > > > > http://digitalpebble.blogspot.com/
> > > > > > > > > > > > > > > > > #digitalpebble
> > > > > > > > > > > > > > > > > <http://twitter.com/digitalpebble>
> > > > > > > > > > > > > >
> > > > > >
> > > > >
> > > >
>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Richard Zowalla <rz...@apache.org>.
Maybe just run grype [1] on the lib directories and post the output on
the list / thread, so we can create tickets for it
There might be some false positives but will give good insights.
Gruß
Richard
[1] https://github.com/anchore/grype
Am Donnerstag, dem 09.11.2023 um 08:49 +0100 schrieb Alexandre
Vermeerbergen:
> Yet another finding on my side to "not call yet for vote on a Storm
> 2.6.0", we have some CVEs and we shoudn't release a version having
> known CVEs.
>
> For example, we have
>
> find . -name "commons-text*jar" -print
> ./lib-webapp/commons-text-1.9.jar
> ./external/storm-autocreds/commons-text-1.10.0.jar
>
> The second one is okay from
> https://www.cvedetails.com/cve/CVE-2022-42889/ perspective, but not
> the first one.
>
> I will continue checking if there are other CVEs...
>
> Thanks,
> Alexandre
>
> Le mar. 7 nov. 2023 à 17:43, Alexandre Vermeerbergen
> <av...@gmail.com> a écrit :
> >
> > Oh thank you very much for this cherry-pinking, I was too busy
> > doing
> > other unrelated stuff..
> >
> > Le mar. 7 nov. 2023 à 17:35, Richard Zowalla <rz...@apache.org> a
> > écrit :
> > >
> > > I cherry-picked your commit and put that on master
> > >
> > >
> > > Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard
> > > Zowalla:
> > > > You pushed on your own branch. You would need to push to
> > > > apache/storm
> > > > main branch or open a PR ;-)
> > > >
> > > > Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb Alexandre
> > > > Vermeerbergen:
> > > > > Just did a 'git push' with that trivial fix... is that
> > > > > enough?
> > > > >
> > > > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla
> > > > > <ri...@zowalla.com>
> > > > > a
> > > > > écrit :
> > > > > >
> > > > > > You can do as you like ;-)
> > > > > >
> > > > > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre
> > > > > > Vermeerbergen
> > > > > > <av...@gmail.com>:
> > > > > > > I mean :
> > > > > > >
> > > > > > > Can I keep using my
> > > > > > > 'STORM-3958-Capacity-to-set-St
> > > > > > > orm-UI's-title-in-conf/storm.yaml'
> > > > > > > branch to push this fix, or should I create a new one?
> > > > > > >
> > > > > > > Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > > > > > <av...@gmail.com> a écrit :
> > > > > > > >
> > > > > > > > Can you keep using my
> > > > > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > > > > > conf/storm.yaml'
> > > > > > > > branch to push this fix, or should I create a new one?
> > > > > > > >
> > > > > > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > > > > > <ri...@zowalla.com> a écrit :
> > > > > > > > >
> > > > > > > > > Just push a fix for it (referencing the original
> > > > > > > > > issue) ;-)
> > > > > > > > > -
> > > > > > > > > we hadn't a release in between, so not a big deal.
> > > > > > > > >
> > > > > > > > > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre
> > > > > > > > > Vermeerbergen <av...@gmail.com>:
> > > > > > > > > > Temporary workaround for the Storm UI title's
> > > > > > > > > > regression:
> > > > > > > > > > run this
> > > > > > > > > > command on the title-template.html file (in
> > > > > > > > > > storm-stable/public/templates directory) after
> > > > > > > > > > having
> > > > > > > > > > extracted the
> > > > > > > > > > binary archive:
> > > > > > > > > >
> > > > > > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-
> > > > > > > > > > template.html
> > > > > > > > > >
> > > > > > > > > > Again sorry for having introduced this regression.
> > > > > > > > > >
> > > > > > > > > > Alexandre
> > > > > > > > > >
> > > > > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre
> > > > > > > > > > Vermeerbergen
> > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > >
> > > > > > > > > > > Updates regarding my test with Storm 2.6.0
> > > > > > > > > > > Snapshot
> > > > > > > > > > > 2023/11/02:
> > > > > > > > > > > 1. I have modified my KO bolts using this
> > > > > > > > > > > pattern in
> > > > > > > > > > > their prepare() method:
> > > > > > > > > > >
> > > > > > > > > > > long cacheMaxSize =
> > > > > > > > > > > Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CAC
> > > > > > > > > > > HE_SIZ
> > > > > > > > > > > E)
> > > > > > > > > > > .toString());
> > > > > > > > > > >
> > > > > > > > > > > instead of:
> > > > > > > > > > >
> > > > > > > > > > > long cacheMaxSize = (Long)
> > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > > > >
> > > > > > > > > > > 2. I have found the cause of the regression in
> > > > > > > > > > > loss of
> > > > > > > > > > > 'Storm UI'
> > > > > > > > > > > title at the top left-hand side of all Storm UI's
> > > > > > > > > > > pages.
> > > > > > > > > > > This is a mistake of mine in this PR:
> > > > > > > > > > >
> > > > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > > > > > > > > More precisely, at line 19 of this file:
> > > > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > > > > > > > > > we must have:
> > > > > > > > > > > <h1><a href="/">{{title}}</a></h1>
> > > > > > > > > > >
> > > > > > > > > > > instead of:
> > > > > > > > > > >
> > > > > > > > > > > <h1><a href="/">{{ui.title}}</a></h1>
> > > > > > > > > > >
> > > > > > > > > > > I think I missed this during my tests because
> > > > > > > > > > > of
> > > > > > > > > > > browser's cache
> > > > > > > > > > > (bad excuse, I know...): probably I tested by-
> > > > > > > > > > > patching
> > > > > > > > > > > my
> > > > > > > > > > > deployed
> > > > > > > > > > > Storm cluster, and I finally forgot to checkout
> > > > > > > > > > > this
> > > > > > > > > > > latest change of
> > > > > > > > > > > my delivery.
> > > > > > > > > > >
> > > > > > > > > > > I had to use "private navigation" to make sure
> > > > > > > > > > > this
> > > > > > > > > > > time it fixes this issue
> > > > > > > > > > >
> > > > > > > > > > > What would be the quickest way for this very
> > > > > > > > > > > simple fix
> > > > > > > > > > > to be delivered ?
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > Alexandre
> > > > > > > > > > >
> > > > > > > > > > > Le ven. 3 nov. 2023 à 20:57, Alexandre
> > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks Richard for the analysis.
> > > > > > > > > > > >
> > > > > > > > > > > > My thought on this are very simple: let me fix
> > > > > > > > > > > > my own
> > > > > > > > > > > > old code, after
> > > > > > > > > > > > all this was impacting only one of my
> > > > > > > > > > > > topologies, the
> > > > > > > > > > > > other one were
> > > > > > > > > > > > using more careful Integer.parseInt or
> > > > > > > > > > > > Integer.parseLong to avoid
> > > > > > > > > > > > making assumption on the way the values in the
> > > > > > > > > > > > config
> > > > > > > > > > > > Map were
> > > > > > > > > > > > serialized.
> > > > > > > > > > > >
> > > > > > > > > > > > My other concern with 2.6.0 Snapshot is that in
> > > > > > > > > > > > Storm
> > > > > > > > > > > > UI, the "Storm
> > > > > > > > > > > > UI" title at the top left hand side of each
> > > > > > > > > > > > page is
> > > > > > > > > > > > missing. This may
> > > > > > > > > > > > be the result of one of my own PR, I'll have to
> > > > > > > > > > > > check
> > > > > > > > > > > > why...
> > > > > > > > > > > >
> > > > > > > > > > > > Alex
> > > > > > > > > > > >
> > > > > > > > > > > > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla
> > > > > > > > > > > > <rz...@apache.org> a écrit :
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hi,
> > > > > > > > > > > > >
> > > > > > > > > > > > > thanks for the feedback.
> > > > > > > > > > > > >
> > > > > > > > > > > > > The main difference to 2.5.0 is, that we
> > > > > > > > > > > > > switched
> > > > > > > > > > > > > the
> > > > > > > > > > > > > JSON
> > > > > > > > > > > > > implementation in Utils, which relied on a
> > > > > > > > > > > > > super old JSON library. What is happening
> > > > > > > > > > > > > here is,
> > > > > > > > > > > > > that the new JSON
> > > > > > > > > > > > > library (net.minidev) behaves differently.
> > > > > > > > > > > > >
> > > > > > > > > > > > > We can see it by writing some simple code
> > > > > > > > > > > > > without
> > > > > > > > > > > > > Storm (The related
> > > > > > > > > > > > > code line with the parsing is taken from
> > > > > > > > > > > > > Storm
> > > > > > > > > > > > > Utils)
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Map<String, Long> conf = new HashMap<>();
> > > > > > > > > > > > > conf.put("test", 100000000L);
> > > > > > > > > > > > >
> > > > > > > > > > > > > Map<String, Object> deserTopoConf =
> > > > > > > > > > > > > (Map<String,
> > > > > > > > > > > > > Object>)
> > > > > > > > > > > > > JSONValue.parseWithException(JSONValue.toJSON
> > > > > > > > > > > > > String
> > > > > > > > > > > > > (c
> > > > > > > > > > > > > onf));
> > > > > > > > > > > > >
> > > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > >
> > > > > > > > > > > > > System.out.println(e.getValue().getClass().ge
> > > > > > > > > > > > > tSimpl
> > > > > > > > > > > > > eN
> > > > > > > > > > > > > ame());
> > > > > > > > > > > > > }
> > > > > > > > > > > > >
> > > > > > > > > > > > > This will print "Integer". Using
> > > > > > > > > > > > > "10000000000000000L"
> > > > > > > > > > > > > as a value will
> > > > > > > > > > > > > result in "Long".
> > > > > > > > > > > > >
> > > > > > > > > > > > > The main issue derives from the fact, that we
> > > > > > > > > > > > > serialize the config map
> > > > > > > > > > > > > into a JSON String and parse the result.
> > > > > > > > > > > > > Based on the String representation,
> > > > > > > > > > > > > "100000000L"
> > > > > > > > > > > > > will
> > > > > > > > > > > > > still fit into an
> > > > > > > > > > > > > Integer, so the parser will not use Long ;-)
> > > > > > > > > > > > >
> > > > > > > > > > > > > If the number is big enough, the parser will
> > > > > > > > > > > > > switch
> > > > > > > > > > > > > to a Long> >> > >> > > The old library (> 10
> > > > > > > > > > > > > years
> > > > > > > > > > > > > old now):
> > > > > > > > > > > > >
> > > > > > > > > > > > > deserTopoConf = (Map<String, Object>)
> > > > > > > > > > > > > org.json.simple.JSONValue.parseWithException(
> > > > > > > > > > > > > org.js
> > > > > > > > > > > > > on
> > > > > > > > > > > > > .simple.JSONValue.
> > > > > > > > > > > > > toJSONString(conf));
> > > > > > > > > > > > >
> > > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > > >
> > > > > > > > > > > > > System.out.println(e.getValue().getClass().ge
> > > > > > > > > > > > > tSimpl
> > > > > > > > > > > > > eN
> > > > > > > > > > > > > ame());
> > > > > > > > > > > > > }
> > > > > > > > > > > > >
> > > > > > > > > > > > > This will print "Long".
> > > > > > > > > > > > >
> > > > > > > > > > > > > The difference between these libraries is,
> > > > > > > > > > > > > that the
> > > > > > > > > > > > > super old
> > > > > > > > > > > > > org.json.simple just always returns Long as
> > > > > > > > > > > > > the
> > > > > > > > > > > > > type.
> > > > > > > > > > > > >
> > > > > > > > > > > > > If can eventually tune the new JSON parser to
> > > > > > > > > > > > > something like:
> > > > > > > > > > > > >
> > > > > > > > > > > > > deserTopoConf = (Map<String, Object>) new
> > > > > > > > > > > > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED)
> > > > > > > > > > > > > .parse
> > > > > > > > > > > > > (J
> > > > > > > > > > > > > SONValue= toJSONStr
> > > > > > > > > > > > > ing(conf), JSONValue.defaultReader.DEFAULT);
> > > > > > > > > > > > >
> > > > > > > > > > > > > This will result in "Long" but might have
> > > > > > > > > > > > > other
> > > > > > > > > > > > > side-
> > > > > > > > > > > > > effects.
> > > > > > > > > > > > >
> > > > > > > > > > > > > From my point of view, the newer JSON library
> > > > > > > > > > > > > works
> > > > > > > > > > > > > as expected
> > > > > > > > > > > > > (although it might break backward
> > > > > > > > > > > > > compatibility).
> > > > > > > > > > > > > Unchecked casts are always evil, imho. In
> > > > > > > > > > > > > your
> > > > > > > > > > > > > specific case, it might
> > > > > > > > > > > > > be more save to cast to "Number" and than
> > > > > > > > > > > > > invoke
> > > > > > > > > > > > > "longValue()" instead.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Nevertheless, we might being able to restore
> > > > > > > > > > > > > the
> > > > > > > > > > > > > old
> > > > > > > > > > > > > behaviour by
> > > > > > > > > > > > > tuning the JSONParser configuration.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thoughts?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > Richard
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Am Freitag, dem 03.11.2023 um 18:32 +0100
> > > > > > > > > > > > > schrieb
> > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > On a side note, the Long value that
> > > > > > > > > > > > > > "MyKOBolt" is
> > > > > > > > > > > > > > set by a
> > > > > > > > > > > > > > DefaultTopologyConfig to a value with is a
> > > > > > > > > > > > > > java.lang.Long instance...
> > > > > > > > > > > > > > hence it's surprising to see it internally
> > > > > > > > > > > > > > converted into a
> > > > > > > > > > > > > > java.lang.Integer in the Map passed to
> > > > > > > > > > > > > > prepare()
> > > > > > > > > > > > > > method...
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 18:27, Alexandre
> > > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Hello,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I found a breaking change using Apache
> > > > > > > > > > > > > > > Storm
> > > > > > > > > > > > > > > 2.6.0 snapshot build
> > > > > > > > > > > > > > > dated 20231103 and without changing my
> > > > > > > > > > > > > > > topologies' code (keeping
> > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > same big jars).
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > This seems to be tied to a datatype
> > > > > > > > > > > > > > > change in
> > > > > > > > > > > > > > > Map
> > > > > > > > > > > > > > > argument of
> > > > > > > > > > > > > > > prepare() method of Bolt.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > For example, this Bolt is KO at runtime:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > public class MyKOBolt extends
> > > > > > > > > > > > > > > BaseRichBolt {
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > public void prepare(Map stormConf,
> > > > > > > > > > > > > > > TopologyContext context,
> > > > > > > > > > > > > > > OutputCollector collector) {
> > > > > > > > > > > > > > > cacheMaxSize = (Long)
> > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE
> > > > > > > > > > > > > > > );
> > > > > > > > > > > > > > > cacheTtlMin = (Long)
> > > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_
> > > > > > > > > > > > > > > MIN);
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > as it throws this error:
> > > > > > > > > > > > > > > 2023-11-03 16:42:20.230
> > > > > > > > > > > > > > > o.a.s.e.b.BoltExecutor
> > > > > > > > > > > > > > > Thread-14-__system-executor[-1, -1]
> > > > > > > > > > > > > > > [INFO]
> > > > > > > > > > > > > > > Preparing bolt
> > > > > > > > > > > > > > > __system:[-1]
> > > > > > > > > > > > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > > > > > > > > > > > > > Thread-15-evaluateTriggers-executor[11,
> > > > > > > > > > > > > > > 11]
> > > > > > > > > > > > > > > [ERROR] Async loop
> > > > > > > > > > > > > > > died!
> > > > > > > > > > > > > > > java.lang.ClassCastException:
> > > > > > > > > > > > > > > java.lang.Integer
> > > > > > > > > > > > > > > incompatible with
> > > > > > > > > > > > > > > java.lang.Long
> > > > > > > > > > > > > > > at com.acme.storm.alerting.MyKOBolt
> > > > > > > > > > > > > > > .prepare(EvaluationBolt.java:71)
> > > > > > > > > > > > > > > ~[stormjar.jar:?]
> > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecut
> > > > > > > > > > > > > > > or.ini
> > > > > > > > > > > > > > > t(
> > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > 128)
> > > > > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecut
> > > > > > > > > > > > > > > or.cal
> > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > 138)
> > > > > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecut
> > > > > > > > > > > > > > > or.cal
> > > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > > 54)
> > > > > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > org.apache.storm.utils.Utils$1.run(Utils.
> > > > > > > > > > > > > > > java:3
> > > > > > > > > > > > > > > 93
> > > > > > > > > > > > > > > )
> > > > > > > > > > > > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > > at java.lang.Thread.run(Thread.java:857)
> > > > > > > > > > > > > > > [?:?]
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > whereas this bolt is fine with Storm
> > > > > > > > > > > > > > > 2.6.0:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > public class MyOKBolt extends
> > > > > > > > > > > > > > > BaseBasicBolt {
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > > public void prepare(final Map
> > > > > > > > > > > > > > > stormConf,
> > > > > > > > > > > > > > > final TopologyContext
> > > > > > > > > > > > > > > context) {
> > > > > > > > > > > > > > > super.prepare(stormConf,
> > > > > > > > > > > > > > > context);
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > this.redisPort
> > > > > > > > > > > > > > > =
> > > > > > > > > > > > > > > Integer.parseInt((String)
> > > > > > > > > > > > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > That said, I can modify the code of
> > > > > > > > > > > > > > > "MyKOBolt"
> > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > use same
> > > > > > > > > > > > > > > Integer.parseInt or Integer.parseLong
> > > > > > > > > > > > > > > trick,
> > > > > > > > > > > > > > > but
> > > > > > > > > > > > > > > it's the first
> > > > > > > > > > > > > > > time
> > > > > > > > > > > > > > > in my long history of upgrades that I
> > > > > > > > > > > > > > > have seen
> > > > > > > > > > > > > > > such runtime
> > > > > > > > > > > > > > > incompatibility.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Maybe "MyKOBolt" was badly written since
> > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > beginning and I have
> > > > > > > > > > > > > > > just
> > > > > > > > > > > > > > > hit the punishment for it: is there a
> > > > > > > > > > > > > > > documentation which clarifies
> > > > > > > > > > > > > > > datatypes of the Map argument of
> > > > > > > > > > > > > > > prepare()
> > > > > > > > > > > > > > > method?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Also worth noting: "MyKOBolt" derives
> > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > BaseRichBolt , while
> > > > > > > > > > > > > > > "MyOKBolt" derives from BaseBasicBolt =>
> > > > > > > > > > > > > > > could
> > > > > > > > > > > > > > > this have any impact
> > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > this finding ?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Last, I'm running this on Redhat Linux 8
> > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > IBM
> > > > > > > > > > > > > > > Semeru JDK
> > > > > > > > > > > > > > > 17.0.8.1.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 15:16, Julien
> > > > > > > > > > > > > > > Nioche
> > > > > > > > > > > > > > > <li...@gmail.com> a écrit :
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Thanks Richard.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Tried the latest snapshot with
> > > > > > > > > > > > > > > > StormCrawler
> > > > > > > > > > > > > > > > both in local and
> > > > > > > > > > > > > > > > deployed mode
> > > > > > > > > > > > > > > > and did not find any issues.
> > > > > > > > > > > > > > > > Will try it on a topology generating
> > > > > > > > > > > > > > > > WARC
> > > > > > > > > > > > > > > > files
> > > > > > > > > > > > > > > > next week to
> > > > > > > > > > > > > > > > check that the
> > > > > > > > > > > > > > > > dependency changes on Hadoop have not
> > > > > > > > > > > > > > > > broken
> > > > > > > > > > > > > > > > anything.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Have a good week end
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Julien
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Thu, 2 Nov 2023 at 19:25, Richard
> > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > <rz...@apache.org>
> > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Short update. Sorted out an issue
> > > > > > > > > > > > > > > > > with the
> > > > > > > > > > > > > > > > > tar.gz/zip with
> > > > > > > > > > > > > > > > > Julien today
> > > > > > > > > > > > > > > > > and re-uploaded them to the nightlies
> > > > > > > > > > > > > > > > > area.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > This new bundle works as expected in
> > > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > > deployment but happy to
> > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > additional feedback before getting up
> > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > first
> > > > > > > > > > > > > > > > > release candidate
> > > > > > > > > > > > > > > > > ;-)
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 08:21
> > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > schrieb Richard
> > > > > > > > > > > > > > > > > Zowalla:
> > > > > > > > > > > > > > > > > > Hi Alexandre,
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > we are not in a hurry here :) -
> > > > > > > > > > > > > > > > > > take as
> > > > > > > > > > > > > > > > > > much as time you
> > > > > > > > > > > > > > > > > > need.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 07:50
> > > > > > > > > > > > > > > > > > +0100
> > > > > > > > > > > > > > > > > > schrieb Alexandre
> > > > > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > > > > Hello Richard,
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Okay, I'm more than happy to do
> > > > > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > my pre-production
> > > > > > > > > > > > > > > > > > > cluster
> > > > > > > > > > > > > > > > > > > (~10 topologies) using the binary
> > > > > > > > > > > > > > > > > > > artifacts.
> > > > > > > > > > > > > > > > > > > Would it be OK if I can use up to
> > > > > > > > > > > > > > > > > > > end
> > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > this week so that
> > > > > > > > > > > > > > > > > > > I'll be
> > > > > > > > > > > > > > > > > > > able to have enough time to check
> > > > > > > > > > > > > > > > > > > all
> > > > > > > > > > > > > > > > > > > potential issues that
> > > > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > > > upgrade bring?
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Le lun. 30 oct. 2023 à 07:35,
> > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > > <rz...@apache.org> a
> > > > > > > > > > > > > > > > > > > écrit :
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > before starting a release of
> > > > > > > > > > > > > > > > > > > > 2.6.0-
> > > > > > > > > > > > > > > > > > > > SNAPSHOT, I would like
> > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > > > some feedback on the current
> > > > > > > > > > > > > > > > > > > > SNAPSHOT
> > > > > > > > > > > > > > > > > > > > build.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > I just uploaded a 2.6.0-
> > > > > > > > > > > > > > > > > > > > SNAPSHOT of
> > > > > > > > > > > > > > > > > > > > Storm build from
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > It is available via the ASF
> > > > > > > > > > > > > > > > > > > > Snapshot
> > > > > > > > > > > > > > > > > > > > repository:
> > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > You can consume by adding
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > <repository>
> > > > > > > > > > > > > > > > > > > > <id>apache.snapshots</id>
> > > > > > > > > > > > > > > > > > > > <name>Apache Snapshot
> > > > > > > > > > > > > > > > > > > > Repository</name>
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > <url>
> > > > > > > > > > > > > > > > > > > > https://repository.apache.org/snap
> > > > > > > > > > > > > > > > > > > > shots</url>
> > > > > > > > > > > > > > > > > > > > <releases>
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > <enabled>false</enabled>
> > > > > > > > > > > > > > > > > > > > </releases>
> > > > > > > > > > > > > > > > > > > > </repository>
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > to your project pom. As we do
> > > > > > > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > > > > > deploy SNAPSHOTS
> > > > > > > > > > > > > > > > > > > > automatically,
> > > > > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > > > should be easy to just consume
> > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > latest SNAPSHOT.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > The packaged binaries are
> > > > > > > > > > > > > > > > > > > > available
> > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > nightlies.apache.org:
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > If you have some minutes left:
> > > > > > > > > > > > > > > > > > > > Please
> > > > > > > > > > > > > > > > > > > > test and report any
> > > > > > > > > > > > > > > > > > > > issues
> > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > this binaries, so we can fix
> > > > > > > > > > > > > > > > > > > > before
> > > > > > > > > > > > > > > > > > > > attempting to
> > > > > > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > The most significant changes
> > > > > > > > > > > > > > > > > > > > are in
> > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > are of
> > > > > > > > > > > > > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > > upgraded from 2.x to 3.x - our
> > > > > > > > > > > > > > > > > > > > own
> > > > > > > > > > > > > > > > > > > > test
> > > > > > > > > > > > > > > > > > > > coverage within
> > > > > > > > > > > > > > > > > > > > the build
> > > > > > > > > > > > > > > > > > > > looks
> > > > > > > > > > > > > > > > > > > > good but would be nice to get
> > > > > > > > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > > > > real world use-case
> > > > > > > > > > > > > > > > > > > > feedback
> > > > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > > this. In addition, we had quite
> > > > > > > > > > > > > > > > > > > > a lof
> > > > > > > > > > > > > > > > > > > > of 3rd party
> > > > > > > > > > > > > > > > > > > > dependency
> > > > > > > > > > > > > > > > > > > > upgrades.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > In addition, it contains the
> > > > > > > > > > > > > > > > > > > > pruning
> > > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > external modules
> > > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > > listed
> > > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > A summaryin Jira is here:
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > *Open Source Solutions for Text
> > > > > > > > > > > > > > > > Engineering*
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > http://www.digitalpebble.com
> > > > > > > > > > > > > > > > http://digitalpebble.blogspot.com/
> > > > > > > > > > > > > > > > #digitalpebble
> > > > > > > > > > > > > > > > <http://twitter.com/digitalpebble>
> > > > > > > > > > > > >
> > > > >
> > > >
> > >
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Yet another finding on my side to "not call yet for vote on a Storm
2.6.0", we have some CVEs and we shoudn't release a version having
known CVEs.
For example, we have
find . -name "commons-text*jar" -print
./lib-webapp/commons-text-1.9.jar
./external/storm-autocreds/commons-text-1.10.0.jar
The second one is okay from
https://www.cvedetails.com/cve/CVE-2022-42889/ perspective, but not
the first one.
I will continue checking if there are other CVEs...
Thanks,
Alexandre
Le mar. 7 nov. 2023 à 17:43, Alexandre Vermeerbergen
<av...@gmail.com> a écrit :
>
> Oh thank you very much for this cherry-pinking, I was too busy doing
> other unrelated stuff..
>
> Le mar. 7 nov. 2023 à 17:35, Richard Zowalla <rz...@apache.org> a écrit :
> >
> > I cherry-picked your commit and put that on master
> >
> >
> > Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard Zowalla:
> > > You pushed on your own branch. You would need to push to apache/storm
> > > main branch or open a PR ;-)
> > >
> > > Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb Alexandre
> > > Vermeerbergen:
> > > > Just did a 'git push' with that trivial fix... is that enough?
> > > >
> > > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla <ri...@zowalla.com>
> > > > a
> > > > écrit :
> > > > >
> > > > > You can do as you like ;-)
> > > > >
> > > > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen
> > > > > <av...@gmail.com>:
> > > > > > I mean :
> > > > > >
> > > > > > Can I keep using my
> > > > > > 'STORM-3958-Capacity-to-set-St
> > > > > > orm-UI's-title-in-conf/storm.yaml'
> > > > > > branch to push this fix, or should I create a new one?
> > > > > >
> > > > > > Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > > > > <av...@gmail.com> a écrit :
> > > > > > >
> > > > > > > Can you keep using my
> > > > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > > > > conf/storm.yaml'
> > > > > > > branch to push this fix, or should I create a new one?
> > > > > > >
> > > > > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > > > > <ri...@zowalla.com> a écrit :
> > > > > > > >
> > > > > > > > Just push a fix for it (referencing the original issue) ;-)
> > > > > > > > -
> > > > > > > > we hadn't a release in between, so not a big deal.
> > > > > > > >
> > > > > > > > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre
> > > > > > > > Vermeerbergen <av...@gmail.com>:
> > > > > > > > > Temporary workaround for the Storm UI title's regression:
> > > > > > > > > run this
> > > > > > > > > command on the title-template.html file (in
> > > > > > > > > storm-stable/public/templates directory) after having
> > > > > > > > > extracted the
> > > > > > > > > binary archive:
> > > > > > > > >
> > > > > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > > > > > > > >
> > > > > > > > > Again sorry for having introduced this regression.
> > > > > > > > >
> > > > > > > > > Alexandre
> > > > > > > > >
> > > > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > >
> > > > > > > > > > Updates regarding my test with Storm 2.6.0 Snapshot
> > > > > > > > > > 2023/11/02:
> > > > > > > > > > 1. I have modified my KO bolts using this pattern in
> > > > > > > > > > their prepare() method:
> > > > > > > > > >
> > > > > > > > > > long cacheMaxSize =
> > > > > > > > > > Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZ
> > > > > > > > > > E)
> > > > > > > > > > .toString());
> > > > > > > > > >
> > > > > > > > > > instead of:
> > > > > > > > > >
> > > > > > > > > > long cacheMaxSize = (Long)
> > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > > >
> > > > > > > > > > 2. I have found the cause of the regression in loss of
> > > > > > > > > > 'Storm UI'
> > > > > > > > > > title at the top left-hand side of all Storm UI's
> > > > > > > > > > pages.
> > > > > > > > > > This is a mistake of mine in this PR:
> > > > > > > > > >
> > > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > > > > > > > More precisely, at line 19 of this file:
> > > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > > > > > > > > we must have:
> > > > > > > > > > <h1><a href="/">{{title}}</a></h1>
> > > > > > > > > >
> > > > > > > > > > instead of:
> > > > > > > > > >
> > > > > > > > > > <h1><a href="/">{{ui.title}}</a></h1>
> > > > > > > > > >
> > > > > > > > > > I think I missed this during my tests because of
> > > > > > > > > > browser's cache
> > > > > > > > > > (bad excuse, I know...): probably I tested by-patching
> > > > > > > > > > my
> > > > > > > > > > deployed
> > > > > > > > > > Storm cluster, and I finally forgot to checkout this
> > > > > > > > > > latest change of
> > > > > > > > > > my delivery.
> > > > > > > > > >
> > > > > > > > > > I had to use "private navigation" to make sure this
> > > > > > > > > > time it fixes this issue
> > > > > > > > > >
> > > > > > > > > > What would be the quickest way for this very simple fix
> > > > > > > > > > to be delivered ?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Alexandre
> > > > > > > > > >
> > > > > > > > > > Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > >
> > > > > > > > > > > Thanks Richard for the analysis.
> > > > > > > > > > >
> > > > > > > > > > > My thought on this are very simple: let me fix my own
> > > > > > > > > > > old code, after
> > > > > > > > > > > all this was impacting only one of my topologies, the
> > > > > > > > > > > other one were
> > > > > > > > > > > using more careful Integer.parseInt or
> > > > > > > > > > > Integer.parseLong to avoid
> > > > > > > > > > > making assumption on the way the values in the config
> > > > > > > > > > > Map were
> > > > > > > > > > > serialized.
> > > > > > > > > > >
> > > > > > > > > > > My other concern with 2.6.0 Snapshot is that in Storm
> > > > > > > > > > > UI, the "Storm
> > > > > > > > > > > UI" title at the top left hand side of each page is
> > > > > > > > > > > missing. This may
> > > > > > > > > > > be the result of one of my own PR, I'll have to check
> > > > > > > > > > > why...
> > > > > > > > > > >
> > > > > > > > > > > Alex
> > > > > > > > > > >
> > > > > > > > > > > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla
> > > > > > > > > > > <rz...@apache.org> a écrit :
> > > > > > > > > > > >
> > > > > > > > > > > > Hi,
> > > > > > > > > > > >
> > > > > > > > > > > > thanks for the feedback.
> > > > > > > > > > > >
> > > > > > > > > > > > The main difference to 2.5.0 is, that we switched
> > > > > > > > > > > > the
> > > > > > > > > > > > JSON
> > > > > > > > > > > > implementation in Utils, which relied on a
> > > > > > > > > > > > super old JSON library. What is happening here is,
> > > > > > > > > > > > that the new JSON
> > > > > > > > > > > > library (net.minidev) behaves differently.
> > > > > > > > > > > >
> > > > > > > > > > > > We can see it by writing some simple code without
> > > > > > > > > > > > Storm (The related
> > > > > > > > > > > > code line with the parsing is taken from Storm
> > > > > > > > > > > > Utils)
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Map<String, Long> conf = new HashMap<>();
> > > > > > > > > > > > conf.put("test", 100000000L);
> > > > > > > > > > > >
> > > > > > > > > > > > Map<String, Object> deserTopoConf =
> > > > > > > > > > > > (Map<String,
> > > > > > > > > > > > Object>)
> > > > > > > > > > > > JSONValue.parseWithException(JSONValue.toJSONString
> > > > > > > > > > > > (c
> > > > > > > > > > > > onf));
> > > > > > > > > > > >
> > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > >
> > > > > > > > > > > > System.out.println(e.getValue().getClass().getSimpl
> > > > > > > > > > > > eN
> > > > > > > > > > > > ame());
> > > > > > > > > > > > }
> > > > > > > > > > > >
> > > > > > > > > > > > This will print "Integer". Using
> > > > > > > > > > > > "10000000000000000L"
> > > > > > > > > > > > as a value will
> > > > > > > > > > > > result in "Long".
> > > > > > > > > > > >
> > > > > > > > > > > > The main issue derives from the fact, that we
> > > > > > > > > > > > serialize the config map
> > > > > > > > > > > > into a JSON String and parse the result.
> > > > > > > > > > > > Based on the String representation, "100000000L"
> > > > > > > > > > > > will
> > > > > > > > > > > > still fit into an
> > > > > > > > > > > > Integer, so the parser will not use Long ;-)
> > > > > > > > > > > >
> > > > > > > > > > > > If the number is big enough, the parser will switch
> > > > > > > > > > > > to a Long> >> > >> > > The old library (> 10 years
> > > > > > > > > > > > old now):
> > > > > > > > > > > >
> > > > > > > > > > > > deserTopoConf = (Map<String, Object>)
> > > > > > > > > > > > org.json.simple.JSONValue.parseWithException(org.js
> > > > > > > > > > > > on
> > > > > > > > > > > > .simple.JSONValue.
> > > > > > > > > > > > toJSONString(conf));
> > > > > > > > > > > >
> > > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > > >
> > > > > > > > > > > > System.out.println(e.getValue().getClass().getSimpl
> > > > > > > > > > > > eN
> > > > > > > > > > > > ame());
> > > > > > > > > > > > }
> > > > > > > > > > > >
> > > > > > > > > > > > This will print "Long".
> > > > > > > > > > > >
> > > > > > > > > > > > The difference between these libraries is, that the
> > > > > > > > > > > > super old
> > > > > > > > > > > > org.json.simple just always returns Long as the
> > > > > > > > > > > > type.
> > > > > > > > > > > >
> > > > > > > > > > > > If can eventually tune the new JSON parser to
> > > > > > > > > > > > something like:
> > > > > > > > > > > >
> > > > > > > > > > > > deserTopoConf = (Map<String, Object>) new
> > > > > > > > > > > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse
> > > > > > > > > > > > (J
> > > > > > > > > > > > SONValue= toJSONStr
> > > > > > > > > > > > ing(conf), JSONValue.defaultReader.DEFAULT);
> > > > > > > > > > > >
> > > > > > > > > > > > This will result in "Long" but might have other
> > > > > > > > > > > > side-
> > > > > > > > > > > > effects.
> > > > > > > > > > > >
> > > > > > > > > > > > From my point of view, the newer JSON library works
> > > > > > > > > > > > as expected
> > > > > > > > > > > > (although it might break backward compatibility).
> > > > > > > > > > > > Unchecked casts are always evil, imho. In your
> > > > > > > > > > > > specific case, it might
> > > > > > > > > > > > be more save to cast to "Number" and than invoke
> > > > > > > > > > > > "longValue()" instead.
> > > > > > > > > > > >
> > > > > > > > > > > > Nevertheless, we might being able to restore the
> > > > > > > > > > > > old
> > > > > > > > > > > > behaviour by
> > > > > > > > > > > > tuning the JSONParser configuration.
> > > > > > > > > > > >
> > > > > > > > > > > > Thoughts?
> > > > > > > > > > > >
> > > > > > > > > > > > Gruß
> > > > > > > > > > > > Richard
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb
> > > > > > > > > > > > Alexandre
> > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > On a side note, the Long value that "MyKOBolt" is
> > > > > > > > > > > > > set by a
> > > > > > > > > > > > > DefaultTopologyConfig to a value with is a
> > > > > > > > > > > > > java.lang.Long instance...
> > > > > > > > > > > > > hence it's surprising to see it internally
> > > > > > > > > > > > > converted into a
> > > > > > > > > > > > > java.lang.Integer in the Map passed to prepare()
> > > > > > > > > > > > > method...
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Le ven. 3 nov. 2023 à 18:27, Alexandre
> > > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Hello,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I found a breaking change using Apache Storm
> > > > > > > > > > > > > > 2.6.0 snapshot build
> > > > > > > > > > > > > > dated 20231103 and without changing my
> > > > > > > > > > > > > > topologies' code (keeping
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > same big jars).
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > This seems to be tied to a datatype change in
> > > > > > > > > > > > > > Map
> > > > > > > > > > > > > > argument of
> > > > > > > > > > > > > > prepare() method of Bolt.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > For example, this Bolt is KO at runtime:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > public class MyKOBolt extends BaseRichBolt {
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > public void prepare(Map stormConf,
> > > > > > > > > > > > > > TopologyContext context,
> > > > > > > > > > > > > > OutputCollector collector) {
> > > > > > > > > > > > > > cacheMaxSize = (Long)
> > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > > > > > > > cacheTtlMin = (Long)
> > > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > as it throws this error:
> > > > > > > > > > > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > > > > > > > > > > > > > Thread-14-__system-executor[-1, -1] [INFO]
> > > > > > > > > > > > > > Preparing bolt
> > > > > > > > > > > > > > __system:[-1]
> > > > > > > > > > > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > > > > > > > > > > > > Thread-15-evaluateTriggers-executor[11, 11]
> > > > > > > > > > > > > > [ERROR] Async loop
> > > > > > > > > > > > > > died!
> > > > > > > > > > > > > > java.lang.ClassCastException: java.lang.Integer
> > > > > > > > > > > > > > incompatible with
> > > > > > > > > > > > > > java.lang.Long
> > > > > > > > > > > > > > at com.acme.storm.alerting.MyKOBolt
> > > > > > > > > > > > > > .prepare(EvaluationBolt.java:71)
> > > > > > > > > > > > > > ~[stormjar.jar:?]
> > > > > > > > > > > > > > at
> > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.ini
> > > > > > > > > > > > > > t(
> > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > 128)
> > > > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > at
> > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.cal
> > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > 138)
> > > > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > at
> > > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.cal
> > > > > > > > > > > > > > l(
> > > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > > 54)
> > > > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > at
> > > > > > > > > > > > > > org.apache.storm.utils.Utils$1.run(Utils.java:3
> > > > > > > > > > > > > > 93
> > > > > > > > > > > > > > )
> > > > > > > > > > > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > whereas this bolt is fine with Storm 2.6.0:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > public class MyOKBolt extends BaseBasicBolt {
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > @Override
> > > > > > > > > > > > > > public void prepare(final Map stormConf,
> > > > > > > > > > > > > > final TopologyContext
> > > > > > > > > > > > > > context) {
> > > > > > > > > > > > > > super.prepare(stormConf, context);
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > this.redisPort =
> > > > > > > > > > > > > > Integer.parseInt((String)
> > > > > > > > > > > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > That said, I can modify the code of "MyKOBolt"
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > use same
> > > > > > > > > > > > > > Integer.parseInt or Integer.parseLong trick,
> > > > > > > > > > > > > > but
> > > > > > > > > > > > > > it's the first
> > > > > > > > > > > > > > time
> > > > > > > > > > > > > > in my long history of upgrades that I have seen
> > > > > > > > > > > > > > such runtime
> > > > > > > > > > > > > > incompatibility.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Maybe "MyKOBolt" was badly written since the
> > > > > > > > > > > > > > beginning and I have
> > > > > > > > > > > > > > just
> > > > > > > > > > > > > > hit the punishment for it: is there a
> > > > > > > > > > > > > > documentation which clarifies
> > > > > > > > > > > > > > datatypes of the Map argument of prepare()
> > > > > > > > > > > > > > method?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Also worth noting: "MyKOBolt" derives from
> > > > > > > > > > > > > > BaseRichBolt , while
> > > > > > > > > > > > > > "MyOKBolt" derives from BaseBasicBolt => could
> > > > > > > > > > > > > > this have any impact
> > > > > > > > > > > > > > on
> > > > > > > > > > > > > > this finding ?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Last, I'm running this on Redhat Linux 8 and
> > > > > > > > > > > > > > IBM
> > > > > > > > > > > > > > Semeru JDK
> > > > > > > > > > > > > > 17.0.8.1.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > > > > > > > > > > > > > <li...@gmail.com> a écrit :
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thanks Richard.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Tried the latest snapshot with StormCrawler
> > > > > > > > > > > > > > > both in local and
> > > > > > > > > > > > > > > deployed mode
> > > > > > > > > > > > > > > and did not find any issues.
> > > > > > > > > > > > > > > Will try it on a topology generating WARC
> > > > > > > > > > > > > > > files
> > > > > > > > > > > > > > > next week to
> > > > > > > > > > > > > > > check that the
> > > > > > > > > > > > > > > dependency changes on Hadoop have not broken
> > > > > > > > > > > > > > > anything.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Have a good week end
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Julien
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla
> > > > > > > > > > > > > > > <rz...@apache.org>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Short update. Sorted out an issue with the
> > > > > > > > > > > > > > > > tar.gz/zip with
> > > > > > > > > > > > > > > > Julien today
> > > > > > > > > > > > > > > > and re-uploaded them to the nightlies area.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > This new bundle works as expected in my
> > > > > > > > > > > > > > > > deployment but happy to
> > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > additional feedback before getting up a
> > > > > > > > > > > > > > > > first
> > > > > > > > > > > > > > > > release candidate
> > > > > > > > > > > > > > > > ;-)
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100
> > > > > > > > > > > > > > > > schrieb Richard
> > > > > > > > > > > > > > > > Zowalla:
> > > > > > > > > > > > > > > > > Hi Alexandre,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > we are not in a hurry here :) - take as
> > > > > > > > > > > > > > > > > much as time you
> > > > > > > > > > > > > > > > > need.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100
> > > > > > > > > > > > > > > > > schrieb Alexandre
> > > > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > > > Hello Richard,
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Okay, I'm more than happy to do that
> > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > my pre-production
> > > > > > > > > > > > > > > > > > cluster
> > > > > > > > > > > > > > > > > > (~10 topologies) using the binary
> > > > > > > > > > > > > > > > > > artifacts.
> > > > > > > > > > > > > > > > > > Would it be OK if I can use up to end
> > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > this week so that
> > > > > > > > > > > > > > > > > > I'll be
> > > > > > > > > > > > > > > > > > able to have enough time to check all
> > > > > > > > > > > > > > > > > > potential issues that
> > > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > > upgrade bring?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard
> > > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > > <rz...@apache.org> a
> > > > > > > > > > > > > > > > > > écrit :
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > before starting a release of 2.6.0-
> > > > > > > > > > > > > > > > > > > SNAPSHOT, I would like
> > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > > some feedback on the current SNAPSHOT
> > > > > > > > > > > > > > > > > > > build.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of
> > > > > > > > > > > > > > > > > > > Storm build from
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > It is available via the ASF Snapshot
> > > > > > > > > > > > > > > > > > > repository:
> > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > You can consume by adding
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > <repository>
> > > > > > > > > > > > > > > > > > > <id>apache.snapshots</id>
> > > > > > > > > > > > > > > > > > > <name>Apache Snapshot
> > > > > > > > > > > > > > > > > > > Repository</name>
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > <url>
> > > > > > > > > > > > > > > > > > > https://repository.apache.org/snap
> > > > > > > > > > > > > > > > > > > shots</url>
> > > > > > > > > > > > > > > > > > > <releases>
> > > > > > > > > > > > > > > > > > > <enabled>false</enabled>
> > > > > > > > > > > > > > > > > > > </releases>
> > > > > > > > > > > > > > > > > > > </repository>
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > to your project pom. As we do not
> > > > > > > > > > > > > > > > > > > deploy SNAPSHOTS
> > > > > > > > > > > > > > > > > > > automatically,
> > > > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > > should be easy to just consume the
> > > > > > > > > > > > > > > > > > > latest SNAPSHOT.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > The packaged binaries are available
> > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > nightlies.apache.org:
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > If you have some minutes left: Please
> > > > > > > > > > > > > > > > > > > test and report any
> > > > > > > > > > > > > > > > > > > issues
> > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > this binaries, so we can fix before
> > > > > > > > > > > > > > > > > > > attempting to
> > > > > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > The most significant changes are in
> > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > are of
> > > > > > > > > > > > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > upgraded from 2.x to 3.x - our own
> > > > > > > > > > > > > > > > > > > test
> > > > > > > > > > > > > > > > > > > coverage within
> > > > > > > > > > > > > > > > > > > the build
> > > > > > > > > > > > > > > > > > > looks
> > > > > > > > > > > > > > > > > > > good but would be nice to get some
> > > > > > > > > > > > > > > > > > > real world use-case
> > > > > > > > > > > > > > > > > > > feedback
> > > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > this. In addition, we had quite a lof
> > > > > > > > > > > > > > > > > > > of 3rd party
> > > > > > > > > > > > > > > > > > > dependency
> > > > > > > > > > > > > > > > > > > upgrades.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > In addition, it contains the pruning
> > > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > external modules
> > > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > listed
> > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > A summaryin Jira is here:
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > *Open Source Solutions for Text Engineering*
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > http://www.digitalpebble.com
> > > > > > > > > > > > > > > http://digitalpebble.blogspot.com/
> > > > > > > > > > > > > > > #digitalpebble
> > > > > > > > > > > > > > > <http://twitter.com/digitalpebble>
> > > > > > > > > > > >
> > > >
> > >
> >
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Oh thank you very much for this cherry-pinking, I was too busy doing
other unrelated stuff..
Le mar. 7 nov. 2023 à 17:35, Richard Zowalla <rz...@apache.org> a écrit :
>
> I cherry-picked your commit and put that on master
>
>
> Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard Zowalla:
> > You pushed on your own branch. You would need to push to apache/storm
> > main branch or open a PR ;-)
> >
> > Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb Alexandre
> > Vermeerbergen:
> > > Just did a 'git push' with that trivial fix... is that enough?
> > >
> > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla <ri...@zowalla.com>
> > > a
> > > écrit :
> > > >
> > > > You can do as you like ;-)
> > > >
> > > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen
> > > > <av...@gmail.com>:
> > > > > I mean :
> > > > >
> > > > > Can I keep using my
> > > > > 'STORM-3958-Capacity-to-set-St
> > > > > orm-UI's-title-in-conf/storm.yaml'
> > > > > branch to push this fix, or should I create a new one?
> > > > >
> > > > > Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > > > <av...@gmail.com> a écrit :
> > > > > >
> > > > > > Can you keep using my
> > > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > > > conf/storm.yaml'
> > > > > > branch to push this fix, or should I create a new one?
> > > > > >
> > > > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > > > <ri...@zowalla.com> a écrit :
> > > > > > >
> > > > > > > Just push a fix for it (referencing the original issue) ;-)
> > > > > > > -
> > > > > > > we hadn't a release in between, so not a big deal.
> > > > > > >
> > > > > > > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre
> > > > > > > Vermeerbergen <av...@gmail.com>:
> > > > > > > > Temporary workaround for the Storm UI title's regression:
> > > > > > > > run this
> > > > > > > > command on the title-template.html file (in
> > > > > > > > storm-stable/public/templates directory) after having
> > > > > > > > extracted the
> > > > > > > > binary archive:
> > > > > > > >
> > > > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > > > > > > >
> > > > > > > > Again sorry for having introduced this regression.
> > > > > > > >
> > > > > > > > Alexandre
> > > > > > > >
> > > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > >
> > > > > > > > > Updates regarding my test with Storm 2.6.0 Snapshot
> > > > > > > > > 2023/11/02:
> > > > > > > > > 1. I have modified my KO bolts using this pattern in
> > > > > > > > > their prepare() method:
> > > > > > > > >
> > > > > > > > > long cacheMaxSize =
> > > > > > > > > Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZ
> > > > > > > > > E)
> > > > > > > > > .toString());
> > > > > > > > >
> > > > > > > > > instead of:
> > > > > > > > >
> > > > > > > > > long cacheMaxSize = (Long)
> > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > >
> > > > > > > > > 2. I have found the cause of the regression in loss of
> > > > > > > > > 'Storm UI'
> > > > > > > > > title at the top left-hand side of all Storm UI's
> > > > > > > > > pages.
> > > > > > > > > This is a mistake of mine in this PR:
> > > > > > > > >
> > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > > > > > > More precisely, at line 19 of this file:
> > > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > > > > > > > we must have:
> > > > > > > > > <h1><a href="/">{{title}}</a></h1>
> > > > > > > > >
> > > > > > > > > instead of:
> > > > > > > > >
> > > > > > > > > <h1><a href="/">{{ui.title}}</a></h1>
> > > > > > > > >
> > > > > > > > > I think I missed this during my tests because of
> > > > > > > > > browser's cache
> > > > > > > > > (bad excuse, I know...): probably I tested by-patching
> > > > > > > > > my
> > > > > > > > > deployed
> > > > > > > > > Storm cluster, and I finally forgot to checkout this
> > > > > > > > > latest change of
> > > > > > > > > my delivery.
> > > > > > > > >
> > > > > > > > > I had to use "private navigation" to make sure this
> > > > > > > > > time it fixes this issue
> > > > > > > > >
> > > > > > > > > What would be the quickest way for this very simple fix
> > > > > > > > > to be delivered ?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Alexandre
> > > > > > > > >
> > > > > > > > > Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > >
> > > > > > > > > > Thanks Richard for the analysis.
> > > > > > > > > >
> > > > > > > > > > My thought on this are very simple: let me fix my own
> > > > > > > > > > old code, after
> > > > > > > > > > all this was impacting only one of my topologies, the
> > > > > > > > > > other one were
> > > > > > > > > > using more careful Integer.parseInt or
> > > > > > > > > > Integer.parseLong to avoid
> > > > > > > > > > making assumption on the way the values in the config
> > > > > > > > > > Map were
> > > > > > > > > > serialized.
> > > > > > > > > >
> > > > > > > > > > My other concern with 2.6.0 Snapshot is that in Storm
> > > > > > > > > > UI, the "Storm
> > > > > > > > > > UI" title at the top left hand side of each page is
> > > > > > > > > > missing. This may
> > > > > > > > > > be the result of one of my own PR, I'll have to check
> > > > > > > > > > why...
> > > > > > > > > >
> > > > > > > > > > Alex
> > > > > > > > > >
> > > > > > > > > > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla
> > > > > > > > > > <rz...@apache.org> a écrit :
> > > > > > > > > > >
> > > > > > > > > > > Hi,
> > > > > > > > > > >
> > > > > > > > > > > thanks for the feedback.
> > > > > > > > > > >
> > > > > > > > > > > The main difference to 2.5.0 is, that we switched
> > > > > > > > > > > the
> > > > > > > > > > > JSON
> > > > > > > > > > > implementation in Utils, which relied on a
> > > > > > > > > > > super old JSON library. What is happening here is,
> > > > > > > > > > > that the new JSON
> > > > > > > > > > > library (net.minidev) behaves differently.
> > > > > > > > > > >
> > > > > > > > > > > We can see it by writing some simple code without
> > > > > > > > > > > Storm (The related
> > > > > > > > > > > code line with the parsing is taken from Storm
> > > > > > > > > > > Utils)
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Map<String, Long> conf = new HashMap<>();
> > > > > > > > > > > conf.put("test", 100000000L);
> > > > > > > > > > >
> > > > > > > > > > > Map<String, Object> deserTopoConf =
> > > > > > > > > > > (Map<String,
> > > > > > > > > > > Object>)
> > > > > > > > > > > JSONValue.parseWithException(JSONValue.toJSONString
> > > > > > > > > > > (c
> > > > > > > > > > > onf));
> > > > > > > > > > >
> > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > >
> > > > > > > > > > > System.out.println(e.getValue().getClass().getSimpl
> > > > > > > > > > > eN
> > > > > > > > > > > ame());
> > > > > > > > > > > }
> > > > > > > > > > >
> > > > > > > > > > > This will print "Integer". Using
> > > > > > > > > > > "10000000000000000L"
> > > > > > > > > > > as a value will
> > > > > > > > > > > result in "Long".
> > > > > > > > > > >
> > > > > > > > > > > The main issue derives from the fact, that we
> > > > > > > > > > > serialize the config map
> > > > > > > > > > > into a JSON String and parse the result.
> > > > > > > > > > > Based on the String representation, "100000000L"
> > > > > > > > > > > will
> > > > > > > > > > > still fit into an
> > > > > > > > > > > Integer, so the parser will not use Long ;-)
> > > > > > > > > > >
> > > > > > > > > > > If the number is big enough, the parser will switch
> > > > > > > > > > > to a Long> >> > >> > > The old library (> 10 years
> > > > > > > > > > > old now):
> > > > > > > > > > >
> > > > > > > > > > > deserTopoConf = (Map<String, Object>)
> > > > > > > > > > > org.json.simple.JSONValue.parseWithException(org.js
> > > > > > > > > > > on
> > > > > > > > > > > .simple.JSONValue.
> > > > > > > > > > > toJSONString(conf));
> > > > > > > > > > >
> > > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > > >
> > > > > > > > > > > System.out.println(e.getValue().getClass().getSimpl
> > > > > > > > > > > eN
> > > > > > > > > > > ame());
> > > > > > > > > > > }
> > > > > > > > > > >
> > > > > > > > > > > This will print "Long".
> > > > > > > > > > >
> > > > > > > > > > > The difference between these libraries is, that the
> > > > > > > > > > > super old
> > > > > > > > > > > org.json.simple just always returns Long as the
> > > > > > > > > > > type.
> > > > > > > > > > >
> > > > > > > > > > > If can eventually tune the new JSON parser to
> > > > > > > > > > > something like:
> > > > > > > > > > >
> > > > > > > > > > > deserTopoConf = (Map<String, Object>) new
> > > > > > > > > > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse
> > > > > > > > > > > (J
> > > > > > > > > > > SONValue= toJSONStr
> > > > > > > > > > > ing(conf), JSONValue.defaultReader.DEFAULT);
> > > > > > > > > > >
> > > > > > > > > > > This will result in "Long" but might have other
> > > > > > > > > > > side-
> > > > > > > > > > > effects.
> > > > > > > > > > >
> > > > > > > > > > > From my point of view, the newer JSON library works
> > > > > > > > > > > as expected
> > > > > > > > > > > (although it might break backward compatibility).
> > > > > > > > > > > Unchecked casts are always evil, imho. In your
> > > > > > > > > > > specific case, it might
> > > > > > > > > > > be more save to cast to "Number" and than invoke
> > > > > > > > > > > "longValue()" instead.
> > > > > > > > > > >
> > > > > > > > > > > Nevertheless, we might being able to restore the
> > > > > > > > > > > old
> > > > > > > > > > > behaviour by
> > > > > > > > > > > tuning the JSONParser configuration.
> > > > > > > > > > >
> > > > > > > > > > > Thoughts?
> > > > > > > > > > >
> > > > > > > > > > > Gruß
> > > > > > > > > > > Richard
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb
> > > > > > > > > > > Alexandre
> > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > On a side note, the Long value that "MyKOBolt" is
> > > > > > > > > > > > set by a
> > > > > > > > > > > > DefaultTopologyConfig to a value with is a
> > > > > > > > > > > > java.lang.Long instance...
> > > > > > > > > > > > hence it's surprising to see it internally
> > > > > > > > > > > > converted into a
> > > > > > > > > > > > java.lang.Integer in the Map passed to prepare()
> > > > > > > > > > > > method...
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Le ven. 3 nov. 2023 à 18:27, Alexandre
> > > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hello,
> > > > > > > > > > > > >
> > > > > > > > > > > > > I found a breaking change using Apache Storm
> > > > > > > > > > > > > 2.6.0 snapshot build
> > > > > > > > > > > > > dated 20231103 and without changing my
> > > > > > > > > > > > > topologies' code (keeping
> > > > > > > > > > > > > the
> > > > > > > > > > > > > same big jars).
> > > > > > > > > > > > >
> > > > > > > > > > > > > This seems to be tied to a datatype change in
> > > > > > > > > > > > > Map
> > > > > > > > > > > > > argument of
> > > > > > > > > > > > > prepare() method of Bolt.
> > > > > > > > > > > > >
> > > > > > > > > > > > > For example, this Bolt is KO at runtime:
> > > > > > > > > > > > >
> > > > > > > > > > > > > public class MyKOBolt extends BaseRichBolt {
> > > > > > > > > > > > >
> > > > > > > > > > > > > @Override
> > > > > > > > > > > > > public void prepare(Map stormConf,
> > > > > > > > > > > > > TopologyContext context,
> > > > > > > > > > > > > OutputCollector collector) {
> > > > > > > > > > > > > cacheMaxSize = (Long)
> > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > > > > > > cacheTtlMin = (Long)
> > > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > > > > > > > > > > > >
> > > > > > > > > > > > > as it throws this error:
> > > > > > > > > > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > > > > > > > > > > > > Thread-14-__system-executor[-1, -1] [INFO]
> > > > > > > > > > > > > Preparing bolt
> > > > > > > > > > > > > __system:[-1]
> > > > > > > > > > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > > > > > > > > > > > Thread-15-evaluateTriggers-executor[11, 11]
> > > > > > > > > > > > > [ERROR] Async loop
> > > > > > > > > > > > > died!
> > > > > > > > > > > > > java.lang.ClassCastException: java.lang.Integer
> > > > > > > > > > > > > incompatible with
> > > > > > > > > > > > > java.lang.Long
> > > > > > > > > > > > > at com.acme.storm.alerting.MyKOBolt
> > > > > > > > > > > > > .prepare(EvaluationBolt.java:71)
> > > > > > > > > > > > > ~[stormjar.jar:?]
> > > > > > > > > > > > > at
> > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.ini
> > > > > > > > > > > > > t(
> > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > 128)
> > > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > at
> > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.cal
> > > > > > > > > > > > > l(
> > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > 138)
> > > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > at
> > > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.cal
> > > > > > > > > > > > > l(
> > > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > > 54)
> > > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > at
> > > > > > > > > > > > > org.apache.storm.utils.Utils$1.run(Utils.java:3
> > > > > > > > > > > > > 93
> > > > > > > > > > > > > )
> > > > > > > > > > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> > > > > > > > > > > > >
> > > > > > > > > > > > > whereas this bolt is fine with Storm 2.6.0:
> > > > > > > > > > > > >
> > > > > > > > > > > > > public class MyOKBolt extends BaseBasicBolt {
> > > > > > > > > > > > >
> > > > > > > > > > > > > @Override
> > > > > > > > > > > > > public void prepare(final Map stormConf,
> > > > > > > > > > > > > final TopologyContext
> > > > > > > > > > > > > context) {
> > > > > > > > > > > > > super.prepare(stormConf, context);
> > > > > > > > > > > > >
> > > > > > > > > > > > > this.redisPort =
> > > > > > > > > > > > > Integer.parseInt((String)
> > > > > > > > > > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > > > > > > > > > > > >
> > > > > > > > > > > > > That said, I can modify the code of "MyKOBolt"
> > > > > > > > > > > > > to
> > > > > > > > > > > > > use same
> > > > > > > > > > > > > Integer.parseInt or Integer.parseLong trick,
> > > > > > > > > > > > > but
> > > > > > > > > > > > > it's the first
> > > > > > > > > > > > > time
> > > > > > > > > > > > > in my long history of upgrades that I have seen
> > > > > > > > > > > > > such runtime
> > > > > > > > > > > > > incompatibility.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Maybe "MyKOBolt" was badly written since the
> > > > > > > > > > > > > beginning and I have
> > > > > > > > > > > > > just
> > > > > > > > > > > > > hit the punishment for it: is there a
> > > > > > > > > > > > > documentation which clarifies
> > > > > > > > > > > > > datatypes of the Map argument of prepare()
> > > > > > > > > > > > > method?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Also worth noting: "MyKOBolt" derives from
> > > > > > > > > > > > > BaseRichBolt , while
> > > > > > > > > > > > > "MyOKBolt" derives from BaseBasicBolt => could
> > > > > > > > > > > > > this have any impact
> > > > > > > > > > > > > on
> > > > > > > > > > > > > this finding ?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Last, I'm running this on Redhat Linux 8 and
> > > > > > > > > > > > > IBM
> > > > > > > > > > > > > Semeru JDK
> > > > > > > > > > > > > 17.0.8.1.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > >
> > > > > > > > > > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > > > > > > > > > > > > <li...@gmail.com> a écrit :
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Thanks Richard.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Tried the latest snapshot with StormCrawler
> > > > > > > > > > > > > > both in local and
> > > > > > > > > > > > > > deployed mode
> > > > > > > > > > > > > > and did not find any issues.
> > > > > > > > > > > > > > Will try it on a topology generating WARC
> > > > > > > > > > > > > > files
> > > > > > > > > > > > > > next week to
> > > > > > > > > > > > > > check that the
> > > > > > > > > > > > > > dependency changes on Hadoop have not broken
> > > > > > > > > > > > > > anything.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Have a good week end
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Julien
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla
> > > > > > > > > > > > > > <rz...@apache.org>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Short update. Sorted out an issue with the
> > > > > > > > > > > > > > > tar.gz/zip with
> > > > > > > > > > > > > > > Julien today
> > > > > > > > > > > > > > > and re-uploaded them to the nightlies area.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > This new bundle works as expected in my
> > > > > > > > > > > > > > > deployment but happy to
> > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > additional feedback before getting up a
> > > > > > > > > > > > > > > first
> > > > > > > > > > > > > > > release candidate
> > > > > > > > > > > > > > > ;-)
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100
> > > > > > > > > > > > > > > schrieb Richard
> > > > > > > > > > > > > > > Zowalla:
> > > > > > > > > > > > > > > > Hi Alexandre,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > we are not in a hurry here :) - take as
> > > > > > > > > > > > > > > > much as time you
> > > > > > > > > > > > > > > > need.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100
> > > > > > > > > > > > > > > > schrieb Alexandre
> > > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > > Hello Richard,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Okay, I'm more than happy to do that
> > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > my pre-production
> > > > > > > > > > > > > > > > > cluster
> > > > > > > > > > > > > > > > > (~10 topologies) using the binary
> > > > > > > > > > > > > > > > > artifacts.
> > > > > > > > > > > > > > > > > Would it be OK if I can use up to end
> > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > this week so that
> > > > > > > > > > > > > > > > > I'll be
> > > > > > > > > > > > > > > > > able to have enough time to check all
> > > > > > > > > > > > > > > > > potential issues that
> > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > upgrade bring?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard
> > > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > > <rz...@apache.org> a
> > > > > > > > > > > > > > > > > écrit :
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > before starting a release of 2.6.0-
> > > > > > > > > > > > > > > > > > SNAPSHOT, I would like
> > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > > some feedback on the current SNAPSHOT
> > > > > > > > > > > > > > > > > > build.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of
> > > > > > > > > > > > > > > > > > Storm build from
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > It is available via the ASF Snapshot
> > > > > > > > > > > > > > > > > > repository:
> > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > You can consume by adding
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > <repository>
> > > > > > > > > > > > > > > > > > <id>apache.snapshots</id>
> > > > > > > > > > > > > > > > > > <name>Apache Snapshot
> > > > > > > > > > > > > > > > > > Repository</name>
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > <url>
> > > > > > > > > > > > > > > > > > https://repository.apache.org/snap
> > > > > > > > > > > > > > > > > > shots</url>
> > > > > > > > > > > > > > > > > > <releases>
> > > > > > > > > > > > > > > > > > <enabled>false</enabled>
> > > > > > > > > > > > > > > > > > </releases>
> > > > > > > > > > > > > > > > > > </repository>
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > to your project pom. As we do not
> > > > > > > > > > > > > > > > > > deploy SNAPSHOTS
> > > > > > > > > > > > > > > > > > automatically,
> > > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > should be easy to just consume the
> > > > > > > > > > > > > > > > > > latest SNAPSHOT.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > The packaged binaries are available
> > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > nightlies.apache.org:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > If you have some minutes left: Please
> > > > > > > > > > > > > > > > > > test and report any
> > > > > > > > > > > > > > > > > > issues
> > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > this binaries, so we can fix before
> > > > > > > > > > > > > > > > > > attempting to
> > > > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > The most significant changes are in
> > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > are of
> > > > > > > > > > > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > upgraded from 2.x to 3.x - our own
> > > > > > > > > > > > > > > > > > test
> > > > > > > > > > > > > > > > > > coverage within
> > > > > > > > > > > > > > > > > > the build
> > > > > > > > > > > > > > > > > > looks
> > > > > > > > > > > > > > > > > > good but would be nice to get some
> > > > > > > > > > > > > > > > > > real world use-case
> > > > > > > > > > > > > > > > > > feedback
> > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > this. In addition, we had quite a lof
> > > > > > > > > > > > > > > > > > of 3rd party
> > > > > > > > > > > > > > > > > > dependency
> > > > > > > > > > > > > > > > > > upgrades.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > In addition, it contains the pruning
> > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > external modules
> > > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > listed
> > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > A summaryin Jira is here:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > *Open Source Solutions for Text Engineering*
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > http://www.digitalpebble.com
> > > > > > > > > > > > > > http://digitalpebble.blogspot.com/
> > > > > > > > > > > > > > #digitalpebble
> > > > > > > > > > > > > > <http://twitter.com/digitalpebble>
> > > > > > > > > > >
> > >
> >
>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Richard Zowalla <rz...@apache.org>.
I cherry-picked your commit and put that on master
Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard Zowalla:
> You pushed on your own branch. You would need to push to apache/storm
> main branch or open a PR ;-)
>
> Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb Alexandre
> Vermeerbergen:
> > Just did a 'git push' with that trivial fix... is that enough?
> >
> > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla <ri...@zowalla.com>
> > a
> > écrit :
> > >
> > > You can do as you like ;-)
> > >
> > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen
> > > <av...@gmail.com>:
> > > > I mean :
> > > >
> > > > Can I keep using my
> > > > 'STORM-3958-Capacity-to-set-St
> > > > orm-UI's-title-in-conf/storm.yaml'
> > > > branch to push this fix, or should I create a new one?
> > > >
> > > > Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > > <av...@gmail.com> a écrit :
> > > > >
> > > > > Can you keep using my
> > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > > conf/storm.yaml'
> > > > > branch to push this fix, or should I create a new one?
> > > > >
> > > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > > <ri...@zowalla.com> a écrit :
> > > > > >
> > > > > > Just push a fix for it (referencing the original issue) ;-)
> > > > > > -
> > > > > > we hadn't a release in between, so not a big deal.
> > > > > >
> > > > > > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre
> > > > > > Vermeerbergen <av...@gmail.com>:
> > > > > > > Temporary workaround for the Storm UI title's regression:
> > > > > > > run this
> > > > > > > command on the title-template.html file (in
> > > > > > > storm-stable/public/templates directory) after having
> > > > > > > extracted the
> > > > > > > binary archive:
> > > > > > >
> > > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > > > > > >
> > > > > > > Again sorry for having introduced this regression.
> > > > > > >
> > > > > > > Alexandre
> > > > > > >
> > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > > > > > > <av...@gmail.com> a écrit :
> > > > > > > >
> > > > > > > > Updates regarding my test with Storm 2.6.0 Snapshot
> > > > > > > > 2023/11/02:
> > > > > > > > 1. I have modified my KO bolts using this pattern in
> > > > > > > > their prepare() method:
> > > > > > > >
> > > > > > > > long cacheMaxSize =
> > > > > > > > Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZ
> > > > > > > > E)
> > > > > > > > .toString());
> > > > > > > >
> > > > > > > > instead of:
> > > > > > > >
> > > > > > > > long cacheMaxSize = (Long)
> > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > >
> > > > > > > > 2. I have found the cause of the regression in loss of
> > > > > > > > 'Storm UI'
> > > > > > > > title at the top left-hand side of all Storm UI's
> > > > > > > > pages.
> > > > > > > > This is a mistake of mine in this PR:
> > > > > > > >
> > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > > > > > More precisely, at line 19 of this file:
> > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > > > > > > we must have:
> > > > > > > > <h1><a href="/">{{title}}</a></h1>
> > > > > > > >
> > > > > > > > instead of:
> > > > > > > >
> > > > > > > > <h1><a href="/">{{ui.title}}</a></h1>
> > > > > > > >
> > > > > > > > I think I missed this during my tests because of
> > > > > > > > browser's cache
> > > > > > > > (bad excuse, I know...): probably I tested by-patching
> > > > > > > > my
> > > > > > > > deployed
> > > > > > > > Storm cluster, and I finally forgot to checkout this
> > > > > > > > latest change of
> > > > > > > > my delivery.
> > > > > > > >
> > > > > > > > I had to use "private navigation" to make sure this
> > > > > > > > time it fixes this issue
> > > > > > > >
> > > > > > > > What would be the quickest way for this very simple fix
> > > > > > > > to be delivered ?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Alexandre
> > > > > > > >
> > > > > > > > Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > >
> > > > > > > > > Thanks Richard for the analysis.
> > > > > > > > >
> > > > > > > > > My thought on this are very simple: let me fix my own
> > > > > > > > > old code, after
> > > > > > > > > all this was impacting only one of my topologies, the
> > > > > > > > > other one were
> > > > > > > > > using more careful Integer.parseInt or
> > > > > > > > > Integer.parseLong to avoid
> > > > > > > > > making assumption on the way the values in the config
> > > > > > > > > Map were
> > > > > > > > > serialized.
> > > > > > > > >
> > > > > > > > > My other concern with 2.6.0 Snapshot is that in Storm
> > > > > > > > > UI, the "Storm
> > > > > > > > > UI" title at the top left hand side of each page is
> > > > > > > > > missing. This may
> > > > > > > > > be the result of one of my own PR, I'll have to check
> > > > > > > > > why...
> > > > > > > > >
> > > > > > > > > Alex
> > > > > > > > >
> > > > > > > > > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla
> > > > > > > > > <rz...@apache.org> a écrit :
> > > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > thanks for the feedback.
> > > > > > > > > >
> > > > > > > > > > The main difference to 2.5.0 is, that we switched
> > > > > > > > > > the
> > > > > > > > > > JSON
> > > > > > > > > > implementation in Utils, which relied on a
> > > > > > > > > > super old JSON library. What is happening here is,
> > > > > > > > > > that the new JSON
> > > > > > > > > > library (net.minidev) behaves differently.
> > > > > > > > > >
> > > > > > > > > > We can see it by writing some simple code without
> > > > > > > > > > Storm (The related
> > > > > > > > > > code line with the parsing is taken from Storm
> > > > > > > > > > Utils)
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Map<String, Long> conf = new HashMap<>();
> > > > > > > > > > conf.put("test", 100000000L);
> > > > > > > > > >
> > > > > > > > > > Map<String, Object> deserTopoConf =
> > > > > > > > > > (Map<String,
> > > > > > > > > > Object>)
> > > > > > > > > > JSONValue.parseWithException(JSONValue.toJSONString
> > > > > > > > > > (c
> > > > > > > > > > onf));
> > > > > > > > > >
> > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > >
> > > > > > > > > > System.out.println(e.getValue().getClass().getSimpl
> > > > > > > > > > eN
> > > > > > > > > > ame());
> > > > > > > > > > }
> > > > > > > > > >
> > > > > > > > > > This will print "Integer". Using
> > > > > > > > > > "10000000000000000L"
> > > > > > > > > > as a value will
> > > > > > > > > > result in "Long".
> > > > > > > > > >
> > > > > > > > > > The main issue derives from the fact, that we
> > > > > > > > > > serialize the config map
> > > > > > > > > > into a JSON String and parse the result.
> > > > > > > > > > Based on the String representation, "100000000L"
> > > > > > > > > > will
> > > > > > > > > > still fit into an
> > > > > > > > > > Integer, so the parser will not use Long ;-)
> > > > > > > > > >
> > > > > > > > > > If the number is big enough, the parser will switch
> > > > > > > > > > to a Long> >> > >> > > The old library (> 10 years
> > > > > > > > > > old now):
> > > > > > > > > >
> > > > > > > > > > deserTopoConf = (Map<String, Object>)
> > > > > > > > > > org.json.simple.JSONValue.parseWithException(org.js
> > > > > > > > > > on
> > > > > > > > > > .simple.JSONValue.
> > > > > > > > > > toJSONString(conf));
> > > > > > > > > >
> > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > >
> > > > > > > > > > System.out.println(e.getValue().getClass().getSimpl
> > > > > > > > > > eN
> > > > > > > > > > ame());
> > > > > > > > > > }
> > > > > > > > > >
> > > > > > > > > > This will print "Long".
> > > > > > > > > >
> > > > > > > > > > The difference between these libraries is, that the
> > > > > > > > > > super old
> > > > > > > > > > org.json.simple just always returns Long as the
> > > > > > > > > > type.
> > > > > > > > > >
> > > > > > > > > > If can eventually tune the new JSON parser to
> > > > > > > > > > something like:
> > > > > > > > > >
> > > > > > > > > > deserTopoConf = (Map<String, Object>) new
> > > > > > > > > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse
> > > > > > > > > > (J
> > > > > > > > > > SONValue= toJSONStr
> > > > > > > > > > ing(conf), JSONValue.defaultReader.DEFAULT);
> > > > > > > > > >
> > > > > > > > > > This will result in "Long" but might have other
> > > > > > > > > > side-
> > > > > > > > > > effects.
> > > > > > > > > >
> > > > > > > > > > From my point of view, the newer JSON library works
> > > > > > > > > > as expected
> > > > > > > > > > (although it might break backward compatibility).
> > > > > > > > > > Unchecked casts are always evil, imho. In your
> > > > > > > > > > specific case, it might
> > > > > > > > > > be more save to cast to "Number" and than invoke
> > > > > > > > > > "longValue()" instead.
> > > > > > > > > >
> > > > > > > > > > Nevertheless, we might being able to restore the
> > > > > > > > > > old
> > > > > > > > > > behaviour by
> > > > > > > > > > tuning the JSONParser configuration.
> > > > > > > > > >
> > > > > > > > > > Thoughts?
> > > > > > > > > >
> > > > > > > > > > Gruß
> > > > > > > > > > Richard
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb
> > > > > > > > > > Alexandre
> > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > On a side note, the Long value that "MyKOBolt" is
> > > > > > > > > > > set by a
> > > > > > > > > > > DefaultTopologyConfig to a value with is a
> > > > > > > > > > > java.lang.Long instance...
> > > > > > > > > > > hence it's surprising to see it internally
> > > > > > > > > > > converted into a
> > > > > > > > > > > java.lang.Integer in the Map passed to prepare()
> > > > > > > > > > > method...
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Le ven. 3 nov. 2023 à 18:27, Alexandre
> > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > >
> > > > > > > > > > > > Hello,
> > > > > > > > > > > >
> > > > > > > > > > > > I found a breaking change using Apache Storm
> > > > > > > > > > > > 2.6.0 snapshot build
> > > > > > > > > > > > dated 20231103 and without changing my
> > > > > > > > > > > > topologies' code (keeping
> > > > > > > > > > > > the
> > > > > > > > > > > > same big jars).
> > > > > > > > > > > >
> > > > > > > > > > > > This seems to be tied to a datatype change in
> > > > > > > > > > > > Map
> > > > > > > > > > > > argument of
> > > > > > > > > > > > prepare() method of Bolt.
> > > > > > > > > > > >
> > > > > > > > > > > > For example, this Bolt is KO at runtime:
> > > > > > > > > > > >
> > > > > > > > > > > > public class MyKOBolt extends BaseRichBolt {
> > > > > > > > > > > >
> > > > > > > > > > > > @Override
> > > > > > > > > > > > public void prepare(Map stormConf,
> > > > > > > > > > > > TopologyContext context,
> > > > > > > > > > > > OutputCollector collector) {
> > > > > > > > > > > > cacheMaxSize = (Long)
> > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > > > > > cacheTtlMin = (Long)
> > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > > > > > > > > > > >
> > > > > > > > > > > > as it throws this error:
> > > > > > > > > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > > > > > > > > > > > Thread-14-__system-executor[-1, -1] [INFO]
> > > > > > > > > > > > Preparing bolt
> > > > > > > > > > > > __system:[-1]
> > > > > > > > > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > > > > > > > > > > Thread-15-evaluateTriggers-executor[11, 11]
> > > > > > > > > > > > [ERROR] Async loop
> > > > > > > > > > > > died!
> > > > > > > > > > > > java.lang.ClassCastException: java.lang.Integer
> > > > > > > > > > > > incompatible with
> > > > > > > > > > > > java.lang.Long
> > > > > > > > > > > > at com.acme.storm.alerting.MyKOBolt
> > > > > > > > > > > > .prepare(EvaluationBolt.java:71)
> > > > > > > > > > > > ~[stormjar.jar:?]
> > > > > > > > > > > > at
> > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.ini
> > > > > > > > > > > > t(
> > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > 128)
> > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > at
> > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.cal
> > > > > > > > > > > > l(
> > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > 138)
> > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > at
> > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.cal
> > > > > > > > > > > > l(
> > > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > > 54)
> > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > at
> > > > > > > > > > > > org.apache.storm.utils.Utils$1.run(Utils.java:3
> > > > > > > > > > > > 93
> > > > > > > > > > > > )
> > > > > > > > > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> > > > > > > > > > > >
> > > > > > > > > > > > whereas this bolt is fine with Storm 2.6.0:
> > > > > > > > > > > >
> > > > > > > > > > > > public class MyOKBolt extends BaseBasicBolt {
> > > > > > > > > > > >
> > > > > > > > > > > > @Override
> > > > > > > > > > > > public void prepare(final Map stormConf,
> > > > > > > > > > > > final TopologyContext
> > > > > > > > > > > > context) {
> > > > > > > > > > > > super.prepare(stormConf, context);
> > > > > > > > > > > >
> > > > > > > > > > > > this.redisPort =
> > > > > > > > > > > > Integer.parseInt((String)
> > > > > > > > > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > > > > > > > > > > >
> > > > > > > > > > > > That said, I can modify the code of "MyKOBolt"
> > > > > > > > > > > > to
> > > > > > > > > > > > use same
> > > > > > > > > > > > Integer.parseInt or Integer.parseLong trick,
> > > > > > > > > > > > but
> > > > > > > > > > > > it's the first
> > > > > > > > > > > > time
> > > > > > > > > > > > in my long history of upgrades that I have seen
> > > > > > > > > > > > such runtime
> > > > > > > > > > > > incompatibility.
> > > > > > > > > > > >
> > > > > > > > > > > > Maybe "MyKOBolt" was badly written since the
> > > > > > > > > > > > beginning and I have
> > > > > > > > > > > > just
> > > > > > > > > > > > hit the punishment for it: is there a
> > > > > > > > > > > > documentation which clarifies
> > > > > > > > > > > > datatypes of the Map argument of prepare()
> > > > > > > > > > > > method?
> > > > > > > > > > > >
> > > > > > > > > > > > Also worth noting: "MyKOBolt" derives from
> > > > > > > > > > > > BaseRichBolt , while
> > > > > > > > > > > > "MyOKBolt" derives from BaseBasicBolt => could
> > > > > > > > > > > > this have any impact
> > > > > > > > > > > > on
> > > > > > > > > > > > this finding ?
> > > > > > > > > > > >
> > > > > > > > > > > > Last, I'm running this on Redhat Linux 8 and
> > > > > > > > > > > > IBM
> > > > > > > > > > > > Semeru JDK
> > > > > > > > > > > > 17.0.8.1.
> > > > > > > > > > > >
> > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > Alexandre
> > > > > > > > > > > >
> > > > > > > > > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > > > > > > > > > > > <li...@gmail.com> a écrit :
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks Richard.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Tried the latest snapshot with StormCrawler
> > > > > > > > > > > > > both in local and
> > > > > > > > > > > > > deployed mode
> > > > > > > > > > > > > and did not find any issues.
> > > > > > > > > > > > > Will try it on a topology generating WARC
> > > > > > > > > > > > > files
> > > > > > > > > > > > > next week to
> > > > > > > > > > > > > check that the
> > > > > > > > > > > > > dependency changes on Hadoop have not broken
> > > > > > > > > > > > > anything.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Have a good week end
> > > > > > > > > > > > >
> > > > > > > > > > > > > Julien
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla
> > > > > > > > > > > > > <rz...@apache.org>
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Short update. Sorted out an issue with the
> > > > > > > > > > > > > > tar.gz/zip with
> > > > > > > > > > > > > > Julien today
> > > > > > > > > > > > > > and re-uploaded them to the nightlies area.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > This new bundle works as expected in my
> > > > > > > > > > > > > > deployment but happy to
> > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > additional feedback before getting up a
> > > > > > > > > > > > > > first
> > > > > > > > > > > > > > release candidate
> > > > > > > > > > > > > > ;-)
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100
> > > > > > > > > > > > > > schrieb Richard
> > > > > > > > > > > > > > Zowalla:
> > > > > > > > > > > > > > > Hi Alexandre,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > we are not in a hurry here :) - take as
> > > > > > > > > > > > > > > much as time you
> > > > > > > > > > > > > > > need.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100
> > > > > > > > > > > > > > > schrieb Alexandre
> > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > Hello Richard,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Okay, I'm more than happy to do that
> > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > my pre-production
> > > > > > > > > > > > > > > > cluster
> > > > > > > > > > > > > > > > (~10 topologies) using the binary
> > > > > > > > > > > > > > > > artifacts.
> > > > > > > > > > > > > > > > Would it be OK if I can use up to end
> > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > this week so that
> > > > > > > > > > > > > > > > I'll be
> > > > > > > > > > > > > > > > able to have enough time to check all
> > > > > > > > > > > > > > > > potential issues that
> > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > upgrade bring?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard
> > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > <rz...@apache.org> a
> > > > > > > > > > > > > > > > écrit :
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > before starting a release of 2.6.0-
> > > > > > > > > > > > > > > > > SNAPSHOT, I would like
> > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > some feedback on the current SNAPSHOT
> > > > > > > > > > > > > > > > > build.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of
> > > > > > > > > > > > > > > > > Storm build from
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > It is available via the ASF Snapshot
> > > > > > > > > > > > > > > > > repository:
> > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > You can consume by adding
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > <repository>
> > > > > > > > > > > > > > > > > <id>apache.snapshots</id>
> > > > > > > > > > > > > > > > > <name>Apache Snapshot
> > > > > > > > > > > > > > > > > Repository</name>
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > <url>
> > > > > > > > > > > > > > > > > https://repository.apache.org/snap
> > > > > > > > > > > > > > > > > shots</url>
> > > > > > > > > > > > > > > > > <releases>
> > > > > > > > > > > > > > > > > <enabled>false</enabled>
> > > > > > > > > > > > > > > > > </releases>
> > > > > > > > > > > > > > > > > </repository>
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > to your project pom. As we do not
> > > > > > > > > > > > > > > > > deploy SNAPSHOTS
> > > > > > > > > > > > > > > > > automatically,
> > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > should be easy to just consume the
> > > > > > > > > > > > > > > > > latest SNAPSHOT.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > The packaged binaries are available
> > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > nightlies.apache.org:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > If you have some minutes left: Please
> > > > > > > > > > > > > > > > > test and report any
> > > > > > > > > > > > > > > > > issues
> > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > this binaries, so we can fix before
> > > > > > > > > > > > > > > > > attempting to
> > > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > The most significant changes are in
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > are of
> > > > > > > > > > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > upgraded from 2.x to 3.x - our own
> > > > > > > > > > > > > > > > > test
> > > > > > > > > > > > > > > > > coverage within
> > > > > > > > > > > > > > > > > the build
> > > > > > > > > > > > > > > > > looks
> > > > > > > > > > > > > > > > > good but would be nice to get some
> > > > > > > > > > > > > > > > > real world use-case
> > > > > > > > > > > > > > > > > feedback
> > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > this. In addition, we had quite a lof
> > > > > > > > > > > > > > > > > of 3rd party
> > > > > > > > > > > > > > > > > dependency
> > > > > > > > > > > > > > > > > upgrades.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > In addition, it contains the pruning
> > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > external modules
> > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > listed
> > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > A summaryin Jira is here:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > >
> > > > > > > > > > > > > *Open Source Solutions for Text Engineering*
> > > > > > > > > > > > >
> > > > > > > > > > > > > http://www.digitalpebble.com
> > > > > > > > > > > > > http://digitalpebble.blogspot.com/
> > > > > > > > > > > > > #digitalpebble
> > > > > > > > > > > > > <http://twitter.com/digitalpebble>
> > > > > > > > > >
> >
>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Richard Zowalla <rz...@apache.org>.
You pushed on your own branch. You would need to push to apache/storm
main branch or open a PR ;-)
Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb Alexandre
Vermeerbergen:
> Just did a 'git push' with that trivial fix... is that enough?
>
> Le sam. 4 nov. 2023 à 09:34, Richard Zowalla <ri...@zowalla.com> a
> écrit :
> >
> > You can do as you like ;-)
> >
> > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen
> > <av...@gmail.com>:
> > > I mean :
> > >
> > > Can I keep using my
> > > 'STORM-3958-Capacity-to-set-St
> > > orm-UI's-title-in-conf/storm.yaml'
> > > branch to push this fix, or should I create a new one?
> > >
> > > Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > <av...@gmail.com> a écrit :
> > > >
> > > > Can you keep using my
> > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > conf/storm.yaml'
> > > > branch to push this fix, or should I create a new one?
> > > >
> > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > <ri...@zowalla.com> a écrit :
> > > > >
> > > > > Just push a fix for it (referencing the original issue) ;-) -
> > > > > we hadn't a release in between, so not a big deal.
> > > > >
> > > > > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre
> > > > > Vermeerbergen <av...@gmail.com>:
> > > > > > Temporary workaround for the Storm UI title's regression:
> > > > > > run this
> > > > > > command on the title-template.html file (in
> > > > > > storm-stable/public/templates directory) after having
> > > > > > extracted the
> > > > > > binary archive:
> > > > > >
> > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > > > > >
> > > > > > Again sorry for having introduced this regression.
> > > > > >
> > > > > > Alexandre
> > > > > >
> > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > > > > > <av...@gmail.com> a écrit :
> > > > > > >
> > > > > > > Updates regarding my test with Storm 2.6.0 Snapshot
> > > > > > > 2023/11/02:
> > > > > > > 1. I have modified my KO bolts using this pattern in
> > > > > > > their prepare() method:
> > > > > > >
> > > > > > > long cacheMaxSize =
> > > > > > > Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE)
> > > > > > > .toString());
> > > > > > >
> > > > > > > instead of:
> > > > > > >
> > > > > > > long cacheMaxSize = (Long)
> > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > >
> > > > > > > 2. I have found the cause of the regression in loss of
> > > > > > > 'Storm UI'
> > > > > > > title at the top left-hand side of all Storm UI's pages.
> > > > > > > This is a mistake of mine in this PR:
> > > > > > >
> > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > > > > More precisely, at line 19 of this file:
> > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > > > > > we must have:
> > > > > > > <h1><a href="/">{{title}}</a></h1>
> > > > > > >
> > > > > > > instead of:
> > > > > > >
> > > > > > > <h1><a href="/">{{ui.title}}</a></h1>
> > > > > > >
> > > > > > > I think I missed this during my tests because of
> > > > > > > browser's cache
> > > > > > > (bad excuse, I know...): probably I tested by-patching my
> > > > > > > deployed
> > > > > > > Storm cluster, and I finally forgot to checkout this
> > > > > > > latest change of
> > > > > > > my delivery.
> > > > > > >
> > > > > > > I had to use "private navigation" to make sure this
> > > > > > > time it fixes this issue
> > > > > > >
> > > > > > > What would be the quickest way for this very simple fix
> > > > > > > to be delivered ?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Alexandre
> > > > > > >
> > > > > > > Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> > > > > > > <av...@gmail.com> a écrit :
> > > > > > > >
> > > > > > > > Thanks Richard for the analysis.
> > > > > > > >
> > > > > > > > My thought on this are very simple: let me fix my own
> > > > > > > > old code, after
> > > > > > > > all this was impacting only one of my topologies, the
> > > > > > > > other one were
> > > > > > > > using more careful Integer.parseInt or
> > > > > > > > Integer.parseLong to avoid
> > > > > > > > making assumption on the way the values in the config
> > > > > > > > Map were
> > > > > > > > serialized.
> > > > > > > >
> > > > > > > > My other concern with 2.6.0 Snapshot is that in Storm
> > > > > > > > UI, the "Storm
> > > > > > > > UI" title at the top left hand side of each page is
> > > > > > > > missing. This may
> > > > > > > > be the result of one of my own PR, I'll have to check
> > > > > > > > why...
> > > > > > > >
> > > > > > > > Alex
> > > > > > > >
> > > > > > > > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla
> > > > > > > > <rz...@apache.org> a écrit :
> > > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > thanks for the feedback.
> > > > > > > > >
> > > > > > > > > The main difference to 2.5.0 is, that we switched the
> > > > > > > > > JSON
> > > > > > > > > implementation in Utils, which relied on a
> > > > > > > > > super old JSON library. What is happening here is,
> > > > > > > > > that the new JSON
> > > > > > > > > library (net.minidev) behaves differently.
> > > > > > > > >
> > > > > > > > > We can see it by writing some simple code without
> > > > > > > > > Storm (The related
> > > > > > > > > code line with the parsing is taken from Storm Utils)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Map<String, Long> conf = new HashMap<>();
> > > > > > > > > conf.put("test", 100000000L);
> > > > > > > > >
> > > > > > > > > Map<String, Object> deserTopoConf = (Map<String,
> > > > > > > > > Object>)
> > > > > > > > > JSONValue.parseWithException(JSONValue.toJSONString(c
> > > > > > > > > onf));
> > > > > > > > >
> > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > >
> > > > > > > > > System.out.println(e.getValue().getClass().getSimpleN
> > > > > > > > > ame());
> > > > > > > > > }
> > > > > > > > >
> > > > > > > > > This will print "Integer". Using "10000000000000000L"
> > > > > > > > > as a value will
> > > > > > > > > result in "Long".
> > > > > > > > >
> > > > > > > > > The main issue derives from the fact, that we
> > > > > > > > > serialize the config map
> > > > > > > > > into a JSON String and parse the result.
> > > > > > > > > Based on the String representation, "100000000L" will
> > > > > > > > > still fit into an
> > > > > > > > > Integer, so the parser will not use Long ;-)
> > > > > > > > >
> > > > > > > > > If the number is big enough, the parser will switch
> > > > > > > > > to a Long> >> > >> > > The old library (> 10 years
> > > > > > > > > old now):
> > > > > > > > >
> > > > > > > > > deserTopoConf = (Map<String, Object>)
> > > > > > > > > org.json.simple.JSONValue.parseWithException(org.json
> > > > > > > > > .simple.JSONValue.
> > > > > > > > > toJSONString(conf));
> > > > > > > > >
> > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > >
> > > > > > > > > System.out.println(e.getValue().getClass().getSimpleN
> > > > > > > > > ame());
> > > > > > > > > }
> > > > > > > > >
> > > > > > > > > This will print "Long".
> > > > > > > > >
> > > > > > > > > The difference between these libraries is, that the
> > > > > > > > > super old
> > > > > > > > > org.json.simple just always returns Long as the type.
> > > > > > > > >
> > > > > > > > > If can eventually tune the new JSON parser to
> > > > > > > > > something like:
> > > > > > > > >
> > > > > > > > > deserTopoConf = (Map<String, Object>) new
> > > > > > > > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(J
> > > > > > > > > SONValue= toJSONStr
> > > > > > > > > ing(conf), JSONValue.defaultReader.DEFAULT);
> > > > > > > > >
> > > > > > > > > This will result in "Long" but might have other side-
> > > > > > > > > effects.
> > > > > > > > >
> > > > > > > > > From my point of view, the newer JSON library works
> > > > > > > > > as expected
> > > > > > > > > (although it might break backward compatibility).
> > > > > > > > > Unchecked casts are always evil, imho. In your
> > > > > > > > > specific case, it might
> > > > > > > > > be more save to cast to "Number" and than invoke
> > > > > > > > > "longValue()" instead.
> > > > > > > > >
> > > > > > > > > Nevertheless, we might being able to restore the old
> > > > > > > > > behaviour by
> > > > > > > > > tuning the JSONParser configuration.
> > > > > > > > >
> > > > > > > > > Thoughts?
> > > > > > > > >
> > > > > > > > > Gruß
> > > > > > > > > Richard
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb
> > > > > > > > > Alexandre
> > > > > > > > > Vermeerbergen:
> > > > > > > > > > On a side note, the Long value that "MyKOBolt" is
> > > > > > > > > > set by a
> > > > > > > > > > DefaultTopologyConfig to a value with is a
> > > > > > > > > > java.lang.Long instance...
> > > > > > > > > > hence it's surprising to see it internally
> > > > > > > > > > converted into a
> > > > > > > > > > java.lang.Integer in the Map passed to prepare()
> > > > > > > > > > method...
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Le ven. 3 nov. 2023 à 18:27, Alexandre
> > > > > > > > > > Vermeerbergen
> > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > >
> > > > > > > > > > > Hello,
> > > > > > > > > > >
> > > > > > > > > > > I found a breaking change using Apache Storm
> > > > > > > > > > > 2.6.0 snapshot build
> > > > > > > > > > > dated 20231103 and without changing my
> > > > > > > > > > > topologies' code (keeping
> > > > > > > > > > > the
> > > > > > > > > > > same big jars).
> > > > > > > > > > >
> > > > > > > > > > > This seems to be tied to a datatype change in Map
> > > > > > > > > > > argument of
> > > > > > > > > > > prepare() method of Bolt.
> > > > > > > > > > >
> > > > > > > > > > > For example, this Bolt is KO at runtime:
> > > > > > > > > > >
> > > > > > > > > > > public class MyKOBolt extends BaseRichBolt {
> > > > > > > > > > >
> > > > > > > > > > > @Override
> > > > > > > > > > > public void prepare(Map stormConf,
> > > > > > > > > > > TopologyContext context,
> > > > > > > > > > > OutputCollector collector) {
> > > > > > > > > > > cacheMaxSize = (Long)
> > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > > > > cacheTtlMin = (Long)
> > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > > > > > > > > > >
> > > > > > > > > > > as it throws this error:
> > > > > > > > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > > > > > > > > > > Thread-14-__system-executor[-1, -1] [INFO]
> > > > > > > > > > > Preparing bolt
> > > > > > > > > > > __system:[-1]
> > > > > > > > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > > > > > > > > > Thread-15-evaluateTriggers-executor[11, 11]
> > > > > > > > > > > [ERROR] Async loop
> > > > > > > > > > > died!
> > > > > > > > > > > java.lang.ClassCastException: java.lang.Integer
> > > > > > > > > > > incompatible with
> > > > > > > > > > > java.lang.Long
> > > > > > > > > > > at com.acme.storm.alerting.MyKOBolt
> > > > > > > > > > > .prepare(EvaluationBolt.java:71)
> > > > > > > > > > > ~[stormjar.jar:?]
> > > > > > > > > > > at
> > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.init(
> > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > 128)
> > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > > > > > > > > at
> > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.call(
> > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > 138)
> > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > > > > > > > > at
> > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.call(
> > > > > > > > > > > BoltExecutor.java:
> > > > > > > > > > > 54)
> > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > > > > > > > > at
> > > > > > > > > > > org.apache.storm.utils.Utils$1.run(Utils.java:393
> > > > > > > > > > > )
> > > > > > > > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > > > > > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> > > > > > > > > > >
> > > > > > > > > > > whereas this bolt is fine with Storm 2.6.0:
> > > > > > > > > > >
> > > > > > > > > > > public class MyOKBolt extends BaseBasicBolt {
> > > > > > > > > > >
> > > > > > > > > > > @Override
> > > > > > > > > > > public void prepare(final Map stormConf,
> > > > > > > > > > > final TopologyContext
> > > > > > > > > > > context) {
> > > > > > > > > > > super.prepare(stormConf, context);
> > > > > > > > > > >
> > > > > > > > > > > this.redisPort =
> > > > > > > > > > > Integer.parseInt((String)
> > > > > > > > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > > > > > > > > > >
> > > > > > > > > > > That said, I can modify the code of "MyKOBolt" to
> > > > > > > > > > > use same
> > > > > > > > > > > Integer.parseInt or Integer.parseLong trick, but
> > > > > > > > > > > it's the first
> > > > > > > > > > > time
> > > > > > > > > > > in my long history of upgrades that I have seen
> > > > > > > > > > > such runtime
> > > > > > > > > > > incompatibility.
> > > > > > > > > > >
> > > > > > > > > > > Maybe "MyKOBolt" was badly written since the
> > > > > > > > > > > beginning and I have
> > > > > > > > > > > just
> > > > > > > > > > > hit the punishment for it: is there a
> > > > > > > > > > > documentation which clarifies
> > > > > > > > > > > datatypes of the Map argument of prepare()
> > > > > > > > > > > method?
> > > > > > > > > > >
> > > > > > > > > > > Also worth noting: "MyKOBolt" derives from
> > > > > > > > > > > BaseRichBolt , while
> > > > > > > > > > > "MyOKBolt" derives from BaseBasicBolt => could
> > > > > > > > > > > this have any impact
> > > > > > > > > > > on
> > > > > > > > > > > this finding ?
> > > > > > > > > > >
> > > > > > > > > > > Last, I'm running this on Redhat Linux 8 and IBM
> > > > > > > > > > > Semeru JDK
> > > > > > > > > > > 17.0.8.1.
> > > > > > > > > > >
> > > > > > > > > > > Kind regards,
> > > > > > > > > > > Alexandre
> > > > > > > > > > >
> > > > > > > > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > > > > > > > > > > <li...@gmail.com> a écrit :
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks Richard.
> > > > > > > > > > > >
> > > > > > > > > > > > Tried the latest snapshot with StormCrawler
> > > > > > > > > > > > both in local and
> > > > > > > > > > > > deployed mode
> > > > > > > > > > > > and did not find any issues.
> > > > > > > > > > > > Will try it on a topology generating WARC files
> > > > > > > > > > > > next week to
> > > > > > > > > > > > check that the
> > > > > > > > > > > > dependency changes on Hadoop have not broken
> > > > > > > > > > > > anything.
> > > > > > > > > > > >
> > > > > > > > > > > > Have a good week end
> > > > > > > > > > > >
> > > > > > > > > > > > Julien
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla
> > > > > > > > > > > > <rz...@apache.org>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Short update. Sorted out an issue with the
> > > > > > > > > > > > > tar.gz/zip with
> > > > > > > > > > > > > Julien today
> > > > > > > > > > > > > and re-uploaded them to the nightlies area.
> > > > > > > > > > > > >
> > > > > > > > > > > > > This new bundle works as expected in my
> > > > > > > > > > > > > deployment but happy to
> > > > > > > > > > > > > receive
> > > > > > > > > > > > > additional feedback before getting up a first
> > > > > > > > > > > > > release candidate
> > > > > > > > > > > > > ;-)
> > > > > > > > > > > > >
> > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100
> > > > > > > > > > > > > schrieb Richard
> > > > > > > > > > > > > Zowalla:
> > > > > > > > > > > > > > Hi Alexandre,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > we are not in a hurry here :) - take as
> > > > > > > > > > > > > > much as time you
> > > > > > > > > > > > > > need.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100
> > > > > > > > > > > > > > schrieb Alexandre
> > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > Hello Richard,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Okay, I'm more than happy to do that with
> > > > > > > > > > > > > > > my pre-production
> > > > > > > > > > > > > > > cluster
> > > > > > > > > > > > > > > (~10 topologies) using the binary
> > > > > > > > > > > > > > > artifacts.
> > > > > > > > > > > > > > > Would it be OK if I can use up to end of
> > > > > > > > > > > > > > > this week so that
> > > > > > > > > > > > > > > I'll be
> > > > > > > > > > > > > > > able to have enough time to check all
> > > > > > > > > > > > > > > potential issues that
> > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > upgrade bring?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard
> > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > <rz...@apache.org> a
> > > > > > > > > > > > > > > écrit :
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > before starting a release of 2.6.0-
> > > > > > > > > > > > > > > > SNAPSHOT, I would like
> > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > some feedback on the current SNAPSHOT
> > > > > > > > > > > > > > > > build.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of
> > > > > > > > > > > > > > > > Storm build from
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > It is available via the ASF Snapshot
> > > > > > > > > > > > > > > > repository:
> > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > You can consume by adding
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > <repository>
> > > > > > > > > > > > > > > > <id>apache.snapshots</id>
> > > > > > > > > > > > > > > > <name>Apache Snapshot
> > > > > > > > > > > > > > > > Repository</name>
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > <url>https://repository.apache.org/snap
> > > > > > > > > > > > > > > > shots</url>
> > > > > > > > > > > > > > > > <releases>
> > > > > > > > > > > > > > > > <enabled>false</enabled>
> > > > > > > > > > > > > > > > </releases>
> > > > > > > > > > > > > > > > </repository>
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > to your project pom. As we do not
> > > > > > > > > > > > > > > > deploy SNAPSHOTS
> > > > > > > > > > > > > > > > automatically,
> > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > should be easy to just consume the
> > > > > > > > > > > > > > > > latest SNAPSHOT.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The packaged binaries are available at
> > > > > > > > > > > > > > > > nightlies.apache.org:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > If you have some minutes left: Please
> > > > > > > > > > > > > > > > test and report any
> > > > > > > > > > > > > > > > issues
> > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > this binaries, so we can fix before
> > > > > > > > > > > > > > > > attempting to
> > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The most significant changes are in the
> > > > > > > > > > > > > > > > are of
> > > > > > > > > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > upgraded from 2.x to 3.x - our own test
> > > > > > > > > > > > > > > > coverage within
> > > > > > > > > > > > > > > > the build
> > > > > > > > > > > > > > > > looks
> > > > > > > > > > > > > > > > good but would be nice to get some
> > > > > > > > > > > > > > > > real world use-case
> > > > > > > > > > > > > > > > feedback
> > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > this. In addition, we had quite a lof
> > > > > > > > > > > > > > > > of 3rd party
> > > > > > > > > > > > > > > > dependency
> > > > > > > > > > > > > > > > upgrades.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > In addition, it contains the pruning of
> > > > > > > > > > > > > > > > external modules
> > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > listed
> > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > A summaryin Jira is here:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > >
> > > > > > > > > > > > *Open Source Solutions for Text Engineering*
> > > > > > > > > > > >
> > > > > > > > > > > > http://www.digitalpebble.com
> > > > > > > > > > > > http://digitalpebble.blogspot.com/
> > > > > > > > > > > > #digitalpebble
> > > > > > > > > > > > <http://twitter.com/digitalpebble>
> > > > > > > > >
>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Interestingly enough, to clean our topologies' code from its use of
Storm shaded classes, I removed storm-shaded-deps.jar from our
dependencies... only to find that in such case we have a compilation
error:
The type org.apache.storm.thrift.TException cannot be resolved. It is
indirectly referenced from required type
org.apache.storm.StormSubmitter
from this type of code (at the last line)
// ----------------------------
// Submit Topology
String displayName =
TopologyHelper.getTopologyDisplayName(topologyShortName,
kafkaBrokersHosts);
StormSubmitter.submitTopology(displayName, topologyConfig,
builder.createTopology());
}
I would have excepted such indirect dependency to be part of
non-shared storm JARs... am I missing something?
Note: in the meantime, let me re-introduce this dependency and use
"find in code" to fix uses of shaded classes..
Thanks,
Alexandre
Le sam. 4 nov. 2023 à 20:57, Alexandre Vermeerbergen
<av...@gmail.com> a écrit :
>
> Yes another regression I found with my topologies (built with 2.5.0
> dependencies) running to Storm 2.6.0 SNAPSHOT 2023/11/02: I have seen
> several exceptions like this one:
>
> 2023-11-04 18:43:31.696 o.a.s.e.e.ReportError
> Thread-16-alertsHistoryUMToKafka-executor[27, 27] [ERROR] Error
> java.lang.RuntimeException: java.lang.NoClassDefFoundError:
> org.apache.storm.shade.org.json.simple.JSONObject
> at org.apache.storm.utils.Utils$1.run(Utils.java:413)
> ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> at java.lang.Thread.run(Thread.java:857) [?:?]
> Caused by: java.lang.NoClassDefFoundError:
> org.apache.storm.shade.org.json.simple.JSONObject
> at com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessageFromTuple(AlertHistoryTupleToKafkaMapper.java:25)
> ~[stormjar.jar:?]
> at com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessageFromTuple(AlertHistoryTupleToKafkaMapper.java:11)
> ~[stormjar.jar:?]
> at org.apache.storm.kafka.bolt.KafkaBolt.process(KafkaBolt.java:177)
> ~[stormjar.jar:?]
>
> and here's the related source code:
>
> package com.acme.storm.evaluator.bolt;
>
> import org.apache.storm.kafka.bolt.mapper.TupleToKafkaMapper;
> import org.apache.storm.shade.org.json.simple.JSONObject;
> import org.apache.storm.tuple.Tuple;
> import org.slf4j.Logger;
> import org.slf4j.LoggerFactory;
>
> import com.acme.storm.util.TupleAlertHistoryFields;
>
> public class AlertHistoryTupleToKafkaMapper implements
> TupleToKafkaMapper<String, String> {
> private static final long serialVersionUID = 5668103889461721134L;
>
> public static final Logger logger =
> LoggerFactory.getLogger(AlertHistoryTupleToKafkaMapper.class);
>
> @Override
> public String getKeyFromTuple(Tuple tuple) {
> return tuple.getStringByField(TupleAlertHistoryFields.SERVICE_DEFINITION_NAME);
> }
>
> @SuppressWarnings("unchecked")
> @Override
> public String getMessageFromTuple(Tuple tuple)
> {
> JSONObject tupleJson = new JSONObject();
>
> tuple.getFields().forEach(f -> {
> tupleJson.put(f, tuple.getValueByField(f));
> });
>
> String jsonString = tupleJson.toJSONString();
>
> return jsonString;
> }
> }
>
> line #25 is this one:
>
> JSONObject tupleJson = new JSONObject();
>
> so, from what I understand, Storm 2.6.0 no longer exposes some shaded
> classes such as:
>
> org.apache.storm.shade.org.json.simple.JSONObject;
>
> Interestingly, it looks similar to this issue:
> https://stackoverflow.com/questions/33377291/java-lang-noclassdeffounderror-org-json-simple-jsonvalue-at-storm-kafka-dynamic
>
> Whatever, I guess I can only blame our own code for using Storm's
> shaded classes, right ?
> Note it's been running as is for long time and not breaking since many
> upgrades, of course it's not an excuse...
>
> I'll fix our code & continue checking if I see other impacts of 2.5.0
> -> 2.6.0 upgrade...
>
> Thanks,
> Alexandre
>
> Le sam. 4 nov. 2023 à 10:36, Alexandre Vermeerbergen
> <av...@gmail.com> a écrit :
> >
> > Just did a 'git push' with that trivial fix... is that enough?
> >
> > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla <ri...@zowalla.com> a écrit :
> > >
> > > You can do as you like ;-)
> > >
> > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen <av...@gmail.com>:
> > > >I mean :
> > > >
> > > >Can I keep using my
> > > >'STORM-3958-Capacity-to-set-St
> > > >orm-UI's-title-in-conf/storm.yaml'
> > > >branch to push this fix, or should I create a new one?
> > > >
> > > >Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > ><av...@gmail.com> a écrit :
> > > >>
> > > >> Can you keep using my
> > > >> 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-conf/storm.yaml'
> > > >> branch to push this fix, or should I create a new one?
> > > >>
> > > >> Le sam. 4 nov. 2023 à 08:14, Richard Zowalla <ri...@zowalla.com> a écrit :
> > > >> >
> > > >> > Just push a fix for it (referencing the original issue) ;-) - we hadn't a release in between, so not a big deal.
> > > >> >
> > > >> > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre Vermeerbergen <av...@gmail.com>:
> > > >> > >Temporary workaround for the Storm UI title's regression: run this
> > > >> > >command on the title-template.html file (in
> > > >> > >storm-stable/public/templates directory) after having extracted the
> > > >> > >binary archive:
> > > >> > >
> > > >> > >sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > > >> > >
> > > >> > >Again sorry for having introduced this regression.
> > > >> > >
> > > >> > >Alexandre
> > > >> > >
> > > >> > >Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > > >> > ><av...@gmail.com> a écrit :
> > > >> > >>
> > > >> > >> Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
> > > >> > >> 1. I have modified my KO bolts using this pattern in their prepare() method:
> > > >> > >>
> > > >> > >> long cacheMaxSize =
> > > >> > >> Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
> > > >> > >>
> > > >> > >> instead of:
> > > >> > >>
> > > >> > >> long cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > >> > >>
> > > >> > >> 2. I have found the cause of the regression in loss of 'Storm UI'
> > > >> > >> title at the top left-hand side of all Storm UI's pages.
> > > >> > >> This is a mistake of mine in this PR:
> > > >> > >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > >> > >> More precisely, at line 19 of this file:
> > > >> > >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > >> > >> we must have:
> > > >> > >> <h1><a href="/">{{title}}</a></h1>
> > > >> > >>
> > > >> > >> instead of:
> > > >> > >>
> > > >> > >> <h1><a href="/">{{ui.title}}</a></h1>
> > > >> > >>
> > > >> > >> I think I missed this during my tests because of browser's cache
> > > >> > >> (bad excuse, I know...): probably I tested by-patching my deployed
> > > >> > >> Storm cluster, and I finally forgot to checkout this latest change of
> > > >> > >> my delivery.
> > > >> > >>
> > > >> > >> I had to use "private navigation" to make sure this time it fixes this issue
> > > >> > >>
> > > >> > >> What would be the quickest way for this very simple fix to be delivered ?
> > > >> > >>
> > > >> > >> Thanks,
> > > >> > >> Alexandre
> > > >> > >>
> > > >> > >> Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> > > >> > >> <av...@gmail.com> a écrit :
> > > >> > >> >
> > > >> > >> > Thanks Richard for the analysis.
> > > >> > >> >
> > > >> > >> > My thought on this are very simple: let me fix my own old code, after
> > > >> > >> > all this was impacting only one of my topologies, the other one were
> > > >> > >> > using more careful Integer.parseInt or Integer.parseLong to avoid
> > > >> > >> > making assumption on the way the values in the config Map were
> > > >> > >> > serialized.
> > > >> > >> >
> > > >> > >> > My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
> > > >> > >> > UI" title at the top left hand side of each page is missing. This may
> > > >> > >> > be the result of one of my own PR, I'll have to check why...
> > > >> > >> >
> > > >> > >> > Alex
> > > >> > >> >
> > > >> > >> > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla <rz...@apache.org> a écrit :
> > > >> > >> > >
> > > >> > >> > > Hi,
> > > >> > >> > >
> > > >> > >> > > thanks for the feedback.
> > > >> > >> > >
> > > >> > >> > > The main difference to 2.5.0 is, that we switched the JSON
> > > >> > >> > > implementation in Utils, which relied on a
> > > >> > >> > > super old JSON library. What is happening here is, that the new JSON
> > > >> > >> > > library (net.minidev) behaves differently.
> > > >> > >> > >
> > > >> > >> > > We can see it by writing some simple code without Storm (The related
> > > >> > >> > > code line with the parsing is taken from Storm Utils)
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> > > Map<String, Long> conf = new HashMap<>();
> > > >> > >> > > conf.put("test", 100000000L);
> > > >> > >> > >
> > > >> > >> > > Map<String, Object> deserTopoConf = (Map<String, Object>)
> > > >> > >> > > JSONValue.parseWithException(JSONValue.toJSONString(conf));
> > > >> > >> > >
> > > >> > >> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> > > >> > >> > > System.out.println(e.getValue().getClass().getSimpleName());
> > > >> > >> > > }
> > > >> > >> > >
> > > >> > >> > > This will print "Integer". Using "10000000000000000L" as a value will
> > > >> > >> > > result in "Long".
> > > >> > >> > >
> > > >> > >> > > The main issue derives from the fact, that we serialize the config map
> > > >> > >> > > into a JSON String and parse the result.
> > > >> > >> > > Based on the String representation, "100000000L" will still fit into an
> > > >> > >> > > Integer, so the parser will not use Long ;-)
> > > >> > >> > >
> > > >> > >> > > If the number is big enough, the parser will switch to a Long.
> > > >> > >> > > The old library (> 10 years old now):
> > > >> > >> > >
> > > >> > >> > > deserTopoConf = (Map<String, Object>)
> > > >> > >> > > org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
> > > >> > >> > > toJSONString(conf));
> > > >> > >> > >
> > > >> > >> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> > > >> > >> > > System.out.println(e.getValue().getClass().getSimpleName());
> > > >> > >> > > }
> > > >> > >> > >
> > > >> > >> > > This will print "Long".
> > > >> > >> > >
> > > >> > >> > > The difference between these libraries is, that the super old
> > > >> > >> > > org.json.simple just always returns Long as the type.
> > > >> > >> > >
> > > >> > >> > > If can eventually tune the new JSON parser to something like:
> > > >> > >> > >
> > > >> > >> > > deserTopoConf = (Map<String, Object>) new
> > > >> > >> > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
> > > >> > >> > > ing(conf), JSONValue.defaultReader.DEFAULT);
> > > >> > >> > >
> > > >> > >> > > This will result in "Long" but might have other side-effects.
> > > >> > >> > >
> > > >> > >> > > From my point of view, the newer JSON library works as expected
> > > >> > >> > > (although it might break backward compatibility).
> > > >> > >> > > Unchecked casts are always evil, imho. In your specific case, it might
> > > >> > >> > > be more save to cast to "Number" and than invoke "longValue()" instead.
> > > >> > >> > >
> > > >> > >> > > Nevertheless, we might being able to restore the old behaviour by
> > > >> > >> > > tuning the JSONParser configuration.
> > > >> > >> > >
> > > >> > >> > > Thoughts?
> > > >> > >> > >
> > > >> > >> > > Gruß
> > > >> > >> > > Richard
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
> > > >> > >> > > Vermeerbergen:
> > > >> > >> > > > On a side note, the Long value that "MyKOBolt" is set by a
> > > >> > >> > > > DefaultTopologyConfig to a value with is a java.lang.Long instance...
> > > >> > >> > > > hence it's surprising to see it internally converted into a
> > > >> > >> > > > java.lang.Integer in the Map passed to prepare() method...
> > > >> > >> > > >
> > > >> > >> > > >
> > > >> > >> > > > Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
> > > >> > >> > > > <av...@gmail.com> a écrit :
> > > >> > >> > > > >
> > > >> > >> > > > > Hello,
> > > >> > >> > > > >
> > > >> > >> > > > > I found a breaking change using Apache Storm 2.6.0 snapshot build
> > > >> > >> > > > > dated 20231103 and without changing my topologies' code (keeping
> > > >> > >> > > > > the
> > > >> > >> > > > > same big jars).
> > > >> > >> > > > >
> > > >> > >> > > > > This seems to be tied to a datatype change in Map argument of
> > > >> > >> > > > > prepare() method of Bolt.
> > > >> > >> > > > >
> > > >> > >> > > > > For example, this Bolt is KO at runtime:
> > > >> > >> > > > >
> > > >> > >> > > > > public class MyKOBolt extends BaseRichBolt {
> > > >> > >> > > > >
> > > >> > >> > > > > @Override
> > > >> > >> > > > > public void prepare(Map stormConf, TopologyContext context,
> > > >> > >> > > > > OutputCollector collector) {
> > > >> > >> > > > > cacheMaxSize = (Long)
> > > >> > >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > >> > >> > > > > cacheTtlMin = (Long)
> > > >> > >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > > >> > >> > > > >
> > > >> > >> > > > > as it throws this error:
> > > >> > >> > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > > >> > >> > > > > Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
> > > >> > >> > > > > __system:[-1]
> > > >> > >> > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > >> > >> > > > > Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop
> > > >> > >> > > > > died!
> > > >> > >> > > > > java.lang.ClassCastException: java.lang.Integer incompatible with
> > > >> > >> > > > > java.lang.Long
> > > >> > >> > > > > at com.acme.storm.alerting.MyKOBolt
> > > >> > >> > > > > .prepare(EvaluationBolt.java:71)
> > > >> > >> > > > > ~[stormjar.jar:?]
> > > >> > >> > > > > at
> > > >> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:
> > > >> > >> > > > > 128)
> > > >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > >> > >> > > > > at
> > > >> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > > >> > >> > > > > 138)
> > > >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > >> > >> > > > > at
> > > >> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > > >> > >> > > > > 54)
> > > >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > >> > >> > > > > at org.apache.storm.utils.Utils$1.run(Utils.java:393)
> > > >> > >> > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > >> > >> > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> > > >> > >> > > > >
> > > >> > >> > > > > whereas this bolt is fine with Storm 2.6.0:
> > > >> > >> > > > >
> > > >> > >> > > > > public class MyOKBolt extends BaseBasicBolt {
> > > >> > >> > > > >
> > > >> > >> > > > > @Override
> > > >> > >> > > > > public void prepare(final Map stormConf, final TopologyContext
> > > >> > >> > > > > context) {
> > > >> > >> > > > > super.prepare(stormConf, context);
> > > >> > >> > > > >
> > > >> > >> > > > > this.redisPort =
> > > >> > >> > > > > Integer.parseInt((String)
> > > >> > >> > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > > >> > >> > > > >
> > > >> > >> > > > > That said, I can modify the code of "MyKOBolt" to use same
> > > >> > >> > > > > Integer.parseInt or Integer.parseLong trick, but it's the first
> > > >> > >> > > > > time
> > > >> > >> > > > > in my long history of upgrades that I have seen such runtime
> > > >> > >> > > > > incompatibility.
> > > >> > >> > > > >
> > > >> > >> > > > > Maybe "MyKOBolt" was badly written since the beginning and I have
> > > >> > >> > > > > just
> > > >> > >> > > > > hit the punishment for it: is there a documentation which clarifies
> > > >> > >> > > > > datatypes of the Map argument of prepare() method?
> > > >> > >> > > > >
> > > >> > >> > > > > Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
> > > >> > >> > > > > "MyOKBolt" derives from BaseBasicBolt => could this have any impact
> > > >> > >> > > > > on
> > > >> > >> > > > > this finding ?
> > > >> > >> > > > >
> > > >> > >> > > > > Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK
> > > >> > >> > > > > 17.0.8.1.
> > > >> > >> > > > >
> > > >> > >> > > > > Kind regards,
> > > >> > >> > > > > Alexandre
> > > >> > >> > > > >
> > > >> > >> > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > > >> > >> > > > > <li...@gmail.com> a écrit :
> > > >> > >> > > > > >
> > > >> > >> > > > > > Thanks Richard.
> > > >> > >> > > > > >
> > > >> > >> > > > > > Tried the latest snapshot with StormCrawler both in local and
> > > >> > >> > > > > > deployed mode
> > > >> > >> > > > > > and did not find any issues.
> > > >> > >> > > > > > Will try it on a topology generating WARC files next week to
> > > >> > >> > > > > > check that the
> > > >> > >> > > > > > dependency changes on Hadoop have not broken anything.
> > > >> > >> > > > > >
> > > >> > >> > > > > > Have a good week end
> > > >> > >> > > > > >
> > > >> > >> > > > > > Julien
> > > >> > >> > > > > >
> > > >> > >> > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <rz...@apache.org>
> > > >> > >> > > > > > wrote:
> > > >> > >> > > > > >
> > > >> > >> > > > > > > Short update. Sorted out an issue with the tar.gz/zip with
> > > >> > >> > > > > > > Julien today
> > > >> > >> > > > > > > and re-uploaded them to the nightlies area.
> > > >> > >> > > > > > >
> > > >> > >> > > > > > > This new bundle works as expected in my deployment but happy to
> > > >> > >> > > > > > > receive
> > > >> > >> > > > > > > additional feedback before getting up a first release candidate
> > > >> > >> > > > > > > ;-)
> > > >> > >> > > > > > >
> > > >> > >> > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard
> > > >> > >> > > > > > > Zowalla:
> > > >> > >> > > > > > > > Hi Alexandre,
> > > >> > >> > > > > > > >
> > > >> > >> > > > > > > > we are not in a hurry here :) - take as much as time you
> > > >> > >> > > > > > > > need.
> > > >> > >> > > > > > > >
> > > >> > >> > > > > > > > Gruß
> > > >> > >> > > > > > > > Richard
> > > >> > >> > > > > > > >
> > > >> > >> > > > > > > >
> > > >> > >> > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > > >> > >> > > > > > > > Vermeerbergen:
> > > >> > >> > > > > > > > > Hello Richard,
> > > >> > >> > > > > > > > >
> > > >> > >> > > > > > > > > Okay, I'm more than happy to do that with my pre-production
> > > >> > >> > > > > > > > > cluster
> > > >> > >> > > > > > > > > (~10 topologies) using the binary artifacts.
> > > >> > >> > > > > > > > > Would it be OK if I can use up to end of this week so that
> > > >> > >> > > > > > > > > I'll be
> > > >> > >> > > > > > > > > able to have enough time to check all potential issues that
> > > >> > >> > > > > > > > > this
> > > >> > >> > > > > > > > > upgrade bring?
> > > >> > >> > > > > > > > >
> > > >> > >> > > > > > > > > Thanks,
> > > >> > >> > > > > > > > > Alexandre
> > > >> > >> > > > > > > > >
> > > >> > >> > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla
> > > >> > >> > > > > > > > > <rz...@apache.org> a
> > > >> > >> > > > > > > > > écrit :
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > Hi all,
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > before starting a release of 2.6.0-SNAPSHOT, I would like
> > > >> > >> > > > > > > > > > to
> > > >> > >> > > > > > > > > > receive
> > > >> > >> > > > > > > > > > some feedback on the current SNAPSHOT build.
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > It is available via the ASF Snapshot repository:
> > > >> > >> > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > You can consume by adding
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > <repository>
> > > >> > >> > > > > > > > > > <id>apache.snapshots</id>
> > > >> > >> > > > > > > > > > <name>Apache Snapshot Repository</name>
> > > >> > >> > > > > > > > > > <url>https://repository.apache.org/snapshots</url>
> > > >> > >> > > > > > > > > > <releases>
> > > >> > >> > > > > > > > > > <enabled>false</enabled>
> > > >> > >> > > > > > > > > > </releases>
> > > >> > >> > > > > > > > > > </repository>
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > to your project pom. As we do not deploy SNAPSHOTS
> > > >> > >> > > > > > > > > > automatically,
> > > >> > >> > > > > > > > > > it
> > > >> > >> > > > > > > > > > should be easy to just consume the latest SNAPSHOT.
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > The packaged binaries are available at
> > > >> > >> > > > > > > > > > nightlies.apache.org:
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > If you have some minutes left: Please test and report any
> > > >> > >> > > > > > > > > > issues
> > > >> > >> > > > > > > > > > with
> > > >> > >> > > > > > > > > > this binaries, so we can fix before attempting to
> > > >> > >> > > > > > > > > > release.
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > The most significant changes are in the are of
> > > >> > >> > > > > > > > > > hadoop/hbase/hdfs
> > > >> > >> > > > > > > > > > as
> > > >> > >> > > > > > > > > > we
> > > >> > >> > > > > > > > > > upgraded from 2.x to 3.x - our own test coverage within
> > > >> > >> > > > > > > > > > the build
> > > >> > >> > > > > > > > > > looks
> > > >> > >> > > > > > > > > > good but would be nice to get some real world use-case
> > > >> > >> > > > > > > > > > feedback
> > > >> > >> > > > > > > > > > on
> > > >> > >> > > > > > > > > > this. In addition, we had quite a lof of 3rd party
> > > >> > >> > > > > > > > > > dependency
> > > >> > >> > > > > > > > > > upgrades.
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > In addition, it contains the pruning of external modules
> > > >> > >> > > > > > > > > > as
> > > >> > >> > > > > > > > > > listed
> > > >> > >> > > > > > > > > > in
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > A summaryin Jira is here:
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > Gruß
> > > >> > >> > > > > > > > > > Richard
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > >
> > > >> > >> > > > > > >
> > > >> > >> > > > > > >
> > > >> > >> > > > > >
> > > >> > >> > > > > > --
> > > >> > >> > > > > >
> > > >> > >> > > > > > *Open Source Solutions for Text Engineering*
> > > >> > >> > > > > >
> > > >> > >> > > > > > http://www.digitalpebble.com
> > > >> > >> > > > > > http://digitalpebble.blogspot.com/
> > > >> > >> > > > > > #digitalpebble <http://twitter.com/digitalpebble>
> > > >> > >> > >
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Hello Julien,
Thanks for your suggestion to rebuild my code, but as I mentioned, my
first test with Storm 2.6.0 SNAPSHOT was to deploy existing topologies
without rebuilding these topologies.
It turned out to be a worthy experience, given the "old bad practices"
in my topologies' code that this exercice allowed me to find.
Thanks,
Alexandre
Le dim. 5 nov. 2023 à 16:32, Julien Nioche
<li...@gmail.com> a écrit :
>
> Alexandre,
>
> I had a similar issue when trying the initial snapshot made by Richard but
> it seems to have disappeared with the new one.
> Maybe try building your code with *mvn clean package -U *or delete the
> content of* ~/.m2/repository/org/apache/storm/ *beforehand?
>
> Julien
>
> On Sat, 4 Nov 2023 at 19:57, Alexandre Vermeerbergen <
> avermeerbergen@gmail.com> wrote:
>
> > Yes another regression I found with my topologies (built with 2.5.0
> > dependencies) running to Storm 2.6.0 SNAPSHOT 2023/11/02: I have seen
> > several exceptions like this one:
> >
> > 2023-11-04 18:43:31.696 o.a.s.e.e.ReportError
> > Thread-16-alertsHistoryUMToKafka-executor[27, 27] [ERROR] Error
> > java.lang.RuntimeException: java.lang.NoClassDefFoundError:
> > org.apache.storm.shade.org.json.simple.JSONObject
> > at org.apache.storm.utils.Utils$1.run(Utils.java:413)
> > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > at java.lang.Thread.run(Thread.java:857) [?:?]
> > Caused by: java.lang.NoClassDefFoundError:
> > org.apache.storm.shade.org.json.simple.JSONObject
> > at
> > com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessageFromTuple(AlertHistoryTupleToKafkaMapper.java:25)
> > ~[stormjar.jar:?]
> > at
> > com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessageFromTuple(AlertHistoryTupleToKafkaMapper.java:11)
> > ~[stormjar.jar:?]
> > at org.apache.storm.kafka.bolt.KafkaBolt.process(KafkaBolt.java:177)
> > ~[stormjar.jar:?]
> >
> > and here's the related source code:
> >
> > package com.acme.storm.evaluator.bolt;
> >
> > import org.apache.storm.kafka.bolt.mapper.TupleToKafkaMapper;
> > import org.apache.storm.shade.org.json.simple.JSONObject;
> > import org.apache.storm.tuple.Tuple;
> > import org.slf4j.Logger;
> > import org.slf4j.LoggerFactory;
> >
> > import com.acme.storm.util.TupleAlertHistoryFields;
> >
> > public class AlertHistoryTupleToKafkaMapper implements
> > TupleToKafkaMapper<String, String> {
> > private static final long serialVersionUID = 5668103889461721134L;
> >
> > public static final Logger logger =
> > LoggerFactory.getLogger(AlertHistoryTupleToKafkaMapper.class);
> >
> > @Override
> > public String getKeyFromTuple(Tuple tuple) {
> > return
> > tuple.getStringByField(TupleAlertHistoryFields.SERVICE_DEFINITION_NAME);
> > }
> >
> > @SuppressWarnings("unchecked")
> > @Override
> > public String getMessageFromTuple(Tuple tuple)
> > {
> > JSONObject tupleJson = new JSONObject();
> >
> > tuple.getFields().forEach(f -> {
> > tupleJson.put(f, tuple.getValueByField(f));
> > });
> >
> > String jsonString = tupleJson.toJSONString();
> >
> > return jsonString;
> > }
> > }
> >
> > line #25 is this one:
> >
> > JSONObject tupleJson = new JSONObject();
> >
> > so, from what I understand, Storm 2.6.0 no longer exposes some shaded
> > classes such as:
> >
> > org.apache.storm.shade.org.json.simple.JSONObject;
> >
> > Interestingly, it looks similar to this issue:
> >
> > https://stackoverflow.com/questions/33377291/java-lang-noclassdeffounderror-org-json-simple-jsonvalue-at-storm-kafka-dynamic
> >
> > Whatever, I guess I can only blame our own code for using Storm's
> > shaded classes, right ?
> > Note it's been running as is for long time and not breaking since many
> > upgrades, of course it's not an excuse...
> >
> > I'll fix our code & continue checking if I see other impacts of 2.5.0
> > -> 2.6.0 upgrade...
> >
> > Thanks,
> > Alexandre
> >
> > Le sam. 4 nov. 2023 à 10:36, Alexandre Vermeerbergen
> > <av...@gmail.com> a écrit :
> > >
> > > Just did a 'git push' with that trivial fix... is that enough?
> > >
> > > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla <ri...@zowalla.com> a
> > écrit :
> > > >
> > > > You can do as you like ;-)
> > > >
> > > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen <
> > avermeerbergen@gmail.com>:
> > > > >I mean :
> > > > >
> > > > >Can I keep using my
> > > > >'STORM-3958-Capacity-to-set-St
> > > > >orm-UI's-title-in-conf/storm.yaml'
> > > > >branch to push this fix, or should I create a new one?
> > > > >
> > > > >Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > > ><av...@gmail.com> a écrit :
> > > > >>
> > > > >> Can you keep using my
> > > > >> 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-conf/storm.yaml'
> > > > >> branch to push this fix, or should I create a new one?
> > > > >>
> > > > >> Le sam. 4 nov. 2023 à 08:14, Richard Zowalla <ri...@zowalla.com>
> > a écrit :
> > > > >> >
> > > > >> > Just push a fix for it (referencing the original issue) ;-) - we
> > hadn't a release in between, so not a big deal.
> > > > >> >
> > > > >> > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre Vermeerbergen <
> > avermeerbergen@gmail.com>:
> > > > >> > >Temporary workaround for the Storm UI title's regression: run
> > this
> > > > >> > >command on the title-template.html file (in
> > > > >> > >storm-stable/public/templates directory) after having extracted
> > the
> > > > >> > >binary archive:
> > > > >> > >
> > > > >> > >sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > > > >> > >
> > > > >> > >Again sorry for having introduced this regression.
> > > > >> > >
> > > > >> > >Alexandre
> > > > >> > >
> > > > >> > >Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > > > >> > ><av...@gmail.com> a écrit :
> > > > >> > >>
> > > > >> > >> Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
> > > > >> > >> 1. I have modified my KO bolts using this pattern in their
> > prepare() method:
> > > > >> > >>
> > > > >> > >> long cacheMaxSize =
> > > > >> > >>
> > Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
> > > > >> > >>
> > > > >> > >> instead of:
> > > > >> > >>
> > > > >> > >> long cacheMaxSize = (Long)
> > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > >> > >>
> > > > >> > >> 2. I have found the cause of the regression in loss of 'Storm
> > UI'
> > > > >> > >> title at the top left-hand side of all Storm UI's pages.
> > > > >> > >> This is a mistake of mine in this PR:
> > > > >> > >>
> > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > >> > >> More precisely, at line 19 of this file:
> > > > >> > >>
> > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > > >> > >> we must have:
> > > > >> > >> <h1><a href="/">{{title}}</a></h1>
> > > > >> > >>
> > > > >> > >> instead of:
> > > > >> > >>
> > > > >> > >> <h1><a href="/">{{ui.title}}</a></h1>
> > > > >> > >>
> > > > >> > >> I think I missed this during my tests because of browser's
> > cache
> > > > >> > >> (bad excuse, I know...): probably I tested by-patching my
> > deployed
> > > > >> > >> Storm cluster, and I finally forgot to checkout this latest
> > change of
> > > > >> > >> my delivery.
> > > > >> > >>
> > > > >> > >> I had to use "private navigation" to make sure this time it
> > fixes this issue
> > > > >> > >>
> > > > >> > >> What would be the quickest way for this very simple fix to be
> > delivered ?
> > > > >> > >>
> > > > >> > >> Thanks,
> > > > >> > >> Alexandre
> > > > >> > >>
> > > > >> > >> Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> > > > >> > >> <av...@gmail.com> a écrit :
> > > > >> > >> >
> > > > >> > >> > Thanks Richard for the analysis.
> > > > >> > >> >
> > > > >> > >> > My thought on this are very simple: let me fix my own old
> > code, after
> > > > >> > >> > all this was impacting only one of my topologies, the other
> > one were
> > > > >> > >> > using more careful Integer.parseInt or Integer.parseLong to
> > avoid
> > > > >> > >> > making assumption on the way the values in the config Map
> > were
> > > > >> > >> > serialized.
> > > > >> > >> >
> > > > >> > >> > My other concern with 2.6.0 Snapshot is that in Storm UI,
> > the "Storm
> > > > >> > >> > UI" title at the top left hand side of each page is missing.
> > This may
> > > > >> > >> > be the result of one of my own PR, I'll have to check why...
> > > > >> > >> >
> > > > >> > >> > Alex
> > > > >> > >> >
> > > > >> > >> > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla <
> > rzo1@apache.org> a écrit :
> > > > >> > >> > >
> > > > >> > >> > > Hi,
> > > > >> > >> > >
> > > > >> > >> > > thanks for the feedback.
> > > > >> > >> > >
> > > > >> > >> > > The main difference to 2.5.0 is, that we switched the JSON
> > > > >> > >> > > implementation in Utils, which relied on a
> > > > >> > >> > > super old JSON library. What is happening here is, that
> > the new JSON
> > > > >> > >> > > library (net.minidev) behaves differently.
> > > > >> > >> > >
> > > > >> > >> > > We can see it by writing some simple code without Storm
> > (The related
> > > > >> > >> > > code line with the parsing is taken from Storm Utils)
> > > > >> > >> > >
> > > > >> > >> > >
> > > > >> > >> > > Map<String, Long> conf = new HashMap<>();
> > > > >> > >> > > conf.put("test", 100000000L);
> > > > >> > >> > >
> > > > >> > >> > > Map<String, Object> deserTopoConf = (Map<String,
> > Object>)
> > > > >> > >> > > JSONValue.parseWithException(JSONValue.toJSONString(conf));
> > > > >> > >> > >
> > > > >> > >> > > for(Map.Entry<String, Object> e :
> > deserTopoConf.entrySet()) {
> > > > >> > >> > >
> > System.out.println(e.getValue().getClass().getSimpleName());
> > > > >> > >> > > }
> > > > >> > >> > >
> > > > >> > >> > > This will print "Integer". Using "10000000000000000L" as a
> > value will
> > > > >> > >> > > result in "Long".
> > > > >> > >> > >
> > > > >> > >> > > The main issue derives from the fact, that we serialize
> > the config map
> > > > >> > >> > > into a JSON String and parse the result.
> > > > >> > >> > > Based on the String representation, "100000000L" will
> > still fit into an
> > > > >> > >> > > Integer, so the parser will not use Long ;-)
> > > > >> > >> > >
> > > > >> > >> > > If the number is big enough, the parser will switch to a
> > Long.
> > > > >> > >> > > The old library (> 10 years old now):
> > > > >> > >> > >
> > > > >> > >> > > deserTopoConf = (Map<String, Object>)
> > > > >> > >> > >
> > org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
> > > > >> > >> > > toJSONString(conf));
> > > > >> > >> > >
> > > > >> > >> > > for(Map.Entry<String, Object> e :
> > deserTopoConf.entrySet()) {
> > > > >> > >> > >
> > System.out.println(e.getValue().getClass().getSimpleName());
> > > > >> > >> > > }
> > > > >> > >> > >
> > > > >> > >> > > This will print "Long".
> > > > >> > >> > >
> > > > >> > >> > > The difference between these libraries is, that the super
> > old
> > > > >> > >> > > org.json.simple just always returns Long as the type.
> > > > >> > >> > >
> > > > >> > >> > > If can eventually tune the new JSON parser to something
> > like:
> > > > >> > >> > >
> > > > >> > >> > > deserTopoConf = (Map<String, Object>) new
> > > > >> > >> > >
> > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
> > > > >> > >> > > ing(conf), JSONValue.defaultReader.DEFAULT);
> > > > >> > >> > >
> > > > >> > >> > > This will result in "Long" but might have other
> > side-effects.
> > > > >> > >> > >
> > > > >> > >> > > From my point of view, the newer JSON library works as
> > expected
> > > > >> > >> > > (although it might break backward compatibility).
> > > > >> > >> > > Unchecked casts are always evil, imho. In your specific
> > case, it might
> > > > >> > >> > > be more save to cast to "Number" and than invoke
> > "longValue()" instead.
> > > > >> > >> > >
> > > > >> > >> > > Nevertheless, we might being able to restore the old
> > behaviour by
> > > > >> > >> > > tuning the JSONParser configuration.
> > > > >> > >> > >
> > > > >> > >> > > Thoughts?
> > > > >> > >> > >
> > > > >> > >> > > Gruß
> > > > >> > >> > > Richard
> > > > >> > >> > >
> > > > >> > >> > >
> > > > >> > >> > >
> > > > >> > >> > >
> > > > >> > >> > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
> > > > >> > >> > > Vermeerbergen:
> > > > >> > >> > > > On a side note, the Long value that "MyKOBolt" is set by
> > a
> > > > >> > >> > > > DefaultTopologyConfig to a value with is a
> > java.lang.Long instance...
> > > > >> > >> > > > hence it's surprising to see it internally converted
> > into a
> > > > >> > >> > > > java.lang.Integer in the Map passed to prepare()
> > method...
> > > > >> > >> > > >
> > > > >> > >> > > >
> > > > >> > >> > > > Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
> > > > >> > >> > > > <av...@gmail.com> a écrit :
> > > > >> > >> > > > >
> > > > >> > >> > > > > Hello,
> > > > >> > >> > > > >
> > > > >> > >> > > > > I found a breaking change using Apache Storm 2.6.0
> > snapshot build
> > > > >> > >> > > > > dated 20231103 and without changing my topologies'
> > code (keeping
> > > > >> > >> > > > > the
> > > > >> > >> > > > > same big jars).
> > > > >> > >> > > > >
> > > > >> > >> > > > > This seems to be tied to a datatype change in Map
> > argument of
> > > > >> > >> > > > > prepare() method of Bolt.
> > > > >> > >> > > > >
> > > > >> > >> > > > > For example, this Bolt is KO at runtime:
> > > > >> > >> > > > >
> > > > >> > >> > > > > public class MyKOBolt extends BaseRichBolt {
> > > > >> > >> > > > >
> > > > >> > >> > > > > @Override
> > > > >> > >> > > > > public void prepare(Map stormConf, TopologyContext
> > context,
> > > > >> > >> > > > > OutputCollector collector) {
> > > > >> > >> > > > > cacheMaxSize = (Long)
> > > > >> > >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > >> > >> > > > > cacheTtlMin = (Long)
> > > > >> > >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > > > >> > >> > > > >
> > > > >> > >> > > > > as it throws this error:
> > > > >> > >> > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > > > >> > >> > > > > Thread-14-__system-executor[-1, -1] [INFO] Preparing
> > bolt
> > > > >> > >> > > > > __system:[-1]
> > > > >> > >> > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > > >> > >> > > > > Thread-15-evaluateTriggers-executor[11, 11] [ERROR]
> > Async loop
> > > > >> > >> > > > > died!
> > > > >> > >> > > > > java.lang.ClassCastException: java.lang.Integer
> > incompatible with
> > > > >> > >> > > > > java.lang.Long
> > > > >> > >> > > > > at com.acme.storm.alerting.MyKOBolt
> > > > >> > >> > > > > .prepare(EvaluationBolt.java:71)
> > > > >> > >> > > > > ~[stormjar.jar:?]
> > > > >> > >> > > > > at
> > > > >> > >> > > > >
> > org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:
> > > > >> > >> > > > > 128)
> > > > >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > >> > >> > > > > at
> > > > >> > >> > > > >
> > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > > > >> > >> > > > > 138)
> > > > >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > >> > >> > > > > at
> > > > >> > >> > > > >
> > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > > > >> > >> > > > > 54)
> > > > >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > >> > >> > > > > at org.apache.storm.utils.Utils$1.run(Utils.java:393)
> > > > >> > >> > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > >> > >> > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> > > > >> > >> > > > >
> > > > >> > >> > > > > whereas this bolt is fine with Storm 2.6.0:
> > > > >> > >> > > > >
> > > > >> > >> > > > > public class MyOKBolt extends BaseBasicBolt {
> > > > >> > >> > > > >
> > > > >> > >> > > > > @Override
> > > > >> > >> > > > > public void prepare(final Map stormConf, final
> > TopologyContext
> > > > >> > >> > > > > context) {
> > > > >> > >> > > > > super.prepare(stormConf, context);
> > > > >> > >> > > > >
> > > > >> > >> > > > > this.redisPort =
> > > > >> > >> > > > > Integer.parseInt((String)
> > > > >> > >> > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > > > >> > >> > > > >
> > > > >> > >> > > > > That said, I can modify the code of "MyKOBolt" to use
> > same
> > > > >> > >> > > > > Integer.parseInt or Integer.parseLong trick, but it's
> > the first
> > > > >> > >> > > > > time
> > > > >> > >> > > > > in my long history of upgrades that I have seen such
> > runtime
> > > > >> > >> > > > > incompatibility.
> > > > >> > >> > > > >
> > > > >> > >> > > > > Maybe "MyKOBolt" was badly written since the beginning
> > and I have
> > > > >> > >> > > > > just
> > > > >> > >> > > > > hit the punishment for it: is there a documentation
> > which clarifies
> > > > >> > >> > > > > datatypes of the Map argument of prepare() method?
> > > > >> > >> > > > >
> > > > >> > >> > > > > Also worth noting: "MyKOBolt" derives from
> > BaseRichBolt , while
> > > > >> > >> > > > > "MyOKBolt" derives from BaseBasicBolt => could this
> > have any impact
> > > > >> > >> > > > > on
> > > > >> > >> > > > > this finding ?
> > > > >> > >> > > > >
> > > > >> > >> > > > > Last, I'm running this on Redhat Linux 8 and IBM
> > Semeru JDK
> > > > >> > >> > > > > 17.0.8.1.
> > > > >> > >> > > > >
> > > > >> > >> > > > > Kind regards,
> > > > >> > >> > > > > Alexandre
> > > > >> > >> > > > >
> > > > >> > >> > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > > > >> > >> > > > > <li...@gmail.com> a écrit :
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > Thanks Richard.
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > Tried the latest snapshot with StormCrawler both in
> > local and
> > > > >> > >> > > > > > deployed mode
> > > > >> > >> > > > > > and did not find any issues.
> > > > >> > >> > > > > > Will try it on a topology generating WARC files next
> > week to
> > > > >> > >> > > > > > check that the
> > > > >> > >> > > > > > dependency changes on Hadoop have not broken
> > anything.
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > Have a good week end
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > Julien
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <
> > rzo1@apache.org>
> > > > >> > >> > > > > > wrote:
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > > Short update. Sorted out an issue with the
> > tar.gz/zip with
> > > > >> > >> > > > > > > Julien today
> > > > >> > >> > > > > > > and re-uploaded them to the nightlies area.
> > > > >> > >> > > > > > >
> > > > >> > >> > > > > > > This new bundle works as expected in my deployment
> > but happy to
> > > > >> > >> > > > > > > receive
> > > > >> > >> > > > > > > additional feedback before getting up a first
> > release candidate
> > > > >> > >> > > > > > > ;-)
> > > > >> > >> > > > > > >
> > > > >> > >> > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb
> > Richard
> > > > >> > >> > > > > > > Zowalla:
> > > > >> > >> > > > > > > > Hi Alexandre,
> > > > >> > >> > > > > > > >
> > > > >> > >> > > > > > > > we are not in a hurry here :) - take as much as
> > time you
> > > > >> > >> > > > > > > > need.
> > > > >> > >> > > > > > > >
> > > > >> > >> > > > > > > > Gruß
> > > > >> > >> > > > > > > > Richard
> > > > >> > >> > > > > > > >
> > > > >> > >> > > > > > > >
> > > > >> > >> > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb
> > Alexandre
> > > > >> > >> > > > > > > > Vermeerbergen:
> > > > >> > >> > > > > > > > > Hello Richard,
> > > > >> > >> > > > > > > > >
> > > > >> > >> > > > > > > > > Okay, I'm more than happy to do that with my
> > pre-production
> > > > >> > >> > > > > > > > > cluster
> > > > >> > >> > > > > > > > > (~10 topologies) using the binary artifacts.
> > > > >> > >> > > > > > > > > Would it be OK if I can use up to end of this
> > week so that
> > > > >> > >> > > > > > > > > I'll be
> > > > >> > >> > > > > > > > > able to have enough time to check all
> > potential issues that
> > > > >> > >> > > > > > > > > this
> > > > >> > >> > > > > > > > > upgrade bring?
> > > > >> > >> > > > > > > > >
> > > > >> > >> > > > > > > > > Thanks,
> > > > >> > >> > > > > > > > > Alexandre
> > > > >> > >> > > > > > > > >
> > > > >> > >> > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla
> > > > >> > >> > > > > > > > > <rz...@apache.org> a
> > > > >> > >> > > > > > > > > écrit :
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > > Hi all,
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > > before starting a release of 2.6.0-SNAPSHOT,
> > I would like
> > > > >> > >> > > > > > > > > > to
> > > > >> > >> > > > > > > > > > receive
> > > > >> > >> > > > > > > > > > some feedback on the current SNAPSHOT build.
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm
> > build from
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > >
> > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > > It is available via the ASF Snapshot
> > repository:
> > > > >> > >> > > > > > > > > >
> > https://repository.apache.org/content/repositories/snapshots/
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > > You can consume by adding
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > > <repository>
> > > > >> > >> > > > > > > > > > <id>apache.snapshots</id>
> > > > >> > >> > > > > > > > > > <name>Apache Snapshot Repository</name>
> > > > >> > >> > > > > > > > > > <url>
> > https://repository.apache.org/snapshots</url>
> > > > >> > >> > > > > > > > > > <releases>
> > > > >> > >> > > > > > > > > > <enabled>false</enabled>
> > > > >> > >> > > > > > > > > > </releases>
> > > > >> > >> > > > > > > > > > </repository>
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > > to your project pom. As we do not deploy
> > SNAPSHOTS
> > > > >> > >> > > > > > > > > > automatically,
> > > > >> > >> > > > > > > > > > it
> > > > >> > >> > > > > > > > > > should be easy to just consume the latest
> > SNAPSHOT.
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > > The packaged binaries are available at
> > > > >> > >> > > > > > > > > > nightlies.apache.org:
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > >
> > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > > If you have some minutes left: Please test
> > and report any
> > > > >> > >> > > > > > > > > > issues
> > > > >> > >> > > > > > > > > > with
> > > > >> > >> > > > > > > > > > this binaries, so we can fix before
> > attempting to
> > > > >> > >> > > > > > > > > > release.
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > > The most significant changes are in the are
> > of
> > > > >> > >> > > > > > > > > > hadoop/hbase/hdfs
> > > > >> > >> > > > > > > > > > as
> > > > >> > >> > > > > > > > > > we
> > > > >> > >> > > > > > > > > > upgraded from 2.x to 3.x - our own test
> > coverage within
> > > > >> > >> > > > > > > > > > the build
> > > > >> > >> > > > > > > > > > looks
> > > > >> > >> > > > > > > > > > good but would be nice to get some real
> > world use-case
> > > > >> > >> > > > > > > > > > feedback
> > > > >> > >> > > > > > > > > > on
> > > > >> > >> > > > > > > > > > this. In addition, we had quite a lof of 3rd
> > party
> > > > >> > >> > > > > > > > > > dependency
> > > > >> > >> > > > > > > > > > upgrades.
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > > In addition, it contains the pruning of
> > external modules
> > > > >> > >> > > > > > > > > > as
> > > > >> > >> > > > > > > > > > listed
> > > > >> > >> > > > > > > > > > in
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > >
> > https://issues.apache.org/jira/browse/STORM-3988
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > > A summaryin Jira is here:
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > > Gruß
> > > > >> > >> > > > > > > > > > Richard
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > > > >
> > > > >> > >> > > > > > > >
> > > > >> > >> > > > > > >
> > > > >> > >> > > > > > >
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > --
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > *Open Source Solutions for Text Engineering*
> > > > >> > >> > > > > >
> > > > >> > >> > > > > > http://www.digitalpebble.com
> > > > >> > >> > > > > > http://digitalpebble.blogspot.com/
> > > > >> > >> > > > > > #digitalpebble <http://twitter.com/digitalpebble>
> > > > >> > >> > >
> >
>
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble <http://twitter.com/digitalpebble>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Julien Nioche <li...@gmail.com>.
Alexandre,
I had a similar issue when trying the initial snapshot made by Richard but
it seems to have disappeared with the new one.
Maybe try building your code with *mvn clean package -U *or delete the
content of* ~/.m2/repository/org/apache/storm/ *beforehand?
Julien
On Sat, 4 Nov 2023 at 19:57, Alexandre Vermeerbergen <
avermeerbergen@gmail.com> wrote:
> Yes another regression I found with my topologies (built with 2.5.0
> dependencies) running to Storm 2.6.0 SNAPSHOT 2023/11/02: I have seen
> several exceptions like this one:
>
> 2023-11-04 18:43:31.696 o.a.s.e.e.ReportError
> Thread-16-alertsHistoryUMToKafka-executor[27, 27] [ERROR] Error
> java.lang.RuntimeException: java.lang.NoClassDefFoundError:
> org.apache.storm.shade.org.json.simple.JSONObject
> at org.apache.storm.utils.Utils$1.run(Utils.java:413)
> ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> at java.lang.Thread.run(Thread.java:857) [?:?]
> Caused by: java.lang.NoClassDefFoundError:
> org.apache.storm.shade.org.json.simple.JSONObject
> at
> com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessageFromTuple(AlertHistoryTupleToKafkaMapper.java:25)
> ~[stormjar.jar:?]
> at
> com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessageFromTuple(AlertHistoryTupleToKafkaMapper.java:11)
> ~[stormjar.jar:?]
> at org.apache.storm.kafka.bolt.KafkaBolt.process(KafkaBolt.java:177)
> ~[stormjar.jar:?]
>
> and here's the related source code:
>
> package com.acme.storm.evaluator.bolt;
>
> import org.apache.storm.kafka.bolt.mapper.TupleToKafkaMapper;
> import org.apache.storm.shade.org.json.simple.JSONObject;
> import org.apache.storm.tuple.Tuple;
> import org.slf4j.Logger;
> import org.slf4j.LoggerFactory;
>
> import com.acme.storm.util.TupleAlertHistoryFields;
>
> public class AlertHistoryTupleToKafkaMapper implements
> TupleToKafkaMapper<String, String> {
> private static final long serialVersionUID = 5668103889461721134L;
>
> public static final Logger logger =
> LoggerFactory.getLogger(AlertHistoryTupleToKafkaMapper.class);
>
> @Override
> public String getKeyFromTuple(Tuple tuple) {
> return
> tuple.getStringByField(TupleAlertHistoryFields.SERVICE_DEFINITION_NAME);
> }
>
> @SuppressWarnings("unchecked")
> @Override
> public String getMessageFromTuple(Tuple tuple)
> {
> JSONObject tupleJson = new JSONObject();
>
> tuple.getFields().forEach(f -> {
> tupleJson.put(f, tuple.getValueByField(f));
> });
>
> String jsonString = tupleJson.toJSONString();
>
> return jsonString;
> }
> }
>
> line #25 is this one:
>
> JSONObject tupleJson = new JSONObject();
>
> so, from what I understand, Storm 2.6.0 no longer exposes some shaded
> classes such as:
>
> org.apache.storm.shade.org.json.simple.JSONObject;
>
> Interestingly, it looks similar to this issue:
>
> https://stackoverflow.com/questions/33377291/java-lang-noclassdeffounderror-org-json-simple-jsonvalue-at-storm-kafka-dynamic
>
> Whatever, I guess I can only blame our own code for using Storm's
> shaded classes, right ?
> Note it's been running as is for long time and not breaking since many
> upgrades, of course it's not an excuse...
>
> I'll fix our code & continue checking if I see other impacts of 2.5.0
> -> 2.6.0 upgrade...
>
> Thanks,
> Alexandre
>
> Le sam. 4 nov. 2023 à 10:36, Alexandre Vermeerbergen
> <av...@gmail.com> a écrit :
> >
> > Just did a 'git push' with that trivial fix... is that enough?
> >
> > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla <ri...@zowalla.com> a
> écrit :
> > >
> > > You can do as you like ;-)
> > >
> > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen <
> avermeerbergen@gmail.com>:
> > > >I mean :
> > > >
> > > >Can I keep using my
> > > >'STORM-3958-Capacity-to-set-St
> > > >orm-UI's-title-in-conf/storm.yaml'
> > > >branch to push this fix, or should I create a new one?
> > > >
> > > >Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > ><av...@gmail.com> a écrit :
> > > >>
> > > >> Can you keep using my
> > > >> 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-conf/storm.yaml'
> > > >> branch to push this fix, or should I create a new one?
> > > >>
> > > >> Le sam. 4 nov. 2023 à 08:14, Richard Zowalla <ri...@zowalla.com>
> a écrit :
> > > >> >
> > > >> > Just push a fix for it (referencing the original issue) ;-) - we
> hadn't a release in between, so not a big deal.
> > > >> >
> > > >> > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre Vermeerbergen <
> avermeerbergen@gmail.com>:
> > > >> > >Temporary workaround for the Storm UI title's regression: run
> this
> > > >> > >command on the title-template.html file (in
> > > >> > >storm-stable/public/templates directory) after having extracted
> the
> > > >> > >binary archive:
> > > >> > >
> > > >> > >sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > > >> > >
> > > >> > >Again sorry for having introduced this regression.
> > > >> > >
> > > >> > >Alexandre
> > > >> > >
> > > >> > >Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > > >> > ><av...@gmail.com> a écrit :
> > > >> > >>
> > > >> > >> Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
> > > >> > >> 1. I have modified my KO bolts using this pattern in their
> prepare() method:
> > > >> > >>
> > > >> > >> long cacheMaxSize =
> > > >> > >>
> Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
> > > >> > >>
> > > >> > >> instead of:
> > > >> > >>
> > > >> > >> long cacheMaxSize = (Long)
> stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > >> > >>
> > > >> > >> 2. I have found the cause of the regression in loss of 'Storm
> UI'
> > > >> > >> title at the top left-hand side of all Storm UI's pages.
> > > >> > >> This is a mistake of mine in this PR:
> > > >> > >>
> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > >> > >> More precisely, at line 19 of this file:
> > > >> > >>
> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > >> > >> we must have:
> > > >> > >> <h1><a href="/">{{title}}</a></h1>
> > > >> > >>
> > > >> > >> instead of:
> > > >> > >>
> > > >> > >> <h1><a href="/">{{ui.title}}</a></h1>
> > > >> > >>
> > > >> > >> I think I missed this during my tests because of browser's
> cache
> > > >> > >> (bad excuse, I know...): probably I tested by-patching my
> deployed
> > > >> > >> Storm cluster, and I finally forgot to checkout this latest
> change of
> > > >> > >> my delivery.
> > > >> > >>
> > > >> > >> I had to use "private navigation" to make sure this time it
> fixes this issue
> > > >> > >>
> > > >> > >> What would be the quickest way for this very simple fix to be
> delivered ?
> > > >> > >>
> > > >> > >> Thanks,
> > > >> > >> Alexandre
> > > >> > >>
> > > >> > >> Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> > > >> > >> <av...@gmail.com> a écrit :
> > > >> > >> >
> > > >> > >> > Thanks Richard for the analysis.
> > > >> > >> >
> > > >> > >> > My thought on this are very simple: let me fix my own old
> code, after
> > > >> > >> > all this was impacting only one of my topologies, the other
> one were
> > > >> > >> > using more careful Integer.parseInt or Integer.parseLong to
> avoid
> > > >> > >> > making assumption on the way the values in the config Map
> were
> > > >> > >> > serialized.
> > > >> > >> >
> > > >> > >> > My other concern with 2.6.0 Snapshot is that in Storm UI,
> the "Storm
> > > >> > >> > UI" title at the top left hand side of each page is missing.
> This may
> > > >> > >> > be the result of one of my own PR, I'll have to check why...
> > > >> > >> >
> > > >> > >> > Alex
> > > >> > >> >
> > > >> > >> > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla <
> rzo1@apache.org> a écrit :
> > > >> > >> > >
> > > >> > >> > > Hi,
> > > >> > >> > >
> > > >> > >> > > thanks for the feedback.
> > > >> > >> > >
> > > >> > >> > > The main difference to 2.5.0 is, that we switched the JSON
> > > >> > >> > > implementation in Utils, which relied on a
> > > >> > >> > > super old JSON library. What is happening here is, that
> the new JSON
> > > >> > >> > > library (net.minidev) behaves differently.
> > > >> > >> > >
> > > >> > >> > > We can see it by writing some simple code without Storm
> (The related
> > > >> > >> > > code line with the parsing is taken from Storm Utils)
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> > > Map<String, Long> conf = new HashMap<>();
> > > >> > >> > > conf.put("test", 100000000L);
> > > >> > >> > >
> > > >> > >> > > Map<String, Object> deserTopoConf = (Map<String,
> Object>)
> > > >> > >> > > JSONValue.parseWithException(JSONValue.toJSONString(conf));
> > > >> > >> > >
> > > >> > >> > > for(Map.Entry<String, Object> e :
> deserTopoConf.entrySet()) {
> > > >> > >> > >
> System.out.println(e.getValue().getClass().getSimpleName());
> > > >> > >> > > }
> > > >> > >> > >
> > > >> > >> > > This will print "Integer". Using "10000000000000000L" as a
> value will
> > > >> > >> > > result in "Long".
> > > >> > >> > >
> > > >> > >> > > The main issue derives from the fact, that we serialize
> the config map
> > > >> > >> > > into a JSON String and parse the result.
> > > >> > >> > > Based on the String representation, "100000000L" will
> still fit into an
> > > >> > >> > > Integer, so the parser will not use Long ;-)
> > > >> > >> > >
> > > >> > >> > > If the number is big enough, the parser will switch to a
> Long.
> > > >> > >> > > The old library (> 10 years old now):
> > > >> > >> > >
> > > >> > >> > > deserTopoConf = (Map<String, Object>)
> > > >> > >> > >
> org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
> > > >> > >> > > toJSONString(conf));
> > > >> > >> > >
> > > >> > >> > > for(Map.Entry<String, Object> e :
> deserTopoConf.entrySet()) {
> > > >> > >> > >
> System.out.println(e.getValue().getClass().getSimpleName());
> > > >> > >> > > }
> > > >> > >> > >
> > > >> > >> > > This will print "Long".
> > > >> > >> > >
> > > >> > >> > > The difference between these libraries is, that the super
> old
> > > >> > >> > > org.json.simple just always returns Long as the type.
> > > >> > >> > >
> > > >> > >> > > If can eventually tune the new JSON parser to something
> like:
> > > >> > >> > >
> > > >> > >> > > deserTopoConf = (Map<String, Object>) new
> > > >> > >> > >
> JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
> > > >> > >> > > ing(conf), JSONValue.defaultReader.DEFAULT);
> > > >> > >> > >
> > > >> > >> > > This will result in "Long" but might have other
> side-effects.
> > > >> > >> > >
> > > >> > >> > > From my point of view, the newer JSON library works as
> expected
> > > >> > >> > > (although it might break backward compatibility).
> > > >> > >> > > Unchecked casts are always evil, imho. In your specific
> case, it might
> > > >> > >> > > be more save to cast to "Number" and than invoke
> "longValue()" instead.
> > > >> > >> > >
> > > >> > >> > > Nevertheless, we might being able to restore the old
> behaviour by
> > > >> > >> > > tuning the JSONParser configuration.
> > > >> > >> > >
> > > >> > >> > > Thoughts?
> > > >> > >> > >
> > > >> > >> > > Gruß
> > > >> > >> > > Richard
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
> > > >> > >> > > Vermeerbergen:
> > > >> > >> > > > On a side note, the Long value that "MyKOBolt" is set by
> a
> > > >> > >> > > > DefaultTopologyConfig to a value with is a
> java.lang.Long instance...
> > > >> > >> > > > hence it's surprising to see it internally converted
> into a
> > > >> > >> > > > java.lang.Integer in the Map passed to prepare()
> method...
> > > >> > >> > > >
> > > >> > >> > > >
> > > >> > >> > > > Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
> > > >> > >> > > > <av...@gmail.com> a écrit :
> > > >> > >> > > > >
> > > >> > >> > > > > Hello,
> > > >> > >> > > > >
> > > >> > >> > > > > I found a breaking change using Apache Storm 2.6.0
> snapshot build
> > > >> > >> > > > > dated 20231103 and without changing my topologies'
> code (keeping
> > > >> > >> > > > > the
> > > >> > >> > > > > same big jars).
> > > >> > >> > > > >
> > > >> > >> > > > > This seems to be tied to a datatype change in Map
> argument of
> > > >> > >> > > > > prepare() method of Bolt.
> > > >> > >> > > > >
> > > >> > >> > > > > For example, this Bolt is KO at runtime:
> > > >> > >> > > > >
> > > >> > >> > > > > public class MyKOBolt extends BaseRichBolt {
> > > >> > >> > > > >
> > > >> > >> > > > > @Override
> > > >> > >> > > > > public void prepare(Map stormConf, TopologyContext
> context,
> > > >> > >> > > > > OutputCollector collector) {
> > > >> > >> > > > > cacheMaxSize = (Long)
> > > >> > >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > >> > >> > > > > cacheTtlMin = (Long)
> > > >> > >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > > >> > >> > > > >
> > > >> > >> > > > > as it throws this error:
> > > >> > >> > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > > >> > >> > > > > Thread-14-__system-executor[-1, -1] [INFO] Preparing
> bolt
> > > >> > >> > > > > __system:[-1]
> > > >> > >> > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > >> > >> > > > > Thread-15-evaluateTriggers-executor[11, 11] [ERROR]
> Async loop
> > > >> > >> > > > > died!
> > > >> > >> > > > > java.lang.ClassCastException: java.lang.Integer
> incompatible with
> > > >> > >> > > > > java.lang.Long
> > > >> > >> > > > > at com.acme.storm.alerting.MyKOBolt
> > > >> > >> > > > > .prepare(EvaluationBolt.java:71)
> > > >> > >> > > > > ~[stormjar.jar:?]
> > > >> > >> > > > > at
> > > >> > >> > > > >
> org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:
> > > >> > >> > > > > 128)
> > > >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > >> > >> > > > > at
> > > >> > >> > > > >
> org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > > >> > >> > > > > 138)
> > > >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > >> > >> > > > > at
> > > >> > >> > > > >
> org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > > >> > >> > > > > 54)
> > > >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > >> > >> > > > > at org.apache.storm.utils.Utils$1.run(Utils.java:393)
> > > >> > >> > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > >> > >> > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> > > >> > >> > > > >
> > > >> > >> > > > > whereas this bolt is fine with Storm 2.6.0:
> > > >> > >> > > > >
> > > >> > >> > > > > public class MyOKBolt extends BaseBasicBolt {
> > > >> > >> > > > >
> > > >> > >> > > > > @Override
> > > >> > >> > > > > public void prepare(final Map stormConf, final
> TopologyContext
> > > >> > >> > > > > context) {
> > > >> > >> > > > > super.prepare(stormConf, context);
> > > >> > >> > > > >
> > > >> > >> > > > > this.redisPort =
> > > >> > >> > > > > Integer.parseInt((String)
> > > >> > >> > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > > >> > >> > > > >
> > > >> > >> > > > > That said, I can modify the code of "MyKOBolt" to use
> same
> > > >> > >> > > > > Integer.parseInt or Integer.parseLong trick, but it's
> the first
> > > >> > >> > > > > time
> > > >> > >> > > > > in my long history of upgrades that I have seen such
> runtime
> > > >> > >> > > > > incompatibility.
> > > >> > >> > > > >
> > > >> > >> > > > > Maybe "MyKOBolt" was badly written since the beginning
> and I have
> > > >> > >> > > > > just
> > > >> > >> > > > > hit the punishment for it: is there a documentation
> which clarifies
> > > >> > >> > > > > datatypes of the Map argument of prepare() method?
> > > >> > >> > > > >
> > > >> > >> > > > > Also worth noting: "MyKOBolt" derives from
> BaseRichBolt , while
> > > >> > >> > > > > "MyOKBolt" derives from BaseBasicBolt => could this
> have any impact
> > > >> > >> > > > > on
> > > >> > >> > > > > this finding ?
> > > >> > >> > > > >
> > > >> > >> > > > > Last, I'm running this on Redhat Linux 8 and IBM
> Semeru JDK
> > > >> > >> > > > > 17.0.8.1.
> > > >> > >> > > > >
> > > >> > >> > > > > Kind regards,
> > > >> > >> > > > > Alexandre
> > > >> > >> > > > >
> > > >> > >> > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > > >> > >> > > > > <li...@gmail.com> a écrit :
> > > >> > >> > > > > >
> > > >> > >> > > > > > Thanks Richard.
> > > >> > >> > > > > >
> > > >> > >> > > > > > Tried the latest snapshot with StormCrawler both in
> local and
> > > >> > >> > > > > > deployed mode
> > > >> > >> > > > > > and did not find any issues.
> > > >> > >> > > > > > Will try it on a topology generating WARC files next
> week to
> > > >> > >> > > > > > check that the
> > > >> > >> > > > > > dependency changes on Hadoop have not broken
> anything.
> > > >> > >> > > > > >
> > > >> > >> > > > > > Have a good week end
> > > >> > >> > > > > >
> > > >> > >> > > > > > Julien
> > > >> > >> > > > > >
> > > >> > >> > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <
> rzo1@apache.org>
> > > >> > >> > > > > > wrote:
> > > >> > >> > > > > >
> > > >> > >> > > > > > > Short update. Sorted out an issue with the
> tar.gz/zip with
> > > >> > >> > > > > > > Julien today
> > > >> > >> > > > > > > and re-uploaded them to the nightlies area.
> > > >> > >> > > > > > >
> > > >> > >> > > > > > > This new bundle works as expected in my deployment
> but happy to
> > > >> > >> > > > > > > receive
> > > >> > >> > > > > > > additional feedback before getting up a first
> release candidate
> > > >> > >> > > > > > > ;-)
> > > >> > >> > > > > > >
> > > >> > >> > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb
> Richard
> > > >> > >> > > > > > > Zowalla:
> > > >> > >> > > > > > > > Hi Alexandre,
> > > >> > >> > > > > > > >
> > > >> > >> > > > > > > > we are not in a hurry here :) - take as much as
> time you
> > > >> > >> > > > > > > > need.
> > > >> > >> > > > > > > >
> > > >> > >> > > > > > > > Gruß
> > > >> > >> > > > > > > > Richard
> > > >> > >> > > > > > > >
> > > >> > >> > > > > > > >
> > > >> > >> > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb
> Alexandre
> > > >> > >> > > > > > > > Vermeerbergen:
> > > >> > >> > > > > > > > > Hello Richard,
> > > >> > >> > > > > > > > >
> > > >> > >> > > > > > > > > Okay, I'm more than happy to do that with my
> pre-production
> > > >> > >> > > > > > > > > cluster
> > > >> > >> > > > > > > > > (~10 topologies) using the binary artifacts.
> > > >> > >> > > > > > > > > Would it be OK if I can use up to end of this
> week so that
> > > >> > >> > > > > > > > > I'll be
> > > >> > >> > > > > > > > > able to have enough time to check all
> potential issues that
> > > >> > >> > > > > > > > > this
> > > >> > >> > > > > > > > > upgrade bring?
> > > >> > >> > > > > > > > >
> > > >> > >> > > > > > > > > Thanks,
> > > >> > >> > > > > > > > > Alexandre
> > > >> > >> > > > > > > > >
> > > >> > >> > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla
> > > >> > >> > > > > > > > > <rz...@apache.org> a
> > > >> > >> > > > > > > > > écrit :
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > Hi all,
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > before starting a release of 2.6.0-SNAPSHOT,
> I would like
> > > >> > >> > > > > > > > > > to
> > > >> > >> > > > > > > > > > receive
> > > >> > >> > > > > > > > > > some feedback on the current SNAPSHOT build.
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm
> build from
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > >
> https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > It is available via the ASF Snapshot
> repository:
> > > >> > >> > > > > > > > > >
> https://repository.apache.org/content/repositories/snapshots/
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > You can consume by adding
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > <repository>
> > > >> > >> > > > > > > > > > <id>apache.snapshots</id>
> > > >> > >> > > > > > > > > > <name>Apache Snapshot Repository</name>
> > > >> > >> > > > > > > > > > <url>
> https://repository.apache.org/snapshots</url>
> > > >> > >> > > > > > > > > > <releases>
> > > >> > >> > > > > > > > > > <enabled>false</enabled>
> > > >> > >> > > > > > > > > > </releases>
> > > >> > >> > > > > > > > > > </repository>
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > to your project pom. As we do not deploy
> SNAPSHOTS
> > > >> > >> > > > > > > > > > automatically,
> > > >> > >> > > > > > > > > > it
> > > >> > >> > > > > > > > > > should be easy to just consume the latest
> SNAPSHOT.
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > The packaged binaries are available at
> > > >> > >> > > > > > > > > > nightlies.apache.org:
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > >
> https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > If you have some minutes left: Please test
> and report any
> > > >> > >> > > > > > > > > > issues
> > > >> > >> > > > > > > > > > with
> > > >> > >> > > > > > > > > > this binaries, so we can fix before
> attempting to
> > > >> > >> > > > > > > > > > release.
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > The most significant changes are in the are
> of
> > > >> > >> > > > > > > > > > hadoop/hbase/hdfs
> > > >> > >> > > > > > > > > > as
> > > >> > >> > > > > > > > > > we
> > > >> > >> > > > > > > > > > upgraded from 2.x to 3.x - our own test
> coverage within
> > > >> > >> > > > > > > > > > the build
> > > >> > >> > > > > > > > > > looks
> > > >> > >> > > > > > > > > > good but would be nice to get some real
> world use-case
> > > >> > >> > > > > > > > > > feedback
> > > >> > >> > > > > > > > > > on
> > > >> > >> > > > > > > > > > this. In addition, we had quite a lof of 3rd
> party
> > > >> > >> > > > > > > > > > dependency
> > > >> > >> > > > > > > > > > upgrades.
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > In addition, it contains the pruning of
> external modules
> > > >> > >> > > > > > > > > > as
> > > >> > >> > > > > > > > > > listed
> > > >> > >> > > > > > > > > > in
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > >
> https://issues.apache.org/jira/browse/STORM-3988
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > A summaryin Jira is here:
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > > Gruß
> > > >> > >> > > > > > > > > > Richard
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > > > >
> > > >> > >> > > > > > > >
> > > >> > >> > > > > > >
> > > >> > >> > > > > > >
> > > >> > >> > > > > >
> > > >> > >> > > > > > --
> > > >> > >> > > > > >
> > > >> > >> > > > > > *Open Source Solutions for Text Engineering*
> > > >> > >> > > > > >
> > > >> > >> > > > > > http://www.digitalpebble.com
> > > >> > >> > > > > > http://digitalpebble.blogspot.com/
> > > >> > >> > > > > > #digitalpebble <http://twitter.com/digitalpebble>
> > > >> > >> > >
>
--
*Open Source Solutions for Text Engineering*
http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble <http://twitter.com/digitalpebble>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Richard Zowalla <rz...@apache.org>.
We dropped "org.json.simple.*" and replaced it with JSON from
"net.minidev".
So it would be best to replace that dependency (should be a namespace
shift only).
Am Samstag, dem 04.11.2023 um 20:57 +0100 schrieb Alexandre
Vermeerbergen:
> Yes another regression I found with my topologies (built with 2.5.0
> dependencies) running to Storm 2.6.0 SNAPSHOT 2023/11/02: I have seen
> several exceptions like this one:
>
> 2023-11-04 18:43:31.696 o.a.s.e.e.ReportError
> Thread-16-alertsHistoryUMToKafka-executor[27, 27] [ERROR] Error
> java.lang.RuntimeException: java.lang.NoClassDefFoundError:
> org.apache.storm.shade.org.json.simple.JSONObject
> at org.apache.storm.utils.Utils$1.run(Utils.java:413)
> ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> at java.lang.Thread.run(Thread.java:857) [?:?]
> Caused by: java.lang.NoClassDefFoundError:
> org.apache.storm.shade.org.json.simple.JSONObject
> at
> com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessa
> geFromTuple(AlertHistoryTupleToKafkaMapper.java:25)
> ~[stormjar.jar:?]
> at
> com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessa
> geFromTuple(AlertHistoryTupleToKafkaMapper.java:11)
> ~[stormjar.jar:?]
> at
> org.apache.storm.kafka.bolt.KafkaBolt.process(KafkaBolt.java:177)
> ~[stormjar.jar:?]
>
> and here's the related source code:
>
> package com.acme.storm.evaluator.bolt;
>
> import org.apache.storm.kafka.bolt.mapper.TupleToKafkaMapper;
> import org.apache.storm.shade.org.json.simple.JSONObject;
> import org.apache.storm.tuple.Tuple;
> import org.slf4j.Logger;
> import org.slf4j.LoggerFactory;
>
> import com.acme.storm.util.TupleAlertHistoryFields;
>
> public class AlertHistoryTupleToKafkaMapper implements
> TupleToKafkaMapper<String, String> {
> private static final long serialVersionUID =
> 5668103889461721134L;
>
> public static final Logger logger =
> LoggerFactory.getLogger(AlertHistoryTupleToKafkaMapper.class);
>
> @Override
> public String getKeyFromTuple(Tuple tuple) {
> return
> tuple.getStringByField(TupleAlertHistoryFields.SERVICE_DEFINITION_NAM
> E);
> }
>
> @SuppressWarnings("unchecked")
> @Override
> public String getMessageFromTuple(Tuple tuple)
> {
> JSONObject tupleJson = new JSONObject();
>
> tuple.getFields().forEach(f -> {
> tupleJson.put(f, tuple.getValueByField(f));
> });
>
> String jsonString = tupleJson.toJSONString();
>
> return jsonString;
> }
> }
>
> line #25 is this one:
>
> JSONObject tupleJson = new JSONObject();
>
> so, from what I understand, Storm 2.6.0 no longer exposes some shaded
> classes such as:
>
> org.apache.storm.shade.org.json.simple.JSONObject;
>
> Interestingly, it looks similar to this issue:
> https://stackoverflow.com/questions/33377291/java-lang-noclassdeffounderror-org-json-simple-jsonvalue-at-storm-kafka-dynamic
>
> Whatever, I guess I can only blame our own code for using Storm's
> shaded classes, right ?
> Note it's been running as is for long time and not breaking since
> many
> upgrades, of course it's not an excuse...
>
> I'll fix our code & continue checking if I see other impacts of 2.5.0
> -> 2.6.0 upgrade...
>
> Thanks,
> Alexandre
>
> Le sam. 4 nov. 2023 à 10:36, Alexandre Vermeerbergen
> <av...@gmail.com> a écrit :
> >
> > Just did a 'git push' with that trivial fix... is that enough?
> >
> > Le sam. 4 nov. 2023 à 09:34, Richard Zowalla <ri...@zowalla.com>
> > a écrit :
> > >
> > > You can do as you like ;-)
> > >
> > > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen
> > > <av...@gmail.com>:
> > > > I mean :
> > > >
> > > > Can I keep using my
> > > > 'STORM-3958-Capacity-to-set-St
> > > > orm-UI's-title-in-conf/storm.yaml'
> > > > branch to push this fix, or should I create a new one?
> > > >
> > > > Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > > > <av...@gmail.com> a écrit :
> > > > >
> > > > > Can you keep using my
> > > > > 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-
> > > > > conf/storm.yaml'
> > > > > branch to push this fix, or should I create a new one?
> > > > >
> > > > > Le sam. 4 nov. 2023 à 08:14, Richard Zowalla
> > > > > <ri...@zowalla.com> a écrit :
> > > > > >
> > > > > > Just push a fix for it (referencing the original issue) ;-)
> > > > > > - we hadn't a release in between, so not a big deal.
> > > > > >
> > > > > > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre
> > > > > > Vermeerbergen <av...@gmail.com>:
> > > > > > > Temporary workaround for the Storm UI title's regression:
> > > > > > > run this
> > > > > > > command on the title-template.html file (in
> > > > > > > storm-stable/public/templates directory) after having
> > > > > > > extracted the
> > > > > > > binary archive:
> > > > > > >
> > > > > > > sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > > > > > >
> > > > > > > Again sorry for having introduced this regression.
> > > > > > >
> > > > > > > Alexandre
> > > > > > >
> > > > > > > Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > > > > > > <av...@gmail.com> a écrit :
> > > > > > > >
> > > > > > > > Updates regarding my test with Storm 2.6.0 Snapshot
> > > > > > > > 2023/11/02:
> > > > > > > > 1. I have modified my KO bolts using this pattern in
> > > > > > > > their prepare() method:
> > > > > > > >
> > > > > > > > long cacheMaxSize =
> > > > > > > > Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZ
> > > > > > > > E).toString());
> > > > > > > >
> > > > > > > > instead of:
> > > > > > > >
> > > > > > > > long cacheMaxSize = (Long)
> > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > >
> > > > > > > > 2. I have found the cause of the regression in loss of
> > > > > > > > 'Storm UI'
> > > > > > > > title at the top left-hand side of all Storm UI's
> > > > > > > > pages.
> > > > > > > > This is a mistake of mine in this PR:
> > > > > > > >
> > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > > > > > > > More precisely, at line 19 of this file:
> > > > > > > > https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > > > > > > > we must have:
> > > > > > > > <h1><a href="/">{{title}}</a></h1>
> > > > > > > >
> > > > > > > > instead of:
> > > > > > > >
> > > > > > > > <h1><a href="/">{{ui.title}}</a></h1>
> > > > > > > >
> > > > > > > > I think I missed this during my tests because of
> > > > > > > > browser's cache
> > > > > > > > (bad excuse, I know...): probably I tested by-patching
> > > > > > > > my deployed
> > > > > > > > Storm cluster, and I finally forgot to checkout this
> > > > > > > > latest change of
> > > > > > > > my delivery.
> > > > > > > >
> > > > > > > > I had to use "private navigation" to make sure this
> > > > > > > > time it fixes this issue
> > > > > > > >
> > > > > > > > What would be the quickest way for this very simple fix
> > > > > > > > to be delivered ?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Alexandre
> > > > > > > >
> > > > > > > > Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > >
> > > > > > > > > Thanks Richard for the analysis.
> > > > > > > > >
> > > > > > > > > My thought on this are very simple: let me fix my own
> > > > > > > > > old code, after
> > > > > > > > > all this was impacting only one of my topologies, the
> > > > > > > > > other one were
> > > > > > > > > using more careful Integer.parseInt or
> > > > > > > > > Integer.parseLong to avoid
> > > > > > > > > making assumption on the way the values in the config
> > > > > > > > > Map were
> > > > > > > > > serialized.
> > > > > > > > >
> > > > > > > > > My other concern with 2.6.0 Snapshot is that in Storm
> > > > > > > > > UI, the "Storm
> > > > > > > > > UI" title at the top left hand side of each page is
> > > > > > > > > missing. This may
> > > > > > > > > be the result of one of my own PR, I'll have to check
> > > > > > > > > why...
> > > > > > > > >
> > > > > > > > > Alex
> > > > > > > > >
> > > > > > > > > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla
> > > > > > > > > <rz...@apache.org> a écrit :
> > > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > thanks for the feedback.
> > > > > > > > > >
> > > > > > > > > > The main difference to 2.5.0 is, that we switched
> > > > > > > > > > the JSON
> > > > > > > > > > implementation in Utils, which relied on a
> > > > > > > > > > super old JSON library. What is happening here is,
> > > > > > > > > > that the new JSON
> > > > > > > > > > library (net.minidev) behaves differently.
> > > > > > > > > >
> > > > > > > > > > We can see it by writing some simple code without
> > > > > > > > > > Storm (The related
> > > > > > > > > > code line with the parsing is taken from Storm
> > > > > > > > > > Utils)
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Map<String, Long> conf = new HashMap<>();
> > > > > > > > > > conf.put("test", 100000000L);
> > > > > > > > > >
> > > > > > > > > > Map<String, Object> deserTopoConf =
> > > > > > > > > > (Map<String, Object>)
> > > > > > > > > > JSONValue.parseWithException(JSONValue.toJSONString
> > > > > > > > > > (conf));
> > > > > > > > > >
> > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > >
> > > > > > > > > > System.out.println(e.getValue().getClass().getSimpl
> > > > > > > > > > eName());
> > > > > > > > > > }
> > > > > > > > > >
> > > > > > > > > > This will print "Integer". Using
> > > > > > > > > > "10000000000000000L" as a value will
> > > > > > > > > > result in "Long".
> > > > > > > > > >
> > > > > > > > > > The main issue derives from the fact, that we
> > > > > > > > > > serialize the config map
> > > > > > > > > > into a JSON String and parse the result.
> > > > > > > > > > Based on the String representation, "100000000L"
> > > > > > > > > > will still fit into an
> > > > > > > > > > Integer, so the parser will not use Long ;-)
> > > > > > > > > >
> > > > > > > > > > If the number is big enough, the parser will switch
> > > > > > > > > > to a Long.
> > > > > > > > > > The old library (> 10 years old now):
> > > > > > > > > >
> > > > > > > > > > deserTopoConf = (Map<String, Object>)
> > > > > > > > > > org.json.simple.JSONValue.parseWithException(org.js
> > > > > > > > > > on.simple.JSONValue.
> > > > > > > > > > toJSONString(conf));
> > > > > > > > > >
> > > > > > > > > > for(Map.Entry<String, Object> e :
> > > > > > > > > > deserTopoConf.entrySet()) {
> > > > > > > > > >
> > > > > > > > > > System.out.println(e.getValue().getClass().getSimpl
> > > > > > > > > > eName());
> > > > > > > > > > }
> > > > > > > > > >
> > > > > > > > > > This will print "Long".
> > > > > > > > > >
> > > > > > > > > > The difference between these libraries is, that the
> > > > > > > > > > super old
> > > > > > > > > > org.json.simple just always returns Long as the
> > > > > > > > > > type.
> > > > > > > > > >
> > > > > > > > > > If can eventually tune the new JSON parser to
> > > > > > > > > > something like:
> > > > > > > > > >
> > > > > > > > > > deserTopoConf = (Map<String, Object>) new
> > > > > > > > > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse
> > > > > > > > > > (JSONValue.toJSONStr
> > > > > > > > > > ing(conf), JSONValue.defaultReader.DEFAULT);
> > > > > > > > > >
> > > > > > > > > > This will result in "Long" but might have other
> > > > > > > > > > side-effects.
> > > > > > > > > >
> > > > > > > > > > From my point of view, the newer JSON library works
> > > > > > > > > > as expected
> > > > > > > > > > (although it might break backward compatibility).
> > > > > > > > > > Unchecked casts are always evil, imho. In your
> > > > > > > > > > specific case, it might
> > > > > > > > > > be more save to cast to "Number" and than invoke
> > > > > > > > > > "longValue()" instead.
> > > > > > > > > >
> > > > > > > > > > Nevertheless, we might being able to restore the
> > > > > > > > > > old behaviour by
> > > > > > > > > > tuning the JSONParser configuration.
> > > > > > > > > >
> > > > > > > > > > Thoughts?
> > > > > > > > > >
> > > > > > > > > > Gruß
> > > > > > > > > > Richard
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb
> > > > > > > > > > Alexandre
> > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > On a side note, the Long value that "MyKOBolt" is
> > > > > > > > > > > set by a
> > > > > > > > > > > DefaultTopologyConfig to a value with is a
> > > > > > > > > > > java.lang.Long instance...
> > > > > > > > > > > hence it's surprising to see it internally
> > > > > > > > > > > converted into a
> > > > > > > > > > > java.lang.Integer in the Map passed to prepare()
> > > > > > > > > > > method..> > >> > >> > > >
> > > > > > > > > > >
> > > > > > > > > > > Le ven. 3 nov. 2023 à 18:27, Alexandre
> > > > > > > > > > > Vermeerbergen
> > > > > > > > > > > <av...@gmail.com> a écrit :
> > > > > > > > > > > >
> > > > > > > > > > > > Hello,
> > > > > > > > > > > >
> > > > > > > > > > > > I found a breaking change using Apache Storm
> > > > > > > > > > > > 2.6.0 snapshot build
> > > > > > > > > > > > dated 20231103 and without changing my
> > > > > > > > > > > > topologies' code (keeping
> > > > > > > > > > > > the
> > > > > > > > > > > > same big jars).
> > > > > > > > > > > >
> > > > > > > > > > > > This seems to be tied to a datatype change in
> > > > > > > > > > > > Map argument of
> > > > > > > > > > > > prepare() method of Bolt.
> > > > > > > > > > > >
> > > > > > > > > > > > For example, this Bolt is KO at runtime:
> > > > > > > > > > > >
> > > > > > > > > > > > public class MyKOBolt extends BaseRichBolt {
> > > > > > > > > > > >
> > > > > > > > > > > > @Override
> > > > > > > > > > > > public void prepare(Map stormConf,
> > > > > > > > > > > > TopologyContext context,
> > > > > > > > > > > > OutputCollector collector) {
> > > > > > > > > > > > cacheMaxSize = (Long)
> > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > > > > > > > > cacheTtlMin = (Long)
> > > > > > > > > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > > > > > > > > > > >
> > > > > > > > > > > > as it throws this error:
> > > > > > > > > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > > > > > > > > > > > Thread-14-__system-executor[-1, -1] [INFO]
> > > > > > > > > > > > Preparing bolt
> > > > > > > > > > > > __system:[-1]
> > > > > > > > > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > > > > > > > > > > Thread-15-evaluateTriggers-executor[11, 11]
> > > > > > > > > > > > [ERROR] Async loop
> > > > > > > > > > > > died!
> > > > > > > > > > > > java.lang.ClassCastException: java.lang.Integer
> > > > > > > > > > > > incompatible with
> > > > > > > > > > > > java.lang.Long
> > > > > > > > > > > > at com.acme.storm.alerting.MyKOBolt
> > > > > > > > > > > > .prepare(EvaluationBolt.java:71)
> > > > > > > > > > > > ~[stormjar.jar:?]
> > > > > > > > > > > > at
> > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.ini
> > > > > > > > > > > > t(BoltExecutor.java:
> > > > > > > > > > > > 128)
> > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > at
> > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.cal
> > > > > > > > > > > > l(BoltExecutor.java:
> > > > > > > > > > > > 138)
> > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > at
> > > > > > > > > > > > org.apache.storm.executor.bolt.BoltExecutor.cal
> > > > > > > > > > > > l(BoltExecutor.java:
> > > > > > > > > > > > 54)
> > > > > > > > > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > at
> > > > > > > > > > > > org.apache.storm.utils.Utils$1.run(Utils.java:3
> > > > > > > > > > > > 93)
> > > > > > > > > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-
> > > > > > > > > > > > SNAPSHOT]
> > > > > > > > > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> > > > > > > > > > > >
> > > > > > > > > > > > whereas this bolt is fine with Storm 2.6.0:
> > > > > > > > > > > >
> > > > > > > > > > > > public class MyOKBolt extends BaseBasicBolt {
> > > > > > > > > > > >
> > > > > > > > > > > > @Override
> > > > > > > > > > > > public void prepare(final Map stormConf,
> > > > > > > > > > > > final TopologyContext
> > > > > > > > > > > > context) {
> > > > > > > > > > > > super.prepare(stormConf, context);
> > > > > > > > > > > >
> > > > > > > > > > > > this.redisPort =
> > > > > > > > > > > > Integer.parseInt((String)
> > > > > > > > > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > > > > > > > > > > >
> > > > > > > > > > > > That said, I can modify the code of "MyKOBolt"
> > > > > > > > > > > > to use same
> > > > > > > > > > > > Integer.parseInt or Integer.parseLong trick,
> > > > > > > > > > > > but it's the first
> > > > > > > > > > > > time
> > > > > > > > > > > > in my long history of upgrades that I have seen
> > > > > > > > > > > > such runtime
> > > > > > > > > > > > incompatibility.
> > > > > > > > > > > >
> > > > > > > > > > > > Maybe "MyKOBolt" was badly written since the
> > > > > > > > > > > > beginning and I have
> > > > > > > > > > > > just
> > > > > > > > > > > > hit the punishment for it: is there a
> > > > > > > > > > > > documentation which clarifies
> > > > > > > > > > > > datatypes of the Map argument of prepare()
> > > > > > > > > > > > method?
> > > > > > > > > > > >
> > > > > > > > > > > > Also worth noting: "MyKOBolt" derives from
> > > > > > > > > > > > BaseRichBolt , while
> > > > > > > > > > > > "MyOKBolt" derives from BaseBasicBolt => could
> > > > > > > > > > > > this have any impact
> > > > > > > > > > > > on
> > > > > > > > > > > > this finding ?
> > > > > > > > > > > >
> > > > > > > > > > > > Last, I'm running this on Redhat Linux 8 and
> > > > > > > > > > > > IBM Semeru JDK
> > > > > > > > > > > > 17.0.8.1.
> > > > > > > > > > > >
> > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > Alexandre
> > > > > > > > > > > >
> > > > > > > > > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > > > > > > > > > > > <li...@gmail.com> a écrit :
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks Richard.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Tried the latest snapshot with StormCrawler
> > > > > > > > > > > > > both in local and
> > > > > > > > > > > > > deployed mode
> > > > > > > > > > > > > and did not find any issues.
> > > > > > > > > > > > > Will try it on a topology generating WARC
> > > > > > > > > > > > > files next week to
> > > > > > > > > > > > > check that the
> > > > > > > > > > > > > dependency changes on Hadoop have not broken
> > > > > > > > > > > > > anything> > >> > >> > > > > >
> > > > > > > > > > > > > Have a good week end
> > > > > > > > > > > > >
> > > > > > > > > > > > > Julien
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla
> > > > > > > > > > > > > <rz...@apache.org>
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Short update. Sorted out an issue with the
> > > > > > > > > > > > > > tar.gz/zip with
> > > > > > > > > > > > > > Julien today
> > > > > > > > > > > > > > and re-uploaded them to the nightlies area.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > This new bundle works as expected in my
> > > > > > > > > > > > > > deployment but happy to
> > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > additional feedback before getting up a
> > > > > > > > > > > > > > first release candidate
> > > > > > > > > > > > > > ;-)
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100
> > > > > > > > > > > > > > schrieb Richard
> > > > > > > > > > > > > > Zowalla:
> > > > > > > > > > > > > > > Hi Alexandre,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > we are not in a hurry here :) - take as
> > > > > > > > > > > > > > > much as time you
> > > > > > > > > > > > > > > need.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100
> > > > > > > > > > > > > > > schrieb Alexandre
> > > > > > > > > > > > > > > Vermeerbergen:
> > > > > > > > > > > > > > > > Hello Richard,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Okay, I'm more than happy to do that
> > > > > > > > > > > > > > > > with my pre-production
> > > > > > > > > > > > > > > > cluster
> > > > > > > > > > > > > > > > (~10 topologies) using the binary
> > > > > > > > > > > > > > > > artifacts.
> > > > > > > > > > > > > > > > Would it be OK if I can use up to end
> > > > > > > > > > > > > > > > of this week so that
> > > > > > > > > > > > > > > > I'll be
> > > > > > > > > > > > > > > > able to have enough time to check all
> > > > > > > > > > > > > > > > potential issues that
> > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > upgrade bring?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > Alexandre
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard
> > > > > > > > > > > > > > > > Zowalla
> > > > > > > > > > > > > > > > <rz...@apache.org> a
> > > > > > > > > > > > > > > > écrit :
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > before starting a release of 2.6.0-
> > > > > > > > > > > > > > > > > SNAPSHOT, I would like
> > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > receive
> > > > > > > > > > > > > > > > > some feedback on the current SNAPSHOT
> > > > > > > > > > > > > > > > > build.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of
> > > > > > > > > > > > > > > > > Storm build from
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > It is available via the ASF Snapshot
> > > > > > > > > > > > > > > > > repository:
> > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > You can consume by adding
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > <repository>
> > > > > > > > > > > > > > > > > <id>apache.snapshots</id>
> > > > > > > > > > > > > > > > > <name>Apache Snapshot
> > > > > > > > > > > > > > > > > Repository</name>
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > <url>https://repository.apache.org/sn
> > > > > > > > > > > > > > > > > apshots</url>
> > > > > > > > > > > > > > > > > <releases>
> > > > > > > > > > > > > > > > > <enabled>false</enabled>
> > > > > > > > > > > > > > > > > </releases>
> > > > > > > > > > > > > > > > > </repository>
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > to your project pom. As we do not
> > > > > > > > > > > > > > > > > deploy SNAPSHOTS
> > > > > > > > > > > > > > > > > automatically,
> > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > should be easy to just consume the
> > > > > > > > > > > > > > > > > latest SNAPSHOT.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > The packaged binaries are available
> > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > nightlies.apache.org:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > If you have some minutes left: Please
> > > > > > > > > > > > > > > > > test and report any
> > > > > > > > > > > > > > > > > issues
> > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > this binaries, so we can fix before
> > > > > > > > > > > > > > > > > attempting to
> > > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > The most significant changes are in
> > > > > > > > > > > > > > > > > the are of
> > > > > > > > > > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > upgraded from 2.x to 3.x - our own
> > > > > > > > > > > > > > > > > test coverage within
> > > > > > > > > > > > > > > > > the build
> > > > > > > > > > > > > > > > > looks
> > > > > > > > > > > > > > > > > good but would be nice to get some
> > > > > > > > > > > > > > > > > real world use-case
> > > > > > > > > > > > > > > > > feedback
> > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > this. In addition, we had quite a lof
> > > > > > > > > > > > > > > > > of 3rd party
> > > > > > > > > > > > > > > > > dependency
> > > > > > > > > > > > > > > > > upgrades.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > In addition, it contains the pruning
> > > > > > > > > > > > > > > > > of external modules
> > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > listed
> > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > A summaryin Jira is here:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Gruß
> > > > > > > > > > > > > > > > > Richard
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > >
> > > > > > > > > > > > > *Open Source Solutions for Text Engineering*
> > > > > > > > > > > > >
> > > > > > > > > > > > > http://www.digitalpebble.com
> > > > > > > > > > > > > http://digitalpebble.blogspot.com/
> > > > > > > > > > > > > #digitalpebble
> > > > > > > > > > > > > <http://twitter.com/digitalpebble>
> > > > > > > > > >
>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Yes another regression I found with my topologies (built with 2.5.0
dependencies) running to Storm 2.6.0 SNAPSHOT 2023/11/02: I have seen
several exceptions like this one:
2023-11-04 18:43:31.696 o.a.s.e.e.ReportError
Thread-16-alertsHistoryUMToKafka-executor[27, 27] [ERROR] Error
java.lang.RuntimeException: java.lang.NoClassDefFoundError:
org.apache.storm.shade.org.json.simple.JSONObject
at org.apache.storm.utils.Utils$1.run(Utils.java:413)
~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
at java.lang.Thread.run(Thread.java:857) [?:?]
Caused by: java.lang.NoClassDefFoundError:
org.apache.storm.shade.org.json.simple.JSONObject
at com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessageFromTuple(AlertHistoryTupleToKafkaMapper.java:25)
~[stormjar.jar:?]
at com.acme.storm.evaluator.bolt.AlertHistoryTupleToKafkaMapper.getMessageFromTuple(AlertHistoryTupleToKafkaMapper.java:11)
~[stormjar.jar:?]
at org.apache.storm.kafka.bolt.KafkaBolt.process(KafkaBolt.java:177)
~[stormjar.jar:?]
and here's the related source code:
package com.acme.storm.evaluator.bolt;
import org.apache.storm.kafka.bolt.mapper.TupleToKafkaMapper;
import org.apache.storm.shade.org.json.simple.JSONObject;
import org.apache.storm.tuple.Tuple;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.acme.storm.util.TupleAlertHistoryFields;
public class AlertHistoryTupleToKafkaMapper implements
TupleToKafkaMapper<String, String> {
private static final long serialVersionUID = 5668103889461721134L;
public static final Logger logger =
LoggerFactory.getLogger(AlertHistoryTupleToKafkaMapper.class);
@Override
public String getKeyFromTuple(Tuple tuple) {
return tuple.getStringByField(TupleAlertHistoryFields.SERVICE_DEFINITION_NAME);
}
@SuppressWarnings("unchecked")
@Override
public String getMessageFromTuple(Tuple tuple)
{
JSONObject tupleJson = new JSONObject();
tuple.getFields().forEach(f -> {
tupleJson.put(f, tuple.getValueByField(f));
});
String jsonString = tupleJson.toJSONString();
return jsonString;
}
}
line #25 is this one:
JSONObject tupleJson = new JSONObject();
so, from what I understand, Storm 2.6.0 no longer exposes some shaded
classes such as:
org.apache.storm.shade.org.json.simple.JSONObject;
Interestingly, it looks similar to this issue:
https://stackoverflow.com/questions/33377291/java-lang-noclassdeffounderror-org-json-simple-jsonvalue-at-storm-kafka-dynamic
Whatever, I guess I can only blame our own code for using Storm's
shaded classes, right ?
Note it's been running as is for long time and not breaking since many
upgrades, of course it's not an excuse...
I'll fix our code & continue checking if I see other impacts of 2.5.0
-> 2.6.0 upgrade...
Thanks,
Alexandre
Le sam. 4 nov. 2023 à 10:36, Alexandre Vermeerbergen
<av...@gmail.com> a écrit :
>
> Just did a 'git push' with that trivial fix... is that enough?
>
> Le sam. 4 nov. 2023 à 09:34, Richard Zowalla <ri...@zowalla.com> a écrit :
> >
> > You can do as you like ;-)
> >
> > Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen <av...@gmail.com>:
> > >I mean :
> > >
> > >Can I keep using my
> > >'STORM-3958-Capacity-to-set-St
> > >orm-UI's-title-in-conf/storm.yaml'
> > >branch to push this fix, or should I create a new one?
> > >
> > >Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> > ><av...@gmail.com> a écrit :
> > >>
> > >> Can you keep using my
> > >> 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-conf/storm.yaml'
> > >> branch to push this fix, or should I create a new one?
> > >>
> > >> Le sam. 4 nov. 2023 à 08:14, Richard Zowalla <ri...@zowalla.com> a écrit :
> > >> >
> > >> > Just push a fix for it (referencing the original issue) ;-) - we hadn't a release in between, so not a big deal.
> > >> >
> > >> > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre Vermeerbergen <av...@gmail.com>:
> > >> > >Temporary workaround for the Storm UI title's regression: run this
> > >> > >command on the title-template.html file (in
> > >> > >storm-stable/public/templates directory) after having extracted the
> > >> > >binary archive:
> > >> > >
> > >> > >sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > >> > >
> > >> > >Again sorry for having introduced this regression.
> > >> > >
> > >> > >Alexandre
> > >> > >
> > >> > >Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > >> > ><av...@gmail.com> a écrit :
> > >> > >>
> > >> > >> Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
> > >> > >> 1. I have modified my KO bolts using this pattern in their prepare() method:
> > >> > >>
> > >> > >> long cacheMaxSize =
> > >> > >> Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
> > >> > >>
> > >> > >> instead of:
> > >> > >>
> > >> > >> long cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > >> > >>
> > >> > >> 2. I have found the cause of the regression in loss of 'Storm UI'
> > >> > >> title at the top left-hand side of all Storm UI's pages.
> > >> > >> This is a mistake of mine in this PR:
> > >> > >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > >> > >> More precisely, at line 19 of this file:
> > >> > >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > >> > >> we must have:
> > >> > >> <h1><a href="/">{{title}}</a></h1>
> > >> > >>
> > >> > >> instead of:
> > >> > >>
> > >> > >> <h1><a href="/">{{ui.title}}</a></h1>
> > >> > >>
> > >> > >> I think I missed this during my tests because of browser's cache
> > >> > >> (bad excuse, I know...): probably I tested by-patching my deployed
> > >> > >> Storm cluster, and I finally forgot to checkout this latest change of
> > >> > >> my delivery.
> > >> > >>
> > >> > >> I had to use "private navigation" to make sure this time it fixes this issue
> > >> > >>
> > >> > >> What would be the quickest way for this very simple fix to be delivered ?
> > >> > >>
> > >> > >> Thanks,
> > >> > >> Alexandre
> > >> > >>
> > >> > >> Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> > >> > >> <av...@gmail.com> a écrit :
> > >> > >> >
> > >> > >> > Thanks Richard for the analysis.
> > >> > >> >
> > >> > >> > My thought on this are very simple: let me fix my own old code, after
> > >> > >> > all this was impacting only one of my topologies, the other one were
> > >> > >> > using more careful Integer.parseInt or Integer.parseLong to avoid
> > >> > >> > making assumption on the way the values in the config Map were
> > >> > >> > serialized.
> > >> > >> >
> > >> > >> > My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
> > >> > >> > UI" title at the top left hand side of each page is missing. This may
> > >> > >> > be the result of one of my own PR, I'll have to check why...
> > >> > >> >
> > >> > >> > Alex
> > >> > >> >
> > >> > >> > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla <rz...@apache.org> a écrit :
> > >> > >> > >
> > >> > >> > > Hi,
> > >> > >> > >
> > >> > >> > > thanks for the feedback.
> > >> > >> > >
> > >> > >> > > The main difference to 2.5.0 is, that we switched the JSON
> > >> > >> > > implementation in Utils, which relied on a
> > >> > >> > > super old JSON library. What is happening here is, that the new JSON
> > >> > >> > > library (net.minidev) behaves differently.
> > >> > >> > >
> > >> > >> > > We can see it by writing some simple code without Storm (The related
> > >> > >> > > code line with the parsing is taken from Storm Utils)
> > >> > >> > >
> > >> > >> > >
> > >> > >> > > Map<String, Long> conf = new HashMap<>();
> > >> > >> > > conf.put("test", 100000000L);
> > >> > >> > >
> > >> > >> > > Map<String, Object> deserTopoConf = (Map<String, Object>)
> > >> > >> > > JSONValue.parseWithException(JSONValue.toJSONString(conf));
> > >> > >> > >
> > >> > >> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> > >> > >> > > System.out.println(e.getValue().getClass().getSimpleName());
> > >> > >> > > }
> > >> > >> > >
> > >> > >> > > This will print "Integer". Using "10000000000000000L" as a value will
> > >> > >> > > result in "Long".
> > >> > >> > >
> > >> > >> > > The main issue derives from the fact, that we serialize the config map
> > >> > >> > > into a JSON String and parse the result.
> > >> > >> > > Based on the String representation, "100000000L" will still fit into an
> > >> > >> > > Integer, so the parser will not use Long ;-)
> > >> > >> > >
> > >> > >> > > If the number is big enough, the parser will switch to a Long.
> > >> > >> > > The old library (> 10 years old now):
> > >> > >> > >
> > >> > >> > > deserTopoConf = (Map<String, Object>)
> > >> > >> > > org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
> > >> > >> > > toJSONString(conf));
> > >> > >> > >
> > >> > >> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> > >> > >> > > System.out.println(e.getValue().getClass().getSimpleName());
> > >> > >> > > }
> > >> > >> > >
> > >> > >> > > This will print "Long".
> > >> > >> > >
> > >> > >> > > The difference between these libraries is, that the super old
> > >> > >> > > org.json.simple just always returns Long as the type.
> > >> > >> > >
> > >> > >> > > If can eventually tune the new JSON parser to something like:
> > >> > >> > >
> > >> > >> > > deserTopoConf = (Map<String, Object>) new
> > >> > >> > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
> > >> > >> > > ing(conf), JSONValue.defaultReader.DEFAULT);
> > >> > >> > >
> > >> > >> > > This will result in "Long" but might have other side-effects.
> > >> > >> > >
> > >> > >> > > From my point of view, the newer JSON library works as expected
> > >> > >> > > (although it might break backward compatibility).
> > >> > >> > > Unchecked casts are always evil, imho. In your specific case, it might
> > >> > >> > > be more save to cast to "Number" and than invoke "longValue()" instead.
> > >> > >> > >
> > >> > >> > > Nevertheless, we might being able to restore the old behaviour by
> > >> > >> > > tuning the JSONParser configuration.
> > >> > >> > >
> > >> > >> > > Thoughts?
> > >> > >> > >
> > >> > >> > > Gruß
> > >> > >> > > Richard
> > >> > >> > >
> > >> > >> > >
> > >> > >> > >
> > >> > >> > >
> > >> > >> > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
> > >> > >> > > Vermeerbergen:
> > >> > >> > > > On a side note, the Long value that "MyKOBolt" is set by a
> > >> > >> > > > DefaultTopologyConfig to a value with is a java.lang.Long instance...
> > >> > >> > > > hence it's surprising to see it internally converted into a
> > >> > >> > > > java.lang.Integer in the Map passed to prepare() method...
> > >> > >> > > >
> > >> > >> > > >
> > >> > >> > > > Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
> > >> > >> > > > <av...@gmail.com> a écrit :
> > >> > >> > > > >
> > >> > >> > > > > Hello,
> > >> > >> > > > >
> > >> > >> > > > > I found a breaking change using Apache Storm 2.6.0 snapshot build
> > >> > >> > > > > dated 20231103 and without changing my topologies' code (keeping
> > >> > >> > > > > the
> > >> > >> > > > > same big jars).
> > >> > >> > > > >
> > >> > >> > > > > This seems to be tied to a datatype change in Map argument of
> > >> > >> > > > > prepare() method of Bolt.
> > >> > >> > > > >
> > >> > >> > > > > For example, this Bolt is KO at runtime:
> > >> > >> > > > >
> > >> > >> > > > > public class MyKOBolt extends BaseRichBolt {
> > >> > >> > > > >
> > >> > >> > > > > @Override
> > >> > >> > > > > public void prepare(Map stormConf, TopologyContext context,
> > >> > >> > > > > OutputCollector collector) {
> > >> > >> > > > > cacheMaxSize = (Long)
> > >> > >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > >> > >> > > > > cacheTtlMin = (Long)
> > >> > >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > >> > >> > > > >
> > >> > >> > > > > as it throws this error:
> > >> > >> > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > >> > >> > > > > Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
> > >> > >> > > > > __system:[-1]
> > >> > >> > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > >> > >> > > > > Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop
> > >> > >> > > > > died!
> > >> > >> > > > > java.lang.ClassCastException: java.lang.Integer incompatible with
> > >> > >> > > > > java.lang.Long
> > >> > >> > > > > at com.acme.storm.alerting.MyKOBolt
> > >> > >> > > > > .prepare(EvaluationBolt.java:71)
> > >> > >> > > > > ~[stormjar.jar:?]
> > >> > >> > > > > at
> > >> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:
> > >> > >> > > > > 128)
> > >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > >> > >> > > > > at
> > >> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > >> > >> > > > > 138)
> > >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > >> > >> > > > > at
> > >> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > >> > >> > > > > 54)
> > >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > >> > >> > > > > at org.apache.storm.utils.Utils$1.run(Utils.java:393)
> > >> > >> > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > >> > >> > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> > >> > >> > > > >
> > >> > >> > > > > whereas this bolt is fine with Storm 2.6.0:
> > >> > >> > > > >
> > >> > >> > > > > public class MyOKBolt extends BaseBasicBolt {
> > >> > >> > > > >
> > >> > >> > > > > @Override
> > >> > >> > > > > public void prepare(final Map stormConf, final TopologyContext
> > >> > >> > > > > context) {
> > >> > >> > > > > super.prepare(stormConf, context);
> > >> > >> > > > >
> > >> > >> > > > > this.redisPort =
> > >> > >> > > > > Integer.parseInt((String)
> > >> > >> > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > >> > >> > > > >
> > >> > >> > > > > That said, I can modify the code of "MyKOBolt" to use same
> > >> > >> > > > > Integer.parseInt or Integer.parseLong trick, but it's the first
> > >> > >> > > > > time
> > >> > >> > > > > in my long history of upgrades that I have seen such runtime
> > >> > >> > > > > incompatibility.
> > >> > >> > > > >
> > >> > >> > > > > Maybe "MyKOBolt" was badly written since the beginning and I have
> > >> > >> > > > > just
> > >> > >> > > > > hit the punishment for it: is there a documentation which clarifies
> > >> > >> > > > > datatypes of the Map argument of prepare() method?
> > >> > >> > > > >
> > >> > >> > > > > Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
> > >> > >> > > > > "MyOKBolt" derives from BaseBasicBolt => could this have any impact
> > >> > >> > > > > on
> > >> > >> > > > > this finding ?
> > >> > >> > > > >
> > >> > >> > > > > Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK
> > >> > >> > > > > 17.0.8.1.
> > >> > >> > > > >
> > >> > >> > > > > Kind regards,
> > >> > >> > > > > Alexandre
> > >> > >> > > > >
> > >> > >> > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > >> > >> > > > > <li...@gmail.com> a écrit :
> > >> > >> > > > > >
> > >> > >> > > > > > Thanks Richard.
> > >> > >> > > > > >
> > >> > >> > > > > > Tried the latest snapshot with StormCrawler both in local and
> > >> > >> > > > > > deployed mode
> > >> > >> > > > > > and did not find any issues.
> > >> > >> > > > > > Will try it on a topology generating WARC files next week to
> > >> > >> > > > > > check that the
> > >> > >> > > > > > dependency changes on Hadoop have not broken anything.
> > >> > >> > > > > >
> > >> > >> > > > > > Have a good week end
> > >> > >> > > > > >
> > >> > >> > > > > > Julien
> > >> > >> > > > > >
> > >> > >> > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <rz...@apache.org>
> > >> > >> > > > > > wrote:
> > >> > >> > > > > >
> > >> > >> > > > > > > Short update. Sorted out an issue with the tar.gz/zip with
> > >> > >> > > > > > > Julien today
> > >> > >> > > > > > > and re-uploaded them to the nightlies area.
> > >> > >> > > > > > >
> > >> > >> > > > > > > This new bundle works as expected in my deployment but happy to
> > >> > >> > > > > > > receive
> > >> > >> > > > > > > additional feedback before getting up a first release candidate
> > >> > >> > > > > > > ;-)
> > >> > >> > > > > > >
> > >> > >> > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard
> > >> > >> > > > > > > Zowalla:
> > >> > >> > > > > > > > Hi Alexandre,
> > >> > >> > > > > > > >
> > >> > >> > > > > > > > we are not in a hurry here :) - take as much as time you
> > >> > >> > > > > > > > need.
> > >> > >> > > > > > > >
> > >> > >> > > > > > > > Gruß
> > >> > >> > > > > > > > Richard
> > >> > >> > > > > > > >
> > >> > >> > > > > > > >
> > >> > >> > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > >> > >> > > > > > > > Vermeerbergen:
> > >> > >> > > > > > > > > Hello Richard,
> > >> > >> > > > > > > > >
> > >> > >> > > > > > > > > Okay, I'm more than happy to do that with my pre-production
> > >> > >> > > > > > > > > cluster
> > >> > >> > > > > > > > > (~10 topologies) using the binary artifacts.
> > >> > >> > > > > > > > > Would it be OK if I can use up to end of this week so that
> > >> > >> > > > > > > > > I'll be
> > >> > >> > > > > > > > > able to have enough time to check all potential issues that
> > >> > >> > > > > > > > > this
> > >> > >> > > > > > > > > upgrade bring?
> > >> > >> > > > > > > > >
> > >> > >> > > > > > > > > Thanks,
> > >> > >> > > > > > > > > Alexandre
> > >> > >> > > > > > > > >
> > >> > >> > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla
> > >> > >> > > > > > > > > <rz...@apache.org> a
> > >> > >> > > > > > > > > écrit :
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > > Hi all,
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > > before starting a release of 2.6.0-SNAPSHOT, I would like
> > >> > >> > > > > > > > > > to
> > >> > >> > > > > > > > > > receive
> > >> > >> > > > > > > > > > some feedback on the current SNAPSHOT build.
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > > It is available via the ASF Snapshot repository:
> > >> > >> > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > > You can consume by adding
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > > <repository>
> > >> > >> > > > > > > > > > <id>apache.snapshots</id>
> > >> > >> > > > > > > > > > <name>Apache Snapshot Repository</name>
> > >> > >> > > > > > > > > > <url>https://repository.apache.org/snapshots</url>
> > >> > >> > > > > > > > > > <releases>
> > >> > >> > > > > > > > > > <enabled>false</enabled>
> > >> > >> > > > > > > > > > </releases>
> > >> > >> > > > > > > > > > </repository>
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > > to your project pom. As we do not deploy SNAPSHOTS
> > >> > >> > > > > > > > > > automatically,
> > >> > >> > > > > > > > > > it
> > >> > >> > > > > > > > > > should be easy to just consume the latest SNAPSHOT.
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > > The packaged binaries are available at
> > >> > >> > > > > > > > > > nightlies.apache.org:
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > > If you have some minutes left: Please test and report any
> > >> > >> > > > > > > > > > issues
> > >> > >> > > > > > > > > > with
> > >> > >> > > > > > > > > > this binaries, so we can fix before attempting to
> > >> > >> > > > > > > > > > release.
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > > The most significant changes are in the are of
> > >> > >> > > > > > > > > > hadoop/hbase/hdfs
> > >> > >> > > > > > > > > > as
> > >> > >> > > > > > > > > > we
> > >> > >> > > > > > > > > > upgraded from 2.x to 3.x - our own test coverage within
> > >> > >> > > > > > > > > > the build
> > >> > >> > > > > > > > > > looks
> > >> > >> > > > > > > > > > good but would be nice to get some real world use-case
> > >> > >> > > > > > > > > > feedback
> > >> > >> > > > > > > > > > on
> > >> > >> > > > > > > > > > this. In addition, we had quite a lof of 3rd party
> > >> > >> > > > > > > > > > dependency
> > >> > >> > > > > > > > > > upgrades.
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > > In addition, it contains the pruning of external modules
> > >> > >> > > > > > > > > > as
> > >> > >> > > > > > > > > > listed
> > >> > >> > > > > > > > > > in
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > > A summaryin Jira is here:
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > > Gruß
> > >> > >> > > > > > > > > > Richard
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > > > >
> > >> > >> > > > > > > >
> > >> > >> > > > > > >
> > >> > >> > > > > > >
> > >> > >> > > > > >
> > >> > >> > > > > > --
> > >> > >> > > > > >
> > >> > >> > > > > > *Open Source Solutions for Text Engineering*
> > >> > >> > > > > >
> > >> > >> > > > > > http://www.digitalpebble.com
> > >> > >> > > > > > http://digitalpebble.blogspot.com/
> > >> > >> > > > > > #digitalpebble <http://twitter.com/digitalpebble>
> > >> > >> > >
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Just did a 'git push' with that trivial fix... is that enough?
Le sam. 4 nov. 2023 à 09:34, Richard Zowalla <ri...@zowalla.com> a écrit :
>
> You can do as you like ;-)
>
> Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen <av...@gmail.com>:
> >I mean :
> >
> >Can I keep using my
> >'STORM-3958-Capacity-to-set-St
> >orm-UI's-title-in-conf/storm.yaml'
> >branch to push this fix, or should I create a new one?
> >
> >Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
> ><av...@gmail.com> a écrit :
> >>
> >> Can you keep using my
> >> 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-conf/storm.yaml'
> >> branch to push this fix, or should I create a new one?
> >>
> >> Le sam. 4 nov. 2023 à 08:14, Richard Zowalla <ri...@zowalla.com> a écrit :
> >> >
> >> > Just push a fix for it (referencing the original issue) ;-) - we hadn't a release in between, so not a big deal.
> >> >
> >> > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre Vermeerbergen <av...@gmail.com>:
> >> > >Temporary workaround for the Storm UI title's regression: run this
> >> > >command on the title-template.html file (in
> >> > >storm-stable/public/templates directory) after having extracted the
> >> > >binary archive:
> >> > >
> >> > >sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> >> > >
> >> > >Again sorry for having introduced this regression.
> >> > >
> >> > >Alexandre
> >> > >
> >> > >Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> >> > ><av...@gmail.com> a écrit :
> >> > >>
> >> > >> Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
> >> > >> 1. I have modified my KO bolts using this pattern in their prepare() method:
> >> > >>
> >> > >> long cacheMaxSize =
> >> > >> Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
> >> > >>
> >> > >> instead of:
> >> > >>
> >> > >> long cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> >> > >>
> >> > >> 2. I have found the cause of the regression in loss of 'Storm UI'
> >> > >> title at the top left-hand side of all Storm UI's pages.
> >> > >> This is a mistake of mine in this PR:
> >> > >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> >> > >> More precisely, at line 19 of this file:
> >> > >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> >> > >> we must have:
> >> > >> <h1><a href="/">{{title}}</a></h1>
> >> > >>
> >> > >> instead of:
> >> > >>
> >> > >> <h1><a href="/">{{ui.title}}</a></h1>
> >> > >>
> >> > >> I think I missed this during my tests because of browser's cache
> >> > >> (bad excuse, I know...): probably I tested by-patching my deployed
> >> > >> Storm cluster, and I finally forgot to checkout this latest change of
> >> > >> my delivery.
> >> > >>
> >> > >> I had to use "private navigation" to make sure this time it fixes this issue
> >> > >>
> >> > >> What would be the quickest way for this very simple fix to be delivered ?
> >> > >>
> >> > >> Thanks,
> >> > >> Alexandre
> >> > >>
> >> > >> Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> >> > >> <av...@gmail.com> a écrit :
> >> > >> >
> >> > >> > Thanks Richard for the analysis.
> >> > >> >
> >> > >> > My thought on this are very simple: let me fix my own old code, after
> >> > >> > all this was impacting only one of my topologies, the other one were
> >> > >> > using more careful Integer.parseInt or Integer.parseLong to avoid
> >> > >> > making assumption on the way the values in the config Map were
> >> > >> > serialized.
> >> > >> >
> >> > >> > My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
> >> > >> > UI" title at the top left hand side of each page is missing. This may
> >> > >> > be the result of one of my own PR, I'll have to check why...
> >> > >> >
> >> > >> > Alex
> >> > >> >
> >> > >> > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla <rz...@apache.org> a écrit :
> >> > >> > >
> >> > >> > > Hi,
> >> > >> > >
> >> > >> > > thanks for the feedback.
> >> > >> > >
> >> > >> > > The main difference to 2.5.0 is, that we switched the JSON
> >> > >> > > implementation in Utils, which relied on a
> >> > >> > > super old JSON library. What is happening here is, that the new JSON
> >> > >> > > library (net.minidev) behaves differently.
> >> > >> > >
> >> > >> > > We can see it by writing some simple code without Storm (The related
> >> > >> > > code line with the parsing is taken from Storm Utils)
> >> > >> > >
> >> > >> > >
> >> > >> > > Map<String, Long> conf = new HashMap<>();
> >> > >> > > conf.put("test", 100000000L);
> >> > >> > >
> >> > >> > > Map<String, Object> deserTopoConf = (Map<String, Object>)
> >> > >> > > JSONValue.parseWithException(JSONValue.toJSONString(conf));
> >> > >> > >
> >> > >> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> >> > >> > > System.out.println(e.getValue().getClass().getSimpleName());
> >> > >> > > }
> >> > >> > >
> >> > >> > > This will print "Integer". Using "10000000000000000L" as a value will
> >> > >> > > result in "Long".
> >> > >> > >
> >> > >> > > The main issue derives from the fact, that we serialize the config map
> >> > >> > > into a JSON String and parse the result.
> >> > >> > > Based on the String representation, "100000000L" will still fit into an
> >> > >> > > Integer, so the parser will not use Long ;-)
> >> > >> > >
> >> > >> > > If the number is big enough, the parser will switch to a Long.
> >> > >> > > The old library (> 10 years old now):
> >> > >> > >
> >> > >> > > deserTopoConf = (Map<String, Object>)
> >> > >> > > org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
> >> > >> > > toJSONString(conf));
> >> > >> > >
> >> > >> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> >> > >> > > System.out.println(e.getValue().getClass().getSimpleName());
> >> > >> > > }
> >> > >> > >
> >> > >> > > This will print "Long".
> >> > >> > >
> >> > >> > > The difference between these libraries is, that the super old
> >> > >> > > org.json.simple just always returns Long as the type.
> >> > >> > >
> >> > >> > > If can eventually tune the new JSON parser to something like:
> >> > >> > >
> >> > >> > > deserTopoConf = (Map<String, Object>) new
> >> > >> > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
> >> > >> > > ing(conf), JSONValue.defaultReader.DEFAULT);
> >> > >> > >
> >> > >> > > This will result in "Long" but might have other side-effects.
> >> > >> > >
> >> > >> > > From my point of view, the newer JSON library works as expected
> >> > >> > > (although it might break backward compatibility).
> >> > >> > > Unchecked casts are always evil, imho. In your specific case, it might
> >> > >> > > be more save to cast to "Number" and than invoke "longValue()" instead.
> >> > >> > >
> >> > >> > > Nevertheless, we might being able to restore the old behaviour by
> >> > >> > > tuning the JSONParser configuration.
> >> > >> > >
> >> > >> > > Thoughts?
> >> > >> > >
> >> > >> > > Gruß
> >> > >> > > Richard
> >> > >> > >
> >> > >> > >
> >> > >> > >
> >> > >> > >
> >> > >> > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
> >> > >> > > Vermeerbergen:
> >> > >> > > > On a side note, the Long value that "MyKOBolt" is set by a
> >> > >> > > > DefaultTopologyConfig to a value with is a java.lang.Long instance...
> >> > >> > > > hence it's surprising to see it internally converted into a
> >> > >> > > > java.lang.Integer in the Map passed to prepare() method...
> >> > >> > > >
> >> > >> > > >
> >> > >> > > > Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
> >> > >> > > > <av...@gmail.com> a écrit :
> >> > >> > > > >
> >> > >> > > > > Hello,
> >> > >> > > > >
> >> > >> > > > > I found a breaking change using Apache Storm 2.6.0 snapshot build
> >> > >> > > > > dated 20231103 and without changing my topologies' code (keeping
> >> > >> > > > > the
> >> > >> > > > > same big jars).
> >> > >> > > > >
> >> > >> > > > > This seems to be tied to a datatype change in Map argument of
> >> > >> > > > > prepare() method of Bolt.
> >> > >> > > > >
> >> > >> > > > > For example, this Bolt is KO at runtime:
> >> > >> > > > >
> >> > >> > > > > public class MyKOBolt extends BaseRichBolt {
> >> > >> > > > >
> >> > >> > > > > @Override
> >> > >> > > > > public void prepare(Map stormConf, TopologyContext context,
> >> > >> > > > > OutputCollector collector) {
> >> > >> > > > > cacheMaxSize = (Long)
> >> > >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> >> > >> > > > > cacheTtlMin = (Long)
> >> > >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> >> > >> > > > >
> >> > >> > > > > as it throws this error:
> >> > >> > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> >> > >> > > > > Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
> >> > >> > > > > __system:[-1]
> >> > >> > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> >> > >> > > > > Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop
> >> > >> > > > > died!
> >> > >> > > > > java.lang.ClassCastException: java.lang.Integer incompatible with
> >> > >> > > > > java.lang.Long
> >> > >> > > > > at com.acme.storm.alerting.MyKOBolt
> >> > >> > > > > .prepare(EvaluationBolt.java:71)
> >> > >> > > > > ~[stormjar.jar:?]
> >> > >> > > > > at
> >> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:
> >> > >> > > > > 128)
> >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> >> > >> > > > > at
> >> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> >> > >> > > > > 138)
> >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> >> > >> > > > > at
> >> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> >> > >> > > > > 54)
> >> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> >> > >> > > > > at org.apache.storm.utils.Utils$1.run(Utils.java:393)
> >> > >> > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> >> > >> > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> >> > >> > > > >
> >> > >> > > > > whereas this bolt is fine with Storm 2.6.0:
> >> > >> > > > >
> >> > >> > > > > public class MyOKBolt extends BaseBasicBolt {
> >> > >> > > > >
> >> > >> > > > > @Override
> >> > >> > > > > public void prepare(final Map stormConf, final TopologyContext
> >> > >> > > > > context) {
> >> > >> > > > > super.prepare(stormConf, context);
> >> > >> > > > >
> >> > >> > > > > this.redisPort =
> >> > >> > > > > Integer.parseInt((String)
> >> > >> > > > > stormConf.get(ConfKeys.REDIS_PORT));
> >> > >> > > > >
> >> > >> > > > > That said, I can modify the code of "MyKOBolt" to use same
> >> > >> > > > > Integer.parseInt or Integer.parseLong trick, but it's the first
> >> > >> > > > > time
> >> > >> > > > > in my long history of upgrades that I have seen such runtime
> >> > >> > > > > incompatibility.
> >> > >> > > > >
> >> > >> > > > > Maybe "MyKOBolt" was badly written since the beginning and I have
> >> > >> > > > > just
> >> > >> > > > > hit the punishment for it: is there a documentation which clarifies
> >> > >> > > > > datatypes of the Map argument of prepare() method?
> >> > >> > > > >
> >> > >> > > > > Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
> >> > >> > > > > "MyOKBolt" derives from BaseBasicBolt => could this have any impact
> >> > >> > > > > on
> >> > >> > > > > this finding ?
> >> > >> > > > >
> >> > >> > > > > Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK
> >> > >> > > > > 17.0.8.1.
> >> > >> > > > >
> >> > >> > > > > Kind regards,
> >> > >> > > > > Alexandre
> >> > >> > > > >
> >> > >> > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> >> > >> > > > > <li...@gmail.com> a écrit :
> >> > >> > > > > >
> >> > >> > > > > > Thanks Richard.
> >> > >> > > > > >
> >> > >> > > > > > Tried the latest snapshot with StormCrawler both in local and
> >> > >> > > > > > deployed mode
> >> > >> > > > > > and did not find any issues.
> >> > >> > > > > > Will try it on a topology generating WARC files next week to
> >> > >> > > > > > check that the
> >> > >> > > > > > dependency changes on Hadoop have not broken anything.
> >> > >> > > > > >
> >> > >> > > > > > Have a good week end
> >> > >> > > > > >
> >> > >> > > > > > Julien
> >> > >> > > > > >
> >> > >> > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <rz...@apache.org>
> >> > >> > > > > > wrote:
> >> > >> > > > > >
> >> > >> > > > > > > Short update. Sorted out an issue with the tar.gz/zip with
> >> > >> > > > > > > Julien today
> >> > >> > > > > > > and re-uploaded them to the nightlies area.
> >> > >> > > > > > >
> >> > >> > > > > > > This new bundle works as expected in my deployment but happy to
> >> > >> > > > > > > receive
> >> > >> > > > > > > additional feedback before getting up a first release candidate
> >> > >> > > > > > > ;-)
> >> > >> > > > > > >
> >> > >> > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard
> >> > >> > > > > > > Zowalla:
> >> > >> > > > > > > > Hi Alexandre,
> >> > >> > > > > > > >
> >> > >> > > > > > > > we are not in a hurry here :) - take as much as time you
> >> > >> > > > > > > > need.
> >> > >> > > > > > > >
> >> > >> > > > > > > > Gruß
> >> > >> > > > > > > > Richard
> >> > >> > > > > > > >
> >> > >> > > > > > > >
> >> > >> > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> >> > >> > > > > > > > Vermeerbergen:
> >> > >> > > > > > > > > Hello Richard,
> >> > >> > > > > > > > >
> >> > >> > > > > > > > > Okay, I'm more than happy to do that with my pre-production
> >> > >> > > > > > > > > cluster
> >> > >> > > > > > > > > (~10 topologies) using the binary artifacts.
> >> > >> > > > > > > > > Would it be OK if I can use up to end of this week so that
> >> > >> > > > > > > > > I'll be
> >> > >> > > > > > > > > able to have enough time to check all potential issues that
> >> > >> > > > > > > > > this
> >> > >> > > > > > > > > upgrade bring?
> >> > >> > > > > > > > >
> >> > >> > > > > > > > > Thanks,
> >> > >> > > > > > > > > Alexandre
> >> > >> > > > > > > > >
> >> > >> > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla
> >> > >> > > > > > > > > <rz...@apache.org> a
> >> > >> > > > > > > > > écrit :
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > > Hi all,
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > > before starting a release of 2.6.0-SNAPSHOT, I would like
> >> > >> > > > > > > > > > to
> >> > >> > > > > > > > > > receive
> >> > >> > > > > > > > > > some feedback on the current SNAPSHOT build.
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> >> > >> > > > > > > > > >
> >> > >> > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > > It is available via the ASF Snapshot repository:
> >> > >> > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > > You can consume by adding
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > > <repository>
> >> > >> > > > > > > > > > <id>apache.snapshots</id>
> >> > >> > > > > > > > > > <name>Apache Snapshot Repository</name>
> >> > >> > > > > > > > > > <url>https://repository.apache.org/snapshots</url>
> >> > >> > > > > > > > > > <releases>
> >> > >> > > > > > > > > > <enabled>false</enabled>
> >> > >> > > > > > > > > > </releases>
> >> > >> > > > > > > > > > </repository>
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > > to your project pom. As we do not deploy SNAPSHOTS
> >> > >> > > > > > > > > > automatically,
> >> > >> > > > > > > > > > it
> >> > >> > > > > > > > > > should be easy to just consume the latest SNAPSHOT.
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > > The packaged binaries are available at
> >> > >> > > > > > > > > > nightlies.apache.org:
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > >
> >> > >> > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > > If you have some minutes left: Please test and report any
> >> > >> > > > > > > > > > issues
> >> > >> > > > > > > > > > with
> >> > >> > > > > > > > > > this binaries, so we can fix before attempting to
> >> > >> > > > > > > > > > release.
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > > The most significant changes are in the are of
> >> > >> > > > > > > > > > hadoop/hbase/hdfs
> >> > >> > > > > > > > > > as
> >> > >> > > > > > > > > > we
> >> > >> > > > > > > > > > upgraded from 2.x to 3.x - our own test coverage within
> >> > >> > > > > > > > > > the build
> >> > >> > > > > > > > > > looks
> >> > >> > > > > > > > > > good but would be nice to get some real world use-case
> >> > >> > > > > > > > > > feedback
> >> > >> > > > > > > > > > on
> >> > >> > > > > > > > > > this. In addition, we had quite a lof of 3rd party
> >> > >> > > > > > > > > > dependency
> >> > >> > > > > > > > > > upgrades.
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > > In addition, it contains the pruning of external modules
> >> > >> > > > > > > > > > as
> >> > >> > > > > > > > > > listed
> >> > >> > > > > > > > > > in
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > > A summaryin Jira is here:
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > >
> >> > >> > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > > Gruß
> >> > >> > > > > > > > > > Richard
> >> > >> > > > > > > > > >
> >> > >> > > > > > > > > >
> >> > >> > > > > > > >
> >> > >> > > > > > >
> >> > >> > > > > > >
> >> > >> > > > > >
> >> > >> > > > > > --
> >> > >> > > > > >
> >> > >> > > > > > *Open Source Solutions for Text Engineering*
> >> > >> > > > > >
> >> > >> > > > > > http://www.digitalpebble.com
> >> > >> > > > > > http://digitalpebble.blogspot.com/
> >> > >> > > > > > #digitalpebble <http://twitter.com/digitalpebble>
> >> > >> > >
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Richard Zowalla <ri...@zowalla.com>.
You can do as you like ;-)
Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen <av...@gmail.com>:
>I mean :
>
>Can I keep using my
>'STORM-3958-Capacity-to-set-St
>orm-UI's-title-in-conf/storm.yaml'
>branch to push this fix, or should I create a new one?
>
>Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
><av...@gmail.com> a écrit :
>>
>> Can you keep using my
>> 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-conf/storm.yaml'
>> branch to push this fix, or should I create a new one?
>>
>> Le sam. 4 nov. 2023 à 08:14, Richard Zowalla <ri...@zowalla.com> a écrit :
>> >
>> > Just push a fix for it (referencing the original issue) ;-) - we hadn't a release in between, so not a big deal.
>> >
>> > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre Vermeerbergen <av...@gmail.com>:
>> > >Temporary workaround for the Storm UI title's regression: run this
>> > >command on the title-template.html file (in
>> > >storm-stable/public/templates directory) after having extracted the
>> > >binary archive:
>> > >
>> > >sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
>> > >
>> > >Again sorry for having introduced this regression.
>> > >
>> > >Alexandre
>> > >
>> > >Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
>> > ><av...@gmail.com> a écrit :
>> > >>
>> > >> Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
>> > >> 1. I have modified my KO bolts using this pattern in their prepare() method:
>> > >>
>> > >> long cacheMaxSize =
>> > >> Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
>> > >>
>> > >> instead of:
>> > >>
>> > >> long cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
>> > >>
>> > >> 2. I have found the cause of the regression in loss of 'Storm UI'
>> > >> title at the top left-hand side of all Storm UI's pages.
>> > >> This is a mistake of mine in this PR:
>> > >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
>> > >> More precisely, at line 19 of this file:
>> > >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
>> > >> we must have:
>> > >> <h1><a href="/">{{title}}</a></h1>
>> > >>
>> > >> instead of:
>> > >>
>> > >> <h1><a href="/">{{ui.title}}</a></h1>
>> > >>
>> > >> I think I missed this during my tests because of browser's cache
>> > >> (bad excuse, I know...): probably I tested by-patching my deployed
>> > >> Storm cluster, and I finally forgot to checkout this latest change of
>> > >> my delivery.
>> > >>
>> > >> I had to use "private navigation" to make sure this time it fixes this issue
>> > >>
>> > >> What would be the quickest way for this very simple fix to be delivered ?
>> > >>
>> > >> Thanks,
>> > >> Alexandre
>> > >>
>> > >> Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
>> > >> <av...@gmail.com> a écrit :
>> > >> >
>> > >> > Thanks Richard for the analysis.
>> > >> >
>> > >> > My thought on this are very simple: let me fix my own old code, after
>> > >> > all this was impacting only one of my topologies, the other one were
>> > >> > using more careful Integer.parseInt or Integer.parseLong to avoid
>> > >> > making assumption on the way the values in the config Map were
>> > >> > serialized.
>> > >> >
>> > >> > My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
>> > >> > UI" title at the top left hand side of each page is missing. This may
>> > >> > be the result of one of my own PR, I'll have to check why...
>> > >> >
>> > >> > Alex
>> > >> >
>> > >> > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla <rz...@apache.org> a écrit :
>> > >> > >
>> > >> > > Hi,
>> > >> > >
>> > >> > > thanks for the feedback.
>> > >> > >
>> > >> > > The main difference to 2.5.0 is, that we switched the JSON
>> > >> > > implementation in Utils, which relied on a
>> > >> > > super old JSON library. What is happening here is, that the new JSON
>> > >> > > library (net.minidev) behaves differently.
>> > >> > >
>> > >> > > We can see it by writing some simple code without Storm (The related
>> > >> > > code line with the parsing is taken from Storm Utils)
>> > >> > >
>> > >> > >
>> > >> > > Map<String, Long> conf = new HashMap<>();
>> > >> > > conf.put("test", 100000000L);
>> > >> > >
>> > >> > > Map<String, Object> deserTopoConf = (Map<String, Object>)
>> > >> > > JSONValue.parseWithException(JSONValue.toJSONString(conf));
>> > >> > >
>> > >> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
>> > >> > > System.out.println(e.getValue().getClass().getSimpleName());
>> > >> > > }
>> > >> > >
>> > >> > > This will print "Integer". Using "10000000000000000L" as a value will
>> > >> > > result in "Long".
>> > >> > >
>> > >> > > The main issue derives from the fact, that we serialize the config map
>> > >> > > into a JSON String and parse the result.
>> > >> > > Based on the String representation, "100000000L" will still fit into an
>> > >> > > Integer, so the parser will not use Long ;-)
>> > >> > >
>> > >> > > If the number is big enough, the parser will switch to a Long.
>> > >> > > The old library (> 10 years old now):
>> > >> > >
>> > >> > > deserTopoConf = (Map<String, Object>)
>> > >> > > org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
>> > >> > > toJSONString(conf));
>> > >> > >
>> > >> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
>> > >> > > System.out.println(e.getValue().getClass().getSimpleName());
>> > >> > > }
>> > >> > >
>> > >> > > This will print "Long".
>> > >> > >
>> > >> > > The difference between these libraries is, that the super old
>> > >> > > org.json.simple just always returns Long as the type.
>> > >> > >
>> > >> > > If can eventually tune the new JSON parser to something like:
>> > >> > >
>> > >> > > deserTopoConf = (Map<String, Object>) new
>> > >> > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
>> > >> > > ing(conf), JSONValue.defaultReader.DEFAULT);
>> > >> > >
>> > >> > > This will result in "Long" but might have other side-effects.
>> > >> > >
>> > >> > > From my point of view, the newer JSON library works as expected
>> > >> > > (although it might break backward compatibility).
>> > >> > > Unchecked casts are always evil, imho. In your specific case, it might
>> > >> > > be more save to cast to "Number" and than invoke "longValue()" instead.
>> > >> > >
>> > >> > > Nevertheless, we might being able to restore the old behaviour by
>> > >> > > tuning the JSONParser configuration.
>> > >> > >
>> > >> > > Thoughts?
>> > >> > >
>> > >> > > Gruß
>> > >> > > Richard
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
>> > >> > > Vermeerbergen:
>> > >> > > > On a side note, the Long value that "MyKOBolt" is set by a
>> > >> > > > DefaultTopologyConfig to a value with is a java.lang.Long instance...
>> > >> > > > hence it's surprising to see it internally converted into a
>> > >> > > > java.lang.Integer in the Map passed to prepare() method...
>> > >> > > >
>> > >> > > >
>> > >> > > > Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
>> > >> > > > <av...@gmail.com> a écrit :
>> > >> > > > >
>> > >> > > > > Hello,
>> > >> > > > >
>> > >> > > > > I found a breaking change using Apache Storm 2.6.0 snapshot build
>> > >> > > > > dated 20231103 and without changing my topologies' code (keeping
>> > >> > > > > the
>> > >> > > > > same big jars).
>> > >> > > > >
>> > >> > > > > This seems to be tied to a datatype change in Map argument of
>> > >> > > > > prepare() method of Bolt.
>> > >> > > > >
>> > >> > > > > For example, this Bolt is KO at runtime:
>> > >> > > > >
>> > >> > > > > public class MyKOBolt extends BaseRichBolt {
>> > >> > > > >
>> > >> > > > > @Override
>> > >> > > > > public void prepare(Map stormConf, TopologyContext context,
>> > >> > > > > OutputCollector collector) {
>> > >> > > > > cacheMaxSize = (Long)
>> > >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
>> > >> > > > > cacheTtlMin = (Long)
>> > >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
>> > >> > > > >
>> > >> > > > > as it throws this error:
>> > >> > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
>> > >> > > > > Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
>> > >> > > > > __system:[-1]
>> > >> > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
>> > >> > > > > Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop
>> > >> > > > > died!
>> > >> > > > > java.lang.ClassCastException: java.lang.Integer incompatible with
>> > >> > > > > java.lang.Long
>> > >> > > > > at com.acme.storm.alerting.MyKOBolt
>> > >> > > > > .prepare(EvaluationBolt.java:71)
>> > >> > > > > ~[stormjar.jar:?]
>> > >> > > > > at
>> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:
>> > >> > > > > 128)
>> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
>> > >> > > > > at
>> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
>> > >> > > > > 138)
>> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
>> > >> > > > > at
>> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
>> > >> > > > > 54)
>> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
>> > >> > > > > at org.apache.storm.utils.Utils$1.run(Utils.java:393)
>> > >> > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
>> > >> > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
>> > >> > > > >
>> > >> > > > > whereas this bolt is fine with Storm 2.6.0:
>> > >> > > > >
>> > >> > > > > public class MyOKBolt extends BaseBasicBolt {
>> > >> > > > >
>> > >> > > > > @Override
>> > >> > > > > public void prepare(final Map stormConf, final TopologyContext
>> > >> > > > > context) {
>> > >> > > > > super.prepare(stormConf, context);
>> > >> > > > >
>> > >> > > > > this.redisPort =
>> > >> > > > > Integer.parseInt((String)
>> > >> > > > > stormConf.get(ConfKeys.REDIS_PORT));
>> > >> > > > >
>> > >> > > > > That said, I can modify the code of "MyKOBolt" to use same
>> > >> > > > > Integer.parseInt or Integer.parseLong trick, but it's the first
>> > >> > > > > time
>> > >> > > > > in my long history of upgrades that I have seen such runtime
>> > >> > > > > incompatibility.
>> > >> > > > >
>> > >> > > > > Maybe "MyKOBolt" was badly written since the beginning and I have
>> > >> > > > > just
>> > >> > > > > hit the punishment for it: is there a documentation which clarifies
>> > >> > > > > datatypes of the Map argument of prepare() method?
>> > >> > > > >
>> > >> > > > > Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
>> > >> > > > > "MyOKBolt" derives from BaseBasicBolt => could this have any impact
>> > >> > > > > on
>> > >> > > > > this finding ?
>> > >> > > > >
>> > >> > > > > Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK
>> > >> > > > > 17.0.8.1.
>> > >> > > > >
>> > >> > > > > Kind regards,
>> > >> > > > > Alexandre
>> > >> > > > >
>> > >> > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
>> > >> > > > > <li...@gmail.com> a écrit :
>> > >> > > > > >
>> > >> > > > > > Thanks Richard.
>> > >> > > > > >
>> > >> > > > > > Tried the latest snapshot with StormCrawler both in local and
>> > >> > > > > > deployed mode
>> > >> > > > > > and did not find any issues.
>> > >> > > > > > Will try it on a topology generating WARC files next week to
>> > >> > > > > > check that the
>> > >> > > > > > dependency changes on Hadoop have not broken anything.
>> > >> > > > > >
>> > >> > > > > > Have a good week end
>> > >> > > > > >
>> > >> > > > > > Julien
>> > >> > > > > >
>> > >> > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <rz...@apache.org>
>> > >> > > > > > wrote:
>> > >> > > > > >
>> > >> > > > > > > Short update. Sorted out an issue with the tar.gz/zip with
>> > >> > > > > > > Julien today
>> > >> > > > > > > and re-uploaded them to the nightlies area.
>> > >> > > > > > >
>> > >> > > > > > > This new bundle works as expected in my deployment but happy to
>> > >> > > > > > > receive
>> > >> > > > > > > additional feedback before getting up a first release candidate
>> > >> > > > > > > ;-)
>> > >> > > > > > >
>> > >> > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard
>> > >> > > > > > > Zowalla:
>> > >> > > > > > > > Hi Alexandre,
>> > >> > > > > > > >
>> > >> > > > > > > > we are not in a hurry here :) - take as much as time you
>> > >> > > > > > > > need.
>> > >> > > > > > > >
>> > >> > > > > > > > Gruß
>> > >> > > > > > > > Richard
>> > >> > > > > > > >
>> > >> > > > > > > >
>> > >> > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
>> > >> > > > > > > > Vermeerbergen:
>> > >> > > > > > > > > Hello Richard,
>> > >> > > > > > > > >
>> > >> > > > > > > > > Okay, I'm more than happy to do that with my pre-production
>> > >> > > > > > > > > cluster
>> > >> > > > > > > > > (~10 topologies) using the binary artifacts.
>> > >> > > > > > > > > Would it be OK if I can use up to end of this week so that
>> > >> > > > > > > > > I'll be
>> > >> > > > > > > > > able to have enough time to check all potential issues that
>> > >> > > > > > > > > this
>> > >> > > > > > > > > upgrade bring?
>> > >> > > > > > > > >
>> > >> > > > > > > > > Thanks,
>> > >> > > > > > > > > Alexandre
>> > >> > > > > > > > >
>> > >> > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla
>> > >> > > > > > > > > <rz...@apache.org> a
>> > >> > > > > > > > > écrit :
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > Hi all,
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > before starting a release of 2.6.0-SNAPSHOT, I would like
>> > >> > > > > > > > > > to
>> > >> > > > > > > > > > receive
>> > >> > > > > > > > > > some feedback on the current SNAPSHOT build.
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
>> > >> > > > > > > > > >
>> > >> > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > It is available via the ASF Snapshot repository:
>> > >> > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > You can consume by adding
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > <repository>
>> > >> > > > > > > > > > <id>apache.snapshots</id>
>> > >> > > > > > > > > > <name>Apache Snapshot Repository</name>
>> > >> > > > > > > > > > <url>https://repository.apache.org/snapshots</url>
>> > >> > > > > > > > > > <releases>
>> > >> > > > > > > > > > <enabled>false</enabled>
>> > >> > > > > > > > > > </releases>
>> > >> > > > > > > > > > </repository>
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > to your project pom. As we do not deploy SNAPSHOTS
>> > >> > > > > > > > > > automatically,
>> > >> > > > > > > > > > it
>> > >> > > > > > > > > > should be easy to just consume the latest SNAPSHOT.
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > The packaged binaries are available at
>> > >> > > > > > > > > > nightlies.apache.org:
>> > >> > > > > > > > > >
>> > >> > > > > > > > > >
>> > >> > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > If you have some minutes left: Please test and report any
>> > >> > > > > > > > > > issues
>> > >> > > > > > > > > > with
>> > >> > > > > > > > > > this binaries, so we can fix before attempting to
>> > >> > > > > > > > > > release.
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > The most significant changes are in the are of
>> > >> > > > > > > > > > hadoop/hbase/hdfs
>> > >> > > > > > > > > > as
>> > >> > > > > > > > > > we
>> > >> > > > > > > > > > upgraded from 2.x to 3.x - our own test coverage within
>> > >> > > > > > > > > > the build
>> > >> > > > > > > > > > looks
>> > >> > > > > > > > > > good but would be nice to get some real world use-case
>> > >> > > > > > > > > > feedback
>> > >> > > > > > > > > > on
>> > >> > > > > > > > > > this. In addition, we had quite a lof of 3rd party
>> > >> > > > > > > > > > dependency
>> > >> > > > > > > > > > upgrades.
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > In addition, it contains the pruning of external modules
>> > >> > > > > > > > > > as
>> > >> > > > > > > > > > listed
>> > >> > > > > > > > > > in
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > A summaryin Jira is here:
>> > >> > > > > > > > > >
>> > >> > > > > > > > > >
>> > >> > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > Gruß
>> > >> > > > > > > > > > Richard
>> > >> > > > > > > > > >
>> > >> > > > > > > > > >
>> > >> > > > > > > >
>> > >> > > > > > >
>> > >> > > > > > >
>> > >> > > > > >
>> > >> > > > > > --
>> > >> > > > > >
>> > >> > > > > > *Open Source Solutions for Text Engineering*
>> > >> > > > > >
>> > >> > > > > > http://www.digitalpebble.com
>> > >> > > > > > http://digitalpebble.blogspot.com/
>> > >> > > > > > #digitalpebble <http://twitter.com/digitalpebble>
>> > >> > >
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
I mean :
Can I keep using my
'STORM-3958-Capacity-to-set-St
orm-UI's-title-in-conf/storm.yaml'
branch to push this fix, or should I create a new one?
Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
<av...@gmail.com> a écrit :
>
> Can you keep using my
> 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-conf/storm.yaml'
> branch to push this fix, or should I create a new one?
>
> Le sam. 4 nov. 2023 à 08:14, Richard Zowalla <ri...@zowalla.com> a écrit :
> >
> > Just push a fix for it (referencing the original issue) ;-) - we hadn't a release in between, so not a big deal.
> >
> > Am 4. November 2023 08:01:54 MEZ schrieb Alexandre Vermeerbergen <av...@gmail.com>:
> > >Temporary workaround for the Storm UI title's regression: run this
> > >command on the title-template.html file (in
> > >storm-stable/public/templates directory) after having extracted the
> > >binary archive:
> > >
> > >sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> > >
> > >Again sorry for having introduced this regression.
> > >
> > >Alexandre
> > >
> > >Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> > ><av...@gmail.com> a écrit :
> > >>
> > >> Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
> > >> 1. I have modified my KO bolts using this pattern in their prepare() method:
> > >>
> > >> long cacheMaxSize =
> > >> Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
> > >>
> > >> instead of:
> > >>
> > >> long cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > >>
> > >> 2. I have found the cause of the regression in loss of 'Storm UI'
> > >> title at the top left-hand side of all Storm UI's pages.
> > >> This is a mistake of mine in this PR:
> > >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> > >> More precisely, at line 19 of this file:
> > >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> > >> we must have:
> > >> <h1><a href="/">{{title}}</a></h1>
> > >>
> > >> instead of:
> > >>
> > >> <h1><a href="/">{{ui.title}}</a></h1>
> > >>
> > >> I think I missed this during my tests because of browser's cache
> > >> (bad excuse, I know...): probably I tested by-patching my deployed
> > >> Storm cluster, and I finally forgot to checkout this latest change of
> > >> my delivery.
> > >>
> > >> I had to use "private navigation" to make sure this time it fixes this issue
> > >>
> > >> What would be the quickest way for this very simple fix to be delivered ?
> > >>
> > >> Thanks,
> > >> Alexandre
> > >>
> > >> Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> > >> <av...@gmail.com> a écrit :
> > >> >
> > >> > Thanks Richard for the analysis.
> > >> >
> > >> > My thought on this are very simple: let me fix my own old code, after
> > >> > all this was impacting only one of my topologies, the other one were
> > >> > using more careful Integer.parseInt or Integer.parseLong to avoid
> > >> > making assumption on the way the values in the config Map were
> > >> > serialized.
> > >> >
> > >> > My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
> > >> > UI" title at the top left hand side of each page is missing. This may
> > >> > be the result of one of my own PR, I'll have to check why...
> > >> >
> > >> > Alex
> > >> >
> > >> > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla <rz...@apache.org> a écrit :
> > >> > >
> > >> > > Hi,
> > >> > >
> > >> > > thanks for the feedback.
> > >> > >
> > >> > > The main difference to 2.5.0 is, that we switched the JSON
> > >> > > implementation in Utils, which relied on a
> > >> > > super old JSON library. What is happening here is, that the new JSON
> > >> > > library (net.minidev) behaves differently.
> > >> > >
> > >> > > We can see it by writing some simple code without Storm (The related
> > >> > > code line with the parsing is taken from Storm Utils)
> > >> > >
> > >> > >
> > >> > > Map<String, Long> conf = new HashMap<>();
> > >> > > conf.put("test", 100000000L);
> > >> > >
> > >> > > Map<String, Object> deserTopoConf = (Map<String, Object>)
> > >> > > JSONValue.parseWithException(JSONValue.toJSONString(conf));
> > >> > >
> > >> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> > >> > > System.out.println(e.getValue().getClass().getSimpleName());
> > >> > > }
> > >> > >
> > >> > > This will print "Integer". Using "10000000000000000L" as a value will
> > >> > > result in "Long".
> > >> > >
> > >> > > The main issue derives from the fact, that we serialize the config map
> > >> > > into a JSON String and parse the result.
> > >> > > Based on the String representation, "100000000L" will still fit into an
> > >> > > Integer, so the parser will not use Long ;-)
> > >> > >
> > >> > > If the number is big enough, the parser will switch to a Long.
> > >> > > The old library (> 10 years old now):
> > >> > >
> > >> > > deserTopoConf = (Map<String, Object>)
> > >> > > org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
> > >> > > toJSONString(conf));
> > >> > >
> > >> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> > >> > > System.out.println(e.getValue().getClass().getSimpleName());
> > >> > > }
> > >> > >
> > >> > > This will print "Long".
> > >> > >
> > >> > > The difference between these libraries is, that the super old
> > >> > > org.json.simple just always returns Long as the type.
> > >> > >
> > >> > > If can eventually tune the new JSON parser to something like:
> > >> > >
> > >> > > deserTopoConf = (Map<String, Object>) new
> > >> > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
> > >> > > ing(conf), JSONValue.defaultReader.DEFAULT);
> > >> > >
> > >> > > This will result in "Long" but might have other side-effects.
> > >> > >
> > >> > > From my point of view, the newer JSON library works as expected
> > >> > > (although it might break backward compatibility).
> > >> > > Unchecked casts are always evil, imho. In your specific case, it might
> > >> > > be more save to cast to "Number" and than invoke "longValue()" instead.
> > >> > >
> > >> > > Nevertheless, we might being able to restore the old behaviour by
> > >> > > tuning the JSONParser configuration.
> > >> > >
> > >> > > Thoughts?
> > >> > >
> > >> > > Gruß
> > >> > > Richard
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
> > >> > > Vermeerbergen:
> > >> > > > On a side note, the Long value that "MyKOBolt" is set by a
> > >> > > > DefaultTopologyConfig to a value with is a java.lang.Long instance...
> > >> > > > hence it's surprising to see it internally converted into a
> > >> > > > java.lang.Integer in the Map passed to prepare() method...
> > >> > > >
> > >> > > >
> > >> > > > Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
> > >> > > > <av...@gmail.com> a écrit :
> > >> > > > >
> > >> > > > > Hello,
> > >> > > > >
> > >> > > > > I found a breaking change using Apache Storm 2.6.0 snapshot build
> > >> > > > > dated 20231103 and without changing my topologies' code (keeping
> > >> > > > > the
> > >> > > > > same big jars).
> > >> > > > >
> > >> > > > > This seems to be tied to a datatype change in Map argument of
> > >> > > > > prepare() method of Bolt.
> > >> > > > >
> > >> > > > > For example, this Bolt is KO at runtime:
> > >> > > > >
> > >> > > > > public class MyKOBolt extends BaseRichBolt {
> > >> > > > >
> > >> > > > > @Override
> > >> > > > > public void prepare(Map stormConf, TopologyContext context,
> > >> > > > > OutputCollector collector) {
> > >> > > > > cacheMaxSize = (Long)
> > >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > >> > > > > cacheTtlMin = (Long)
> > >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > >> > > > >
> > >> > > > > as it throws this error:
> > >> > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > >> > > > > Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
> > >> > > > > __system:[-1]
> > >> > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > >> > > > > Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop
> > >> > > > > died!
> > >> > > > > java.lang.ClassCastException: java.lang.Integer incompatible with
> > >> > > > > java.lang.Long
> > >> > > > > at com.acme.storm.alerting.MyKOBolt
> > >> > > > > .prepare(EvaluationBolt.java:71)
> > >> > > > > ~[stormjar.jar:?]
> > >> > > > > at
> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:
> > >> > > > > 128)
> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > >> > > > > at
> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > >> > > > > 138)
> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > >> > > > > at
> > >> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > >> > > > > 54)
> > >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > >> > > > > at org.apache.storm.utils.Utils$1.run(Utils.java:393)
> > >> > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > >> > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> > >> > > > >
> > >> > > > > whereas this bolt is fine with Storm 2.6.0:
> > >> > > > >
> > >> > > > > public class MyOKBolt extends BaseBasicBolt {
> > >> > > > >
> > >> > > > > @Override
> > >> > > > > public void prepare(final Map stormConf, final TopologyContext
> > >> > > > > context) {
> > >> > > > > super.prepare(stormConf, context);
> > >> > > > >
> > >> > > > > this.redisPort =
> > >> > > > > Integer.parseInt((String)
> > >> > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > >> > > > >
> > >> > > > > That said, I can modify the code of "MyKOBolt" to use same
> > >> > > > > Integer.parseInt or Integer.parseLong trick, but it's the first
> > >> > > > > time
> > >> > > > > in my long history of upgrades that I have seen such runtime
> > >> > > > > incompatibility.
> > >> > > > >
> > >> > > > > Maybe "MyKOBolt" was badly written since the beginning and I have
> > >> > > > > just
> > >> > > > > hit the punishment for it: is there a documentation which clarifies
> > >> > > > > datatypes of the Map argument of prepare() method?
> > >> > > > >
> > >> > > > > Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
> > >> > > > > "MyOKBolt" derives from BaseBasicBolt => could this have any impact
> > >> > > > > on
> > >> > > > > this finding ?
> > >> > > > >
> > >> > > > > Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK
> > >> > > > > 17.0.8.1.
> > >> > > > >
> > >> > > > > Kind regards,
> > >> > > > > Alexandre
> > >> > > > >
> > >> > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > >> > > > > <li...@gmail.com> a écrit :
> > >> > > > > >
> > >> > > > > > Thanks Richard.
> > >> > > > > >
> > >> > > > > > Tried the latest snapshot with StormCrawler both in local and
> > >> > > > > > deployed mode
> > >> > > > > > and did not find any issues.
> > >> > > > > > Will try it on a topology generating WARC files next week to
> > >> > > > > > check that the
> > >> > > > > > dependency changes on Hadoop have not broken anything.
> > >> > > > > >
> > >> > > > > > Have a good week end
> > >> > > > > >
> > >> > > > > > Julien
> > >> > > > > >
> > >> > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <rz...@apache.org>
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Short update. Sorted out an issue with the tar.gz/zip with
> > >> > > > > > > Julien today
> > >> > > > > > > and re-uploaded them to the nightlies area.
> > >> > > > > > >
> > >> > > > > > > This new bundle works as expected in my deployment but happy to
> > >> > > > > > > receive
> > >> > > > > > > additional feedback before getting up a first release candidate
> > >> > > > > > > ;-)
> > >> > > > > > >
> > >> > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard
> > >> > > > > > > Zowalla:
> > >> > > > > > > > Hi Alexandre,
> > >> > > > > > > >
> > >> > > > > > > > we are not in a hurry here :) - take as much as time you
> > >> > > > > > > > need.
> > >> > > > > > > >
> > >> > > > > > > > Gruß
> > >> > > > > > > > Richard
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > >> > > > > > > > Vermeerbergen:
> > >> > > > > > > > > Hello Richard,
> > >> > > > > > > > >
> > >> > > > > > > > > Okay, I'm more than happy to do that with my pre-production
> > >> > > > > > > > > cluster
> > >> > > > > > > > > (~10 topologies) using the binary artifacts.
> > >> > > > > > > > > Would it be OK if I can use up to end of this week so that
> > >> > > > > > > > > I'll be
> > >> > > > > > > > > able to have enough time to check all potential issues that
> > >> > > > > > > > > this
> > >> > > > > > > > > upgrade bring?
> > >> > > > > > > > >
> > >> > > > > > > > > Thanks,
> > >> > > > > > > > > Alexandre
> > >> > > > > > > > >
> > >> > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla
> > >> > > > > > > > > <rz...@apache.org> a
> > >> > > > > > > > > écrit :
> > >> > > > > > > > > >
> > >> > > > > > > > > > Hi all,
> > >> > > > > > > > > >
> > >> > > > > > > > > > before starting a release of 2.6.0-SNAPSHOT, I would like
> > >> > > > > > > > > > to
> > >> > > > > > > > > > receive
> > >> > > > > > > > > > some feedback on the current SNAPSHOT build.
> > >> > > > > > > > > >
> > >> > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > >> > > > > > > > > >
> > >> > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > >> > > > > > > > > >
> > >> > > > > > > > > > It is available via the ASF Snapshot repository:
> > >> > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > >> > > > > > > > > >
> > >> > > > > > > > > > You can consume by adding
> > >> > > > > > > > > >
> > >> > > > > > > > > > <repository>
> > >> > > > > > > > > > <id>apache.snapshots</id>
> > >> > > > > > > > > > <name>Apache Snapshot Repository</name>
> > >> > > > > > > > > > <url>https://repository.apache.org/snapshots</url>
> > >> > > > > > > > > > <releases>
> > >> > > > > > > > > > <enabled>false</enabled>
> > >> > > > > > > > > > </releases>
> > >> > > > > > > > > > </repository>
> > >> > > > > > > > > >
> > >> > > > > > > > > > to your project pom. As we do not deploy SNAPSHOTS
> > >> > > > > > > > > > automatically,
> > >> > > > > > > > > > it
> > >> > > > > > > > > > should be easy to just consume the latest SNAPSHOT.
> > >> > > > > > > > > >
> > >> > > > > > > > > > The packaged binaries are available at
> > >> > > > > > > > > > nightlies.apache.org:
> > >> > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > >> > > > > > > > > >
> > >> > > > > > > > > > If you have some minutes left: Please test and report any
> > >> > > > > > > > > > issues
> > >> > > > > > > > > > with
> > >> > > > > > > > > > this binaries, so we can fix before attempting to
> > >> > > > > > > > > > release.
> > >> > > > > > > > > >
> > >> > > > > > > > > > The most significant changes are in the are of
> > >> > > > > > > > > > hadoop/hbase/hdfs
> > >> > > > > > > > > > as
> > >> > > > > > > > > > we
> > >> > > > > > > > > > upgraded from 2.x to 3.x - our own test coverage within
> > >> > > > > > > > > > the build
> > >> > > > > > > > > > looks
> > >> > > > > > > > > > good but would be nice to get some real world use-case
> > >> > > > > > > > > > feedback
> > >> > > > > > > > > > on
> > >> > > > > > > > > > this. In addition, we had quite a lof of 3rd party
> > >> > > > > > > > > > dependency
> > >> > > > > > > > > > upgrades.
> > >> > > > > > > > > >
> > >> > > > > > > > > > In addition, it contains the pruning of external modules
> > >> > > > > > > > > > as
> > >> > > > > > > > > > listed
> > >> > > > > > > > > > in
> > >> > > > > > > > > >
> > >> > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > >> > > > > > > > > >
> > >> > > > > > > > > > A summaryin Jira is here:
> > >> > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > >> > > > > > > > > >
> > >> > > > > > > > > > Gruß
> > >> > > > > > > > > > Richard
> > >> > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > > > --
> > >> > > > > >
> > >> > > > > > *Open Source Solutions for Text Engineering*
> > >> > > > > >
> > >> > > > > > http://www.digitalpebble.com
> > >> > > > > > http://digitalpebble.blogspot.com/
> > >> > > > > > #digitalpebble <http://twitter.com/digitalpebble>
> > >> > >
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Can you keep using my
'STORM-3958-Capacity-to-set-Storm-UI's-title-in-conf/storm.yaml'
branch to push this fix, or should I create a new one?
Le sam. 4 nov. 2023 à 08:14, Richard Zowalla <ri...@zowalla.com> a écrit :
>
> Just push a fix for it (referencing the original issue) ;-) - we hadn't a release in between, so not a big deal.
>
> Am 4. November 2023 08:01:54 MEZ schrieb Alexandre Vermeerbergen <av...@gmail.com>:
> >Temporary workaround for the Storm UI title's regression: run this
> >command on the title-template.html file (in
> >storm-stable/public/templates directory) after having extracted the
> >binary archive:
> >
> >sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
> >
> >Again sorry for having introduced this regression.
> >
> >Alexandre
> >
> >Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
> ><av...@gmail.com> a écrit :
> >>
> >> Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
> >> 1. I have modified my KO bolts using this pattern in their prepare() method:
> >>
> >> long cacheMaxSize =
> >> Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
> >>
> >> instead of:
> >>
> >> long cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> >>
> >> 2. I have found the cause of the regression in loss of 'Storm UI'
> >> title at the top left-hand side of all Storm UI's pages.
> >> This is a mistake of mine in this PR:
> >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> >> More precisely, at line 19 of this file:
> >> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> >> we must have:
> >> <h1><a href="/">{{title}}</a></h1>
> >>
> >> instead of:
> >>
> >> <h1><a href="/">{{ui.title}}</a></h1>
> >>
> >> I think I missed this during my tests because of browser's cache
> >> (bad excuse, I know...): probably I tested by-patching my deployed
> >> Storm cluster, and I finally forgot to checkout this latest change of
> >> my delivery.
> >>
> >> I had to use "private navigation" to make sure this time it fixes this issue
> >>
> >> What would be the quickest way for this very simple fix to be delivered ?
> >>
> >> Thanks,
> >> Alexandre
> >>
> >> Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> >> <av...@gmail.com> a écrit :
> >> >
> >> > Thanks Richard for the analysis.
> >> >
> >> > My thought on this are very simple: let me fix my own old code, after
> >> > all this was impacting only one of my topologies, the other one were
> >> > using more careful Integer.parseInt or Integer.parseLong to avoid
> >> > making assumption on the way the values in the config Map were
> >> > serialized.
> >> >
> >> > My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
> >> > UI" title at the top left hand side of each page is missing. This may
> >> > be the result of one of my own PR, I'll have to check why...
> >> >
> >> > Alex
> >> >
> >> > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla <rz...@apache.org> a écrit :
> >> > >
> >> > > Hi,
> >> > >
> >> > > thanks for the feedback.
> >> > >
> >> > > The main difference to 2.5.0 is, that we switched the JSON
> >> > > implementation in Utils, which relied on a
> >> > > super old JSON library. What is happening here is, that the new JSON
> >> > > library (net.minidev) behaves differently.
> >> > >
> >> > > We can see it by writing some simple code without Storm (The related
> >> > > code line with the parsing is taken from Storm Utils)
> >> > >
> >> > >
> >> > > Map<String, Long> conf = new HashMap<>();
> >> > > conf.put("test", 100000000L);
> >> > >
> >> > > Map<String, Object> deserTopoConf = (Map<String, Object>)
> >> > > JSONValue.parseWithException(JSONValue.toJSONString(conf));
> >> > >
> >> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> >> > > System.out.println(e.getValue().getClass().getSimpleName());
> >> > > }
> >> > >
> >> > > This will print "Integer". Using "10000000000000000L" as a value will
> >> > > result in "Long".
> >> > >
> >> > > The main issue derives from the fact, that we serialize the config map
> >> > > into a JSON String and parse the result.
> >> > > Based on the String representation, "100000000L" will still fit into an
> >> > > Integer, so the parser will not use Long ;-)
> >> > >
> >> > > If the number is big enough, the parser will switch to a Long.
> >> > > The old library (> 10 years old now):
> >> > >
> >> > > deserTopoConf = (Map<String, Object>)
> >> > > org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
> >> > > toJSONString(conf));
> >> > >
> >> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> >> > > System.out.println(e.getValue().getClass().getSimpleName());
> >> > > }
> >> > >
> >> > > This will print "Long".
> >> > >
> >> > > The difference between these libraries is, that the super old
> >> > > org.json.simple just always returns Long as the type.
> >> > >
> >> > > If can eventually tune the new JSON parser to something like:
> >> > >
> >> > > deserTopoConf = (Map<String, Object>) new
> >> > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
> >> > > ing(conf), JSONValue.defaultReader.DEFAULT);
> >> > >
> >> > > This will result in "Long" but might have other side-effects.
> >> > >
> >> > > From my point of view, the newer JSON library works as expected
> >> > > (although it might break backward compatibility).
> >> > > Unchecked casts are always evil, imho. In your specific case, it might
> >> > > be more save to cast to "Number" and than invoke "longValue()" instead.
> >> > >
> >> > > Nevertheless, we might being able to restore the old behaviour by
> >> > > tuning the JSONParser configuration.
> >> > >
> >> > > Thoughts?
> >> > >
> >> > > Gruß
> >> > > Richard
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
> >> > > Vermeerbergen:
> >> > > > On a side note, the Long value that "MyKOBolt" is set by a
> >> > > > DefaultTopologyConfig to a value with is a java.lang.Long instance...
> >> > > > hence it's surprising to see it internally converted into a
> >> > > > java.lang.Integer in the Map passed to prepare() method...
> >> > > >
> >> > > >
> >> > > > Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
> >> > > > <av...@gmail.com> a écrit :
> >> > > > >
> >> > > > > Hello,
> >> > > > >
> >> > > > > I found a breaking change using Apache Storm 2.6.0 snapshot build
> >> > > > > dated 20231103 and without changing my topologies' code (keeping
> >> > > > > the
> >> > > > > same big jars).
> >> > > > >
> >> > > > > This seems to be tied to a datatype change in Map argument of
> >> > > > > prepare() method of Bolt.
> >> > > > >
> >> > > > > For example, this Bolt is KO at runtime:
> >> > > > >
> >> > > > > public class MyKOBolt extends BaseRichBolt {
> >> > > > >
> >> > > > > @Override
> >> > > > > public void prepare(Map stormConf, TopologyContext context,
> >> > > > > OutputCollector collector) {
> >> > > > > cacheMaxSize = (Long)
> >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> >> > > > > cacheTtlMin = (Long)
> >> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> >> > > > >
> >> > > > > as it throws this error:
> >> > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> >> > > > > Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
> >> > > > > __system:[-1]
> >> > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> >> > > > > Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop
> >> > > > > died!
> >> > > > > java.lang.ClassCastException: java.lang.Integer incompatible with
> >> > > > > java.lang.Long
> >> > > > > at com.acme.storm.alerting.MyKOBolt
> >> > > > > .prepare(EvaluationBolt.java:71)
> >> > > > > ~[stormjar.jar:?]
> >> > > > > at
> >> > > > > org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:
> >> > > > > 128)
> >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> >> > > > > at
> >> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> >> > > > > 138)
> >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> >> > > > > at
> >> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> >> > > > > 54)
> >> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> >> > > > > at org.apache.storm.utils.Utils$1.run(Utils.java:393)
> >> > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> >> > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> >> > > > >
> >> > > > > whereas this bolt is fine with Storm 2.6.0:
> >> > > > >
> >> > > > > public class MyOKBolt extends BaseBasicBolt {
> >> > > > >
> >> > > > > @Override
> >> > > > > public void prepare(final Map stormConf, final TopologyContext
> >> > > > > context) {
> >> > > > > super.prepare(stormConf, context);
> >> > > > >
> >> > > > > this.redisPort =
> >> > > > > Integer.parseInt((String)
> >> > > > > stormConf.get(ConfKeys.REDIS_PORT));
> >> > > > >
> >> > > > > That said, I can modify the code of "MyKOBolt" to use same
> >> > > > > Integer.parseInt or Integer.parseLong trick, but it's the first
> >> > > > > time
> >> > > > > in my long history of upgrades that I have seen such runtime
> >> > > > > incompatibility.
> >> > > > >
> >> > > > > Maybe "MyKOBolt" was badly written since the beginning and I have
> >> > > > > just
> >> > > > > hit the punishment for it: is there a documentation which clarifies
> >> > > > > datatypes of the Map argument of prepare() method?
> >> > > > >
> >> > > > > Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
> >> > > > > "MyOKBolt" derives from BaseBasicBolt => could this have any impact
> >> > > > > on
> >> > > > > this finding ?
> >> > > > >
> >> > > > > Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK
> >> > > > > 17.0.8.1.
> >> > > > >
> >> > > > > Kind regards,
> >> > > > > Alexandre
> >> > > > >
> >> > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> >> > > > > <li...@gmail.com> a écrit :
> >> > > > > >
> >> > > > > > Thanks Richard.
> >> > > > > >
> >> > > > > > Tried the latest snapshot with StormCrawler both in local and
> >> > > > > > deployed mode
> >> > > > > > and did not find any issues.
> >> > > > > > Will try it on a topology generating WARC files next week to
> >> > > > > > check that the
> >> > > > > > dependency changes on Hadoop have not broken anything.
> >> > > > > >
> >> > > > > > Have a good week end
> >> > > > > >
> >> > > > > > Julien
> >> > > > > >
> >> > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <rz...@apache.org>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Short update. Sorted out an issue with the tar.gz/zip with
> >> > > > > > > Julien today
> >> > > > > > > and re-uploaded them to the nightlies area.
> >> > > > > > >
> >> > > > > > > This new bundle works as expected in my deployment but happy to
> >> > > > > > > receive
> >> > > > > > > additional feedback before getting up a first release candidate
> >> > > > > > > ;-)
> >> > > > > > >
> >> > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard
> >> > > > > > > Zowalla:
> >> > > > > > > > Hi Alexandre,
> >> > > > > > > >
> >> > > > > > > > we are not in a hurry here :) - take as much as time you
> >> > > > > > > > need.
> >> > > > > > > >
> >> > > > > > > > Gruß
> >> > > > > > > > Richard
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> >> > > > > > > > Vermeerbergen:
> >> > > > > > > > > Hello Richard,
> >> > > > > > > > >
> >> > > > > > > > > Okay, I'm more than happy to do that with my pre-production
> >> > > > > > > > > cluster
> >> > > > > > > > > (~10 topologies) using the binary artifacts.
> >> > > > > > > > > Would it be OK if I can use up to end of this week so that
> >> > > > > > > > > I'll be
> >> > > > > > > > > able to have enough time to check all potential issues that
> >> > > > > > > > > this
> >> > > > > > > > > upgrade bring?
> >> > > > > > > > >
> >> > > > > > > > > Thanks,
> >> > > > > > > > > Alexandre
> >> > > > > > > > >
> >> > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla
> >> > > > > > > > > <rz...@apache.org> a
> >> > > > > > > > > écrit :
> >> > > > > > > > > >
> >> > > > > > > > > > Hi all,
> >> > > > > > > > > >
> >> > > > > > > > > > before starting a release of 2.6.0-SNAPSHOT, I would like
> >> > > > > > > > > > to
> >> > > > > > > > > > receive
> >> > > > > > > > > > some feedback on the current SNAPSHOT build.
> >> > > > > > > > > >
> >> > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> >> > > > > > > > > >
> >> > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> >> > > > > > > > > >
> >> > > > > > > > > > It is available via the ASF Snapshot repository:
> >> > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> >> > > > > > > > > >
> >> > > > > > > > > > You can consume by adding
> >> > > > > > > > > >
> >> > > > > > > > > > <repository>
> >> > > > > > > > > > <id>apache.snapshots</id>
> >> > > > > > > > > > <name>Apache Snapshot Repository</name>
> >> > > > > > > > > > <url>https://repository.apache.org/snapshots</url>
> >> > > > > > > > > > <releases>
> >> > > > > > > > > > <enabled>false</enabled>
> >> > > > > > > > > > </releases>
> >> > > > > > > > > > </repository>
> >> > > > > > > > > >
> >> > > > > > > > > > to your project pom. As we do not deploy SNAPSHOTS
> >> > > > > > > > > > automatically,
> >> > > > > > > > > > it
> >> > > > > > > > > > should be easy to just consume the latest SNAPSHOT.
> >> > > > > > > > > >
> >> > > > > > > > > > The packaged binaries are available at
> >> > > > > > > > > > nightlies.apache.org:
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> >> > > > > > > > > >
> >> > > > > > > > > > If you have some minutes left: Please test and report any
> >> > > > > > > > > > issues
> >> > > > > > > > > > with
> >> > > > > > > > > > this binaries, so we can fix before attempting to
> >> > > > > > > > > > release.
> >> > > > > > > > > >
> >> > > > > > > > > > The most significant changes are in the are of
> >> > > > > > > > > > hadoop/hbase/hdfs
> >> > > > > > > > > > as
> >> > > > > > > > > > we
> >> > > > > > > > > > upgraded from 2.x to 3.x - our own test coverage within
> >> > > > > > > > > > the build
> >> > > > > > > > > > looks
> >> > > > > > > > > > good but would be nice to get some real world use-case
> >> > > > > > > > > > feedback
> >> > > > > > > > > > on
> >> > > > > > > > > > this. In addition, we had quite a lof of 3rd party
> >> > > > > > > > > > dependency
> >> > > > > > > > > > upgrades.
> >> > > > > > > > > >
> >> > > > > > > > > > In addition, it contains the pruning of external modules
> >> > > > > > > > > > as
> >> > > > > > > > > > listed
> >> > > > > > > > > > in
> >> > > > > > > > > >
> >> > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> >> > > > > > > > > >
> >> > > > > > > > > > A summaryin Jira is here:
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> >> > > > > > > > > >
> >> > > > > > > > > > Gruß
> >> > > > > > > > > > Richard
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > > --
> >> > > > > >
> >> > > > > > *Open Source Solutions for Text Engineering*
> >> > > > > >
> >> > > > > > http://www.digitalpebble.com
> >> > > > > > http://digitalpebble.blogspot.com/
> >> > > > > > #digitalpebble <http://twitter.com/digitalpebble>
> >> > >
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Richard Zowalla <ri...@zowalla.com>.
Just push a fix for it (referencing the original issue) ;-) - we hadn't a release in between, so not a big deal.
Am 4. November 2023 08:01:54 MEZ schrieb Alexandre Vermeerbergen <av...@gmail.com>:
>Temporary workaround for the Storm UI title's regression: run this
>command on the title-template.html file (in
>storm-stable/public/templates directory) after having extracted the
>binary archive:
>
>sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
>
>Again sorry for having introduced this regression.
>
>Alexandre
>
>Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
><av...@gmail.com> a écrit :
>>
>> Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
>> 1. I have modified my KO bolts using this pattern in their prepare() method:
>>
>> long cacheMaxSize =
>> Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
>>
>> instead of:
>>
>> long cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
>>
>> 2. I have found the cause of the regression in loss of 'Storm UI'
>> title at the top left-hand side of all Storm UI's pages.
>> This is a mistake of mine in this PR:
>> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
>> More precisely, at line 19 of this file:
>> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
>> we must have:
>> <h1><a href="/">{{title}}</a></h1>
>>
>> instead of:
>>
>> <h1><a href="/">{{ui.title}}</a></h1>
>>
>> I think I missed this during my tests because of browser's cache
>> (bad excuse, I know...): probably I tested by-patching my deployed
>> Storm cluster, and I finally forgot to checkout this latest change of
>> my delivery.
>>
>> I had to use "private navigation" to make sure this time it fixes this issue
>>
>> What would be the quickest way for this very simple fix to be delivered ?
>>
>> Thanks,
>> Alexandre
>>
>> Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
>> <av...@gmail.com> a écrit :
>> >
>> > Thanks Richard for the analysis.
>> >
>> > My thought on this are very simple: let me fix my own old code, after
>> > all this was impacting only one of my topologies, the other one were
>> > using more careful Integer.parseInt or Integer.parseLong to avoid
>> > making assumption on the way the values in the config Map were
>> > serialized.
>> >
>> > My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
>> > UI" title at the top left hand side of each page is missing. This may
>> > be the result of one of my own PR, I'll have to check why...
>> >
>> > Alex
>> >
>> > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla <rz...@apache.org> a écrit :
>> > >
>> > > Hi,
>> > >
>> > > thanks for the feedback.
>> > >
>> > > The main difference to 2.5.0 is, that we switched the JSON
>> > > implementation in Utils, which relied on a
>> > > super old JSON library. What is happening here is, that the new JSON
>> > > library (net.minidev) behaves differently.
>> > >
>> > > We can see it by writing some simple code without Storm (The related
>> > > code line with the parsing is taken from Storm Utils)
>> > >
>> > >
>> > > Map<String, Long> conf = new HashMap<>();
>> > > conf.put("test", 100000000L);
>> > >
>> > > Map<String, Object> deserTopoConf = (Map<String, Object>)
>> > > JSONValue.parseWithException(JSONValue.toJSONString(conf));
>> > >
>> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
>> > > System.out.println(e.getValue().getClass().getSimpleName());
>> > > }
>> > >
>> > > This will print "Integer". Using "10000000000000000L" as a value will
>> > > result in "Long".
>> > >
>> > > The main issue derives from the fact, that we serialize the config map
>> > > into a JSON String and parse the result.
>> > > Based on the String representation, "100000000L" will still fit into an
>> > > Integer, so the parser will not use Long ;-)
>> > >
>> > > If the number is big enough, the parser will switch to a Long.
>> > > The old library (> 10 years old now):
>> > >
>> > > deserTopoConf = (Map<String, Object>)
>> > > org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
>> > > toJSONString(conf));
>> > >
>> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
>> > > System.out.println(e.getValue().getClass().getSimpleName());
>> > > }
>> > >
>> > > This will print "Long".
>> > >
>> > > The difference between these libraries is, that the super old
>> > > org.json.simple just always returns Long as the type.
>> > >
>> > > If can eventually tune the new JSON parser to something like:
>> > >
>> > > deserTopoConf = (Map<String, Object>) new
>> > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
>> > > ing(conf), JSONValue.defaultReader.DEFAULT);
>> > >
>> > > This will result in "Long" but might have other side-effects.
>> > >
>> > > From my point of view, the newer JSON library works as expected
>> > > (although it might break backward compatibility).
>> > > Unchecked casts are always evil, imho. In your specific case, it might
>> > > be more save to cast to "Number" and than invoke "longValue()" instead.
>> > >
>> > > Nevertheless, we might being able to restore the old behaviour by
>> > > tuning the JSONParser configuration.
>> > >
>> > > Thoughts?
>> > >
>> > > Gruß
>> > > Richard
>> > >
>> > >
>> > >
>> > >
>> > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
>> > > Vermeerbergen:
>> > > > On a side note, the Long value that "MyKOBolt" is set by a
>> > > > DefaultTopologyConfig to a value with is a java.lang.Long instance...
>> > > > hence it's surprising to see it internally converted into a
>> > > > java.lang.Integer in the Map passed to prepare() method...
>> > > >
>> > > >
>> > > > Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
>> > > > <av...@gmail.com> a écrit :
>> > > > >
>> > > > > Hello,
>> > > > >
>> > > > > I found a breaking change using Apache Storm 2.6.0 snapshot build
>> > > > > dated 20231103 and without changing my topologies' code (keeping
>> > > > > the
>> > > > > same big jars).
>> > > > >
>> > > > > This seems to be tied to a datatype change in Map argument of
>> > > > > prepare() method of Bolt.
>> > > > >
>> > > > > For example, this Bolt is KO at runtime:
>> > > > >
>> > > > > public class MyKOBolt extends BaseRichBolt {
>> > > > >
>> > > > > @Override
>> > > > > public void prepare(Map stormConf, TopologyContext context,
>> > > > > OutputCollector collector) {
>> > > > > cacheMaxSize = (Long)
>> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
>> > > > > cacheTtlMin = (Long)
>> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
>> > > > >
>> > > > > as it throws this error:
>> > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
>> > > > > Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
>> > > > > __system:[-1]
>> > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
>> > > > > Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop
>> > > > > died!
>> > > > > java.lang.ClassCastException: java.lang.Integer incompatible with
>> > > > > java.lang.Long
>> > > > > at com.acme.storm.alerting.MyKOBolt
>> > > > > .prepare(EvaluationBolt.java:71)
>> > > > > ~[stormjar.jar:?]
>> > > > > at
>> > > > > org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:
>> > > > > 128)
>> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
>> > > > > at
>> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
>> > > > > 138)
>> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
>> > > > > at
>> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
>> > > > > 54)
>> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
>> > > > > at org.apache.storm.utils.Utils$1.run(Utils.java:393)
>> > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
>> > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
>> > > > >
>> > > > > whereas this bolt is fine with Storm 2.6.0:
>> > > > >
>> > > > > public class MyOKBolt extends BaseBasicBolt {
>> > > > >
>> > > > > @Override
>> > > > > public void prepare(final Map stormConf, final TopologyContext
>> > > > > context) {
>> > > > > super.prepare(stormConf, context);
>> > > > >
>> > > > > this.redisPort =
>> > > > > Integer.parseInt((String)
>> > > > > stormConf.get(ConfKeys.REDIS_PORT));
>> > > > >
>> > > > > That said, I can modify the code of "MyKOBolt" to use same
>> > > > > Integer.parseInt or Integer.parseLong trick, but it's the first
>> > > > > time
>> > > > > in my long history of upgrades that I have seen such runtime
>> > > > > incompatibility.
>> > > > >
>> > > > > Maybe "MyKOBolt" was badly written since the beginning and I have
>> > > > > just
>> > > > > hit the punishment for it: is there a documentation which clarifies
>> > > > > datatypes of the Map argument of prepare() method?
>> > > > >
>> > > > > Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
>> > > > > "MyOKBolt" derives from BaseBasicBolt => could this have any impact
>> > > > > on
>> > > > > this finding ?
>> > > > >
>> > > > > Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK
>> > > > > 17.0.8.1.
>> > > > >
>> > > > > Kind regards,
>> > > > > Alexandre
>> > > > >
>> > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
>> > > > > <li...@gmail.com> a écrit :
>> > > > > >
>> > > > > > Thanks Richard.
>> > > > > >
>> > > > > > Tried the latest snapshot with StormCrawler both in local and
>> > > > > > deployed mode
>> > > > > > and did not find any issues.
>> > > > > > Will try it on a topology generating WARC files next week to
>> > > > > > check that the
>> > > > > > dependency changes on Hadoop have not broken anything.
>> > > > > >
>> > > > > > Have a good week end
>> > > > > >
>> > > > > > Julien
>> > > > > >
>> > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <rz...@apache.org>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Short update. Sorted out an issue with the tar.gz/zip with
>> > > > > > > Julien today
>> > > > > > > and re-uploaded them to the nightlies area.
>> > > > > > >
>> > > > > > > This new bundle works as expected in my deployment but happy to
>> > > > > > > receive
>> > > > > > > additional feedback before getting up a first release candidate
>> > > > > > > ;-)
>> > > > > > >
>> > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard
>> > > > > > > Zowalla:
>> > > > > > > > Hi Alexandre,
>> > > > > > > >
>> > > > > > > > we are not in a hurry here :) - take as much as time you
>> > > > > > > > need.
>> > > > > > > >
>> > > > > > > > Gruß
>> > > > > > > > Richard
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
>> > > > > > > > Vermeerbergen:
>> > > > > > > > > Hello Richard,
>> > > > > > > > >
>> > > > > > > > > Okay, I'm more than happy to do that with my pre-production
>> > > > > > > > > cluster
>> > > > > > > > > (~10 topologies) using the binary artifacts.
>> > > > > > > > > Would it be OK if I can use up to end of this week so that
>> > > > > > > > > I'll be
>> > > > > > > > > able to have enough time to check all potential issues that
>> > > > > > > > > this
>> > > > > > > > > upgrade bring?
>> > > > > > > > >
>> > > > > > > > > Thanks,
>> > > > > > > > > Alexandre
>> > > > > > > > >
>> > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla
>> > > > > > > > > <rz...@apache.org> a
>> > > > > > > > > écrit :
>> > > > > > > > > >
>> > > > > > > > > > Hi all,
>> > > > > > > > > >
>> > > > > > > > > > before starting a release of 2.6.0-SNAPSHOT, I would like
>> > > > > > > > > > to
>> > > > > > > > > > receive
>> > > > > > > > > > some feedback on the current SNAPSHOT build.
>> > > > > > > > > >
>> > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
>> > > > > > > > > >
>> > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
>> > > > > > > > > >
>> > > > > > > > > > It is available via the ASF Snapshot repository:
>> > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
>> > > > > > > > > >
>> > > > > > > > > > You can consume by adding
>> > > > > > > > > >
>> > > > > > > > > > <repository>
>> > > > > > > > > > <id>apache.snapshots</id>
>> > > > > > > > > > <name>Apache Snapshot Repository</name>
>> > > > > > > > > > <url>https://repository.apache.org/snapshots</url>
>> > > > > > > > > > <releases>
>> > > > > > > > > > <enabled>false</enabled>
>> > > > > > > > > > </releases>
>> > > > > > > > > > </repository>
>> > > > > > > > > >
>> > > > > > > > > > to your project pom. As we do not deploy SNAPSHOTS
>> > > > > > > > > > automatically,
>> > > > > > > > > > it
>> > > > > > > > > > should be easy to just consume the latest SNAPSHOT.
>> > > > > > > > > >
>> > > > > > > > > > The packaged binaries are available at
>> > > > > > > > > > nightlies.apache.org:
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
>> > > > > > > > > >
>> > > > > > > > > > If you have some minutes left: Please test and report any
>> > > > > > > > > > issues
>> > > > > > > > > > with
>> > > > > > > > > > this binaries, so we can fix before attempting to
>> > > > > > > > > > release.
>> > > > > > > > > >
>> > > > > > > > > > The most significant changes are in the are of
>> > > > > > > > > > hadoop/hbase/hdfs
>> > > > > > > > > > as
>> > > > > > > > > > we
>> > > > > > > > > > upgraded from 2.x to 3.x - our own test coverage within
>> > > > > > > > > > the build
>> > > > > > > > > > looks
>> > > > > > > > > > good but would be nice to get some real world use-case
>> > > > > > > > > > feedback
>> > > > > > > > > > on
>> > > > > > > > > > this. In addition, we had quite a lof of 3rd party
>> > > > > > > > > > dependency
>> > > > > > > > > > upgrades.
>> > > > > > > > > >
>> > > > > > > > > > In addition, it contains the pruning of external modules
>> > > > > > > > > > as
>> > > > > > > > > > listed
>> > > > > > > > > > in
>> > > > > > > > > >
>> > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
>> > > > > > > > > >
>> > > > > > > > > > A summaryin Jira is here:
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
>> > > > > > > > > >
>> > > > > > > > > > Gruß
>> > > > > > > > > > Richard
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > > --
>> > > > > >
>> > > > > > *Open Source Solutions for Text Engineering*
>> > > > > >
>> > > > > > http://www.digitalpebble.com
>> > > > > > http://digitalpebble.blogspot.com/
>> > > > > > #digitalpebble <http://twitter.com/digitalpebble>
>> > >
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Temporary workaround for the Storm UI title's regression: run this
command on the title-template.html file (in
storm-stable/public/templates directory) after having extracted the
binary archive:
sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
Again sorry for having introduced this regression.
Alexandre
Le sam. 4 nov. 2023 à 07:52, Alexandre Vermeerbergen
<av...@gmail.com> a écrit :
>
> Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
> 1. I have modified my KO bolts using this pattern in their prepare() method:
>
> long cacheMaxSize =
> Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
>
> instead of:
>
> long cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
>
> 2. I have found the cause of the regression in loss of 'Storm UI'
> title at the top left-hand side of all Storm UI's pages.
> This is a mistake of mine in this PR:
> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
> More precisely, at line 19 of this file:
> https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
> we must have:
> <h1><a href="/">{{title}}</a></h1>
>
> instead of:
>
> <h1><a href="/">{{ui.title}}</a></h1>
>
> I think I missed this during my tests because of browser's cache
> (bad excuse, I know...): probably I tested by-patching my deployed
> Storm cluster, and I finally forgot to checkout this latest change of
> my delivery.
>
> I had to use "private navigation" to make sure this time it fixes this issue
>
> What would be the quickest way for this very simple fix to be delivered ?
>
> Thanks,
> Alexandre
>
> Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
> <av...@gmail.com> a écrit :
> >
> > Thanks Richard for the analysis.
> >
> > My thought on this are very simple: let me fix my own old code, after
> > all this was impacting only one of my topologies, the other one were
> > using more careful Integer.parseInt or Integer.parseLong to avoid
> > making assumption on the way the values in the config Map were
> > serialized.
> >
> > My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
> > UI" title at the top left hand side of each page is missing. This may
> > be the result of one of my own PR, I'll have to check why...
> >
> > Alex
> >
> > Le ven. 3 nov. 2023 à 20:23, Richard Zowalla <rz...@apache.org> a écrit :
> > >
> > > Hi,
> > >
> > > thanks for the feedback.
> > >
> > > The main difference to 2.5.0 is, that we switched the JSON
> > > implementation in Utils, which relied on a
> > > super old JSON library. What is happening here is, that the new JSON
> > > library (net.minidev) behaves differently.
> > >
> > > We can see it by writing some simple code without Storm (The related
> > > code line with the parsing is taken from Storm Utils)
> > >
> > >
> > > Map<String, Long> conf = new HashMap<>();
> > > conf.put("test", 100000000L);
> > >
> > > Map<String, Object> deserTopoConf = (Map<String, Object>)
> > > JSONValue.parseWithException(JSONValue.toJSONString(conf));
> > >
> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> > > System.out.println(e.getValue().getClass().getSimpleName());
> > > }
> > >
> > > This will print "Integer". Using "10000000000000000L" as a value will
> > > result in "Long".
> > >
> > > The main issue derives from the fact, that we serialize the config map
> > > into a JSON String and parse the result.
> > > Based on the String representation, "100000000L" will still fit into an
> > > Integer, so the parser will not use Long ;-)
> > >
> > > If the number is big enough, the parser will switch to a Long.
> > > The old library (> 10 years old now):
> > >
> > > deserTopoConf = (Map<String, Object>)
> > > org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
> > > toJSONString(conf));
> > >
> > > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> > > System.out.println(e.getValue().getClass().getSimpleName());
> > > }
> > >
> > > This will print "Long".
> > >
> > > The difference between these libraries is, that the super old
> > > org.json.simple just always returns Long as the type.
> > >
> > > If can eventually tune the new JSON parser to something like:
> > >
> > > deserTopoConf = (Map<String, Object>) new
> > > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
> > > ing(conf), JSONValue.defaultReader.DEFAULT);
> > >
> > > This will result in "Long" but might have other side-effects.
> > >
> > > From my point of view, the newer JSON library works as expected
> > > (although it might break backward compatibility).
> > > Unchecked casts are always evil, imho. In your specific case, it might
> > > be more save to cast to "Number" and than invoke "longValue()" instead.
> > >
> > > Nevertheless, we might being able to restore the old behaviour by
> > > tuning the JSONParser configuration.
> > >
> > > Thoughts?
> > >
> > > Gruß
> > > Richard
> > >
> > >
> > >
> > >
> > > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
> > > Vermeerbergen:
> > > > On a side note, the Long value that "MyKOBolt" is set by a
> > > > DefaultTopologyConfig to a value with is a java.lang.Long instance...
> > > > hence it's surprising to see it internally converted into a
> > > > java.lang.Integer in the Map passed to prepare() method...
> > > >
> > > >
> > > > Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
> > > > <av...@gmail.com> a écrit :
> > > > >
> > > > > Hello,
> > > > >
> > > > > I found a breaking change using Apache Storm 2.6.0 snapshot build
> > > > > dated 20231103 and without changing my topologies' code (keeping
> > > > > the
> > > > > same big jars).
> > > > >
> > > > > This seems to be tied to a datatype change in Map argument of
> > > > > prepare() method of Bolt.
> > > > >
> > > > > For example, this Bolt is KO at runtime:
> > > > >
> > > > > public class MyKOBolt extends BaseRichBolt {
> > > > >
> > > > > @Override
> > > > > public void prepare(Map stormConf, TopologyContext context,
> > > > > OutputCollector collector) {
> > > > > cacheMaxSize = (Long)
> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > > cacheTtlMin = (Long)
> > > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > > > >
> > > > > as it throws this error:
> > > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > > > > Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
> > > > > __system:[-1]
> > > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > > > Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop
> > > > > died!
> > > > > java.lang.ClassCastException: java.lang.Integer incompatible with
> > > > > java.lang.Long
> > > > > at com.acme.storm.alerting.MyKOBolt
> > > > > .prepare(EvaluationBolt.java:71)
> > > > > ~[stormjar.jar:?]
> > > > > at
> > > > > org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:
> > > > > 128)
> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > > at
> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > > > > 138)
> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > > at
> > > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > > > > 54)
> > > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > > at org.apache.storm.utils.Utils$1.run(Utils.java:393)
> > > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> > > > >
> > > > > whereas this bolt is fine with Storm 2.6.0:
> > > > >
> > > > > public class MyOKBolt extends BaseBasicBolt {
> > > > >
> > > > > @Override
> > > > > public void prepare(final Map stormConf, final TopologyContext
> > > > > context) {
> > > > > super.prepare(stormConf, context);
> > > > >
> > > > > this.redisPort =
> > > > > Integer.parseInt((String)
> > > > > stormConf.get(ConfKeys.REDIS_PORT));
> > > > >
> > > > > That said, I can modify the code of "MyKOBolt" to use same
> > > > > Integer.parseInt or Integer.parseLong trick, but it's the first
> > > > > time
> > > > > in my long history of upgrades that I have seen such runtime
> > > > > incompatibility.
> > > > >
> > > > > Maybe "MyKOBolt" was badly written since the beginning and I have
> > > > > just
> > > > > hit the punishment for it: is there a documentation which clarifies
> > > > > datatypes of the Map argument of prepare() method?
> > > > >
> > > > > Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
> > > > > "MyOKBolt" derives from BaseBasicBolt => could this have any impact
> > > > > on
> > > > > this finding ?
> > > > >
> > > > > Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK
> > > > > 17.0.8.1.
> > > > >
> > > > > Kind regards,
> > > > > Alexandre
> > > > >
> > > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > > > > <li...@gmail.com> a écrit :
> > > > > >
> > > > > > Thanks Richard.
> > > > > >
> > > > > > Tried the latest snapshot with StormCrawler both in local and
> > > > > > deployed mode
> > > > > > and did not find any issues.
> > > > > > Will try it on a topology generating WARC files next week to
> > > > > > check that the
> > > > > > dependency changes on Hadoop have not broken anything.
> > > > > >
> > > > > > Have a good week end
> > > > > >
> > > > > > Julien
> > > > > >
> > > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <rz...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > Short update. Sorted out an issue with the tar.gz/zip with
> > > > > > > Julien today
> > > > > > > and re-uploaded them to the nightlies area.
> > > > > > >
> > > > > > > This new bundle works as expected in my deployment but happy to
> > > > > > > receive
> > > > > > > additional feedback before getting up a first release candidate
> > > > > > > ;-)
> > > > > > >
> > > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard
> > > > > > > Zowalla:
> > > > > > > > Hi Alexandre,
> > > > > > > >
> > > > > > > > we are not in a hurry here :) - take as much as time you
> > > > > > > > need.
> > > > > > > >
> > > > > > > > Gruß
> > > > > > > > Richard
> > > > > > > >
> > > > > > > >
> > > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > > > > > > > Vermeerbergen:
> > > > > > > > > Hello Richard,
> > > > > > > > >
> > > > > > > > > Okay, I'm more than happy to do that with my pre-production
> > > > > > > > > cluster
> > > > > > > > > (~10 topologies) using the binary artifacts.
> > > > > > > > > Would it be OK if I can use up to end of this week so that
> > > > > > > > > I'll be
> > > > > > > > > able to have enough time to check all potential issues that
> > > > > > > > > this
> > > > > > > > > upgrade bring?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Alexandre
> > > > > > > > >
> > > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla
> > > > > > > > > <rz...@apache.org> a
> > > > > > > > > écrit :
> > > > > > > > > >
> > > > > > > > > > Hi all,
> > > > > > > > > >
> > > > > > > > > > before starting a release of 2.6.0-SNAPSHOT, I would like
> > > > > > > > > > to
> > > > > > > > > > receive
> > > > > > > > > > some feedback on the current SNAPSHOT build.
> > > > > > > > > >
> > > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > > > > > > > > >
> > > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > > >
> > > > > > > > > > It is available via the ASF Snapshot repository:
> > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > > >
> > > > > > > > > > You can consume by adding
> > > > > > > > > >
> > > > > > > > > > <repository>
> > > > > > > > > > <id>apache.snapshots</id>
> > > > > > > > > > <name>Apache Snapshot Repository</name>
> > > > > > > > > > <url>https://repository.apache.org/snapshots</url>
> > > > > > > > > > <releases>
> > > > > > > > > > <enabled>false</enabled>
> > > > > > > > > > </releases>
> > > > > > > > > > </repository>
> > > > > > > > > >
> > > > > > > > > > to your project pom. As we do not deploy SNAPSHOTS
> > > > > > > > > > automatically,
> > > > > > > > > > it
> > > > > > > > > > should be easy to just consume the latest SNAPSHOT.
> > > > > > > > > >
> > > > > > > > > > The packaged binaries are available at
> > > > > > > > > > nightlies.apache.org:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > > >
> > > > > > > > > > If you have some minutes left: Please test and report any
> > > > > > > > > > issues
> > > > > > > > > > with
> > > > > > > > > > this binaries, so we can fix before attempting to
> > > > > > > > > > release.
> > > > > > > > > >
> > > > > > > > > > The most significant changes are in the are of
> > > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > > as
> > > > > > > > > > we
> > > > > > > > > > upgraded from 2.x to 3.x - our own test coverage within
> > > > > > > > > > the build
> > > > > > > > > > looks
> > > > > > > > > > good but would be nice to get some real world use-case
> > > > > > > > > > feedback
> > > > > > > > > > on
> > > > > > > > > > this. In addition, we had quite a lof of 3rd party
> > > > > > > > > > dependency
> > > > > > > > > > upgrades.
> > > > > > > > > >
> > > > > > > > > > In addition, it contains the pruning of external modules
> > > > > > > > > > as
> > > > > > > > > > listed
> > > > > > > > > > in
> > > > > > > > > >
> > > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > > >
> > > > > > > > > > A summaryin Jira is here:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > > >
> > > > > > > > > > Gruß
> > > > > > > > > > Richard
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > *Open Source Solutions for Text Engineering*
> > > > > >
> > > > > > http://www.digitalpebble.com
> > > > > > http://digitalpebble.blogspot.com/
> > > > > > #digitalpebble <http://twitter.com/digitalpebble>
> > >
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
1. I have modified my KO bolts using this pattern in their prepare() method:
long cacheMaxSize =
Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
instead of:
long cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
2. I have found the cause of the regression in loss of 'Storm UI'
title at the top left-hand side of all Storm UI's pages.
This is a mistake of mine in this PR:
https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd
More precisely, at line 19 of this file:
https://github.com/apache/storm/pull/3580/commits/0320ee6f2d367086b973b504a976ae6aff853ccd#diff-6d3a4162a8af826a25a2b8cd6763c08e9bc5165b736b45890b84fa50597d7481
we must have:
<h1><a href="/">{{title}}</a></h1>
instead of:
<h1><a href="/">{{ui.title}}</a></h1>
I think I missed this during my tests because of browser's cache
(bad excuse, I know...): probably I tested by-patching my deployed
Storm cluster, and I finally forgot to checkout this latest change of
my delivery.
I had to use "private navigation" to make sure this time it fixes this issue
What would be the quickest way for this very simple fix to be delivered ?
Thanks,
Alexandre
Le ven. 3 nov. 2023 à 20:57, Alexandre Vermeerbergen
<av...@gmail.com> a écrit :
>
> Thanks Richard for the analysis.
>
> My thought on this are very simple: let me fix my own old code, after
> all this was impacting only one of my topologies, the other one were
> using more careful Integer.parseInt or Integer.parseLong to avoid
> making assumption on the way the values in the config Map were
> serialized.
>
> My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
> UI" title at the top left hand side of each page is missing. This may
> be the result of one of my own PR, I'll have to check why...
>
> Alex
>
> Le ven. 3 nov. 2023 à 20:23, Richard Zowalla <rz...@apache.org> a écrit :
> >
> > Hi,
> >
> > thanks for the feedback.
> >
> > The main difference to 2.5.0 is, that we switched the JSON
> > implementation in Utils, which relied on a
> > super old JSON library. What is happening here is, that the new JSON
> > library (net.minidev) behaves differently.
> >
> > We can see it by writing some simple code without Storm (The related
> > code line with the parsing is taken from Storm Utils)
> >
> >
> > Map<String, Long> conf = new HashMap<>();
> > conf.put("test", 100000000L);
> >
> > Map<String, Object> deserTopoConf = (Map<String, Object>)
> > JSONValue.parseWithException(JSONValue.toJSONString(conf));
> >
> > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> > System.out.println(e.getValue().getClass().getSimpleName());
> > }
> >
> > This will print "Integer". Using "10000000000000000L" as a value will
> > result in "Long".
> >
> > The main issue derives from the fact, that we serialize the config map
> > into a JSON String and parse the result.
> > Based on the String representation, "100000000L" will still fit into an
> > Integer, so the parser will not use Long ;-)
> >
> > If the number is big enough, the parser will switch to a Long.
> > The old library (> 10 years old now):
> >
> > deserTopoConf = (Map<String, Object>)
> > org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
> > toJSONString(conf));
> >
> > for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> > System.out.println(e.getValue().getClass().getSimpleName());
> > }
> >
> > This will print "Long".
> >
> > The difference between these libraries is, that the super old
> > org.json.simple just always returns Long as the type.
> >
> > If can eventually tune the new JSON parser to something like:
> >
> > deserTopoConf = (Map<String, Object>) new
> > JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
> > ing(conf), JSONValue.defaultReader.DEFAULT);
> >
> > This will result in "Long" but might have other side-effects.
> >
> > From my point of view, the newer JSON library works as expected
> > (although it might break backward compatibility).
> > Unchecked casts are always evil, imho. In your specific case, it might
> > be more save to cast to "Number" and than invoke "longValue()" instead.
> >
> > Nevertheless, we might being able to restore the old behaviour by
> > tuning the JSONParser configuration.
> >
> > Thoughts?
> >
> > Gruß
> > Richard
> >
> >
> >
> >
> > Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
> > Vermeerbergen:
> > > On a side note, the Long value that "MyKOBolt" is set by a
> > > DefaultTopologyConfig to a value with is a java.lang.Long instance...
> > > hence it's surprising to see it internally converted into a
> > > java.lang.Integer in the Map passed to prepare() method...
> > >
> > >
> > > Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
> > > <av...@gmail.com> a écrit :
> > > >
> > > > Hello,
> > > >
> > > > I found a breaking change using Apache Storm 2.6.0 snapshot build
> > > > dated 20231103 and without changing my topologies' code (keeping
> > > > the
> > > > same big jars).
> > > >
> > > > This seems to be tied to a datatype change in Map argument of
> > > > prepare() method of Bolt.
> > > >
> > > > For example, this Bolt is KO at runtime:
> > > >
> > > > public class MyKOBolt extends BaseRichBolt {
> > > >
> > > > @Override
> > > > public void prepare(Map stormConf, TopologyContext context,
> > > > OutputCollector collector) {
> > > > cacheMaxSize = (Long)
> > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > > cacheTtlMin = (Long)
> > > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > > >
> > > > as it throws this error:
> > > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > > > Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
> > > > __system:[-1]
> > > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > > Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop
> > > > died!
> > > > java.lang.ClassCastException: java.lang.Integer incompatible with
> > > > java.lang.Long
> > > > at com.acme.storm.alerting.MyKOBolt
> > > > .prepare(EvaluationBolt.java:71)
> > > > ~[stormjar.jar:?]
> > > > at
> > > > org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:
> > > > 128)
> > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > at
> > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > > > 138)
> > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > at
> > > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > > > 54)
> > > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > at org.apache.storm.utils.Utils$1.run(Utils.java:393)
> > > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > > at java.lang.Thread.run(Thread.java:857) [?:?]
> > > >
> > > > whereas this bolt is fine with Storm 2.6.0:
> > > >
> > > > public class MyOKBolt extends BaseBasicBolt {
> > > >
> > > > @Override
> > > > public void prepare(final Map stormConf, final TopologyContext
> > > > context) {
> > > > super.prepare(stormConf, context);
> > > >
> > > > this.redisPort =
> > > > Integer.parseInt((String)
> > > > stormConf.get(ConfKeys.REDIS_PORT));
> > > >
> > > > That said, I can modify the code of "MyKOBolt" to use same
> > > > Integer.parseInt or Integer.parseLong trick, but it's the first
> > > > time
> > > > in my long history of upgrades that I have seen such runtime
> > > > incompatibility.
> > > >
> > > > Maybe "MyKOBolt" was badly written since the beginning and I have
> > > > just
> > > > hit the punishment for it: is there a documentation which clarifies
> > > > datatypes of the Map argument of prepare() method?
> > > >
> > > > Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
> > > > "MyOKBolt" derives from BaseBasicBolt => could this have any impact
> > > > on
> > > > this finding ?
> > > >
> > > > Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK
> > > > 17.0.8.1.
> > > >
> > > > Kind regards,
> > > > Alexandre
> > > >
> > > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > > > <li...@gmail.com> a écrit :
> > > > >
> > > > > Thanks Richard.
> > > > >
> > > > > Tried the latest snapshot with StormCrawler both in local and
> > > > > deployed mode
> > > > > and did not find any issues.
> > > > > Will try it on a topology generating WARC files next week to
> > > > > check that the
> > > > > dependency changes on Hadoop have not broken anything.
> > > > >
> > > > > Have a good week end
> > > > >
> > > > > Julien
> > > > >
> > > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <rz...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Short update. Sorted out an issue with the tar.gz/zip with
> > > > > > Julien today
> > > > > > and re-uploaded them to the nightlies area.
> > > > > >
> > > > > > This new bundle works as expected in my deployment but happy to
> > > > > > receive
> > > > > > additional feedback before getting up a first release candidate
> > > > > > ;-)
> > > > > >
> > > > > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard
> > > > > > Zowalla:
> > > > > > > Hi Alexandre,
> > > > > > >
> > > > > > > we are not in a hurry here :) - take as much as time you
> > > > > > > need.
> > > > > > >
> > > > > > > Gruß
> > > > > > > Richard
> > > > > > >
> > > > > > >
> > > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > > > > > > Vermeerbergen:
> > > > > > > > Hello Richard,
> > > > > > > >
> > > > > > > > Okay, I'm more than happy to do that with my pre-production
> > > > > > > > cluster
> > > > > > > > (~10 topologies) using the binary artifacts.
> > > > > > > > Would it be OK if I can use up to end of this week so that
> > > > > > > > I'll be
> > > > > > > > able to have enough time to check all potential issues that
> > > > > > > > this
> > > > > > > > upgrade bring?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Alexandre
> > > > > > > >
> > > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla
> > > > > > > > <rz...@apache.org> a
> > > > > > > > écrit :
> > > > > > > > >
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > before starting a release of 2.6.0-SNAPSHOT, I would like
> > > > > > > > > to
> > > > > > > > > receive
> > > > > > > > > some feedback on the current SNAPSHOT build.
> > > > > > > > >
> > > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > > > > > > > >
> > > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > > >
> > > > > > > > > It is available via the ASF Snapshot repository:
> > > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > > >
> > > > > > > > > You can consume by adding
> > > > > > > > >
> > > > > > > > > <repository>
> > > > > > > > > <id>apache.snapshots</id>
> > > > > > > > > <name>Apache Snapshot Repository</name>
> > > > > > > > > <url>https://repository.apache.org/snapshots</url>
> > > > > > > > > <releases>
> > > > > > > > > <enabled>false</enabled>
> > > > > > > > > </releases>
> > > > > > > > > </repository>
> > > > > > > > >
> > > > > > > > > to your project pom. As we do not deploy SNAPSHOTS
> > > > > > > > > automatically,
> > > > > > > > > it
> > > > > > > > > should be easy to just consume the latest SNAPSHOT.
> > > > > > > > >
> > > > > > > > > The packaged binaries are available at
> > > > > > > > > nightlies.apache.org:
> > > > > > > > >
> > > > > > > > >
> > > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > > >
> > > > > > > > > If you have some minutes left: Please test and report any
> > > > > > > > > issues
> > > > > > > > > with
> > > > > > > > > this binaries, so we can fix before attempting to
> > > > > > > > > release.
> > > > > > > > >
> > > > > > > > > The most significant changes are in the are of
> > > > > > > > > hadoop/hbase/hdfs
> > > > > > > > > as
> > > > > > > > > we
> > > > > > > > > upgraded from 2.x to 3.x - our own test coverage within
> > > > > > > > > the build
> > > > > > > > > looks
> > > > > > > > > good but would be nice to get some real world use-case
> > > > > > > > > feedback
> > > > > > > > > on
> > > > > > > > > this. In addition, we had quite a lof of 3rd party
> > > > > > > > > dependency
> > > > > > > > > upgrades.
> > > > > > > > >
> > > > > > > > > In addition, it contains the pruning of external modules
> > > > > > > > > as
> > > > > > > > > listed
> > > > > > > > > in
> > > > > > > > >
> > > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > > >
> > > > > > > > > A summaryin Jira is here:
> > > > > > > > >
> > > > > > > > >
> > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > > >
> > > > > > > > > Gruß
> > > > > > > > > Richard
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > >
> > > > > *Open Source Solutions for Text Engineering*
> > > > >
> > > > > http://www.digitalpebble.com
> > > > > http://digitalpebble.blogspot.com/
> > > > > #digitalpebble <http://twitter.com/digitalpebble>
> >
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Thanks Richard for the analysis.
My thought on this are very simple: let me fix my own old code, after
all this was impacting only one of my topologies, the other one were
using more careful Integer.parseInt or Integer.parseLong to avoid
making assumption on the way the values in the config Map were
serialized.
My other concern with 2.6.0 Snapshot is that in Storm UI, the "Storm
UI" title at the top left hand side of each page is missing. This may
be the result of one of my own PR, I'll have to check why...
Alex
Le ven. 3 nov. 2023 à 20:23, Richard Zowalla <rz...@apache.org> a écrit :
>
> Hi,
>
> thanks for the feedback.
>
> The main difference to 2.5.0 is, that we switched the JSON
> implementation in Utils, which relied on a
> super old JSON library. What is happening here is, that the new JSON
> library (net.minidev) behaves differently.
>
> We can see it by writing some simple code without Storm (The related
> code line with the parsing is taken from Storm Utils)
>
>
> Map<String, Long> conf = new HashMap<>();
> conf.put("test", 100000000L);
>
> Map<String, Object> deserTopoConf = (Map<String, Object>)
> JSONValue.parseWithException(JSONValue.toJSONString(conf));
>
> for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> System.out.println(e.getValue().getClass().getSimpleName());
> }
>
> This will print "Integer". Using "10000000000000000L" as a value will
> result in "Long".
>
> The main issue derives from the fact, that we serialize the config map
> into a JSON String and parse the result.
> Based on the String representation, "100000000L" will still fit into an
> Integer, so the parser will not use Long ;-)
>
> If the number is big enough, the parser will switch to a Long.
> The old library (> 10 years old now):
>
> deserTopoConf = (Map<String, Object>)
> org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
> toJSONString(conf));
>
> for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
> System.out.println(e.getValue().getClass().getSimpleName());
> }
>
> This will print "Long".
>
> The difference between these libraries is, that the super old
> org.json.simple just always returns Long as the type.
>
> If can eventually tune the new JSON parser to something like:
>
> deserTopoConf = (Map<String, Object>) new
> JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
> ing(conf), JSONValue.defaultReader.DEFAULT);
>
> This will result in "Long" but might have other side-effects.
>
> From my point of view, the newer JSON library works as expected
> (although it might break backward compatibility).
> Unchecked casts are always evil, imho. In your specific case, it might
> be more save to cast to "Number" and than invoke "longValue()" instead.
>
> Nevertheless, we might being able to restore the old behaviour by
> tuning the JSONParser configuration.
>
> Thoughts?
>
> Gruß
> Richard
>
>
>
>
> Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
> Vermeerbergen:
> > On a side note, the Long value that "MyKOBolt" is set by a
> > DefaultTopologyConfig to a value with is a java.lang.Long instance...
> > hence it's surprising to see it internally converted into a
> > java.lang.Integer in the Map passed to prepare() method...
> >
> >
> > Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
> > <av...@gmail.com> a écrit :
> > >
> > > Hello,
> > >
> > > I found a breaking change using Apache Storm 2.6.0 snapshot build
> > > dated 20231103 and without changing my topologies' code (keeping
> > > the
> > > same big jars).
> > >
> > > This seems to be tied to a datatype change in Map argument of
> > > prepare() method of Bolt.
> > >
> > > For example, this Bolt is KO at runtime:
> > >
> > > public class MyKOBolt extends BaseRichBolt {
> > >
> > > @Override
> > > public void prepare(Map stormConf, TopologyContext context,
> > > OutputCollector collector) {
> > > cacheMaxSize = (Long)
> > > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > > cacheTtlMin = (Long)
> > > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> > >
> > > as it throws this error:
> > > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > > Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
> > > __system:[-1]
> > > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > > Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop
> > > died!
> > > java.lang.ClassCastException: java.lang.Integer incompatible with
> > > java.lang.Long
> > > at com.acme.storm.alerting.MyKOBolt
> > > .prepare(EvaluationBolt.java:71)
> > > ~[stormjar.jar:?]
> > > at
> > > org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:
> > > 128)
> > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > at
> > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > > 138)
> > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > at
> > > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > > 54)
> > > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > at org.apache.storm.utils.Utils$1.run(Utils.java:393)
> > > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > > at java.lang.Thread.run(Thread.java:857) [?:?]
> > >
> > > whereas this bolt is fine with Storm 2.6.0:
> > >
> > > public class MyOKBolt extends BaseBasicBolt {
> > >
> > > @Override
> > > public void prepare(final Map stormConf, final TopologyContext
> > > context) {
> > > super.prepare(stormConf, context);
> > >
> > > this.redisPort =
> > > Integer.parseInt((String)
> > > stormConf.get(ConfKeys.REDIS_PORT));
> > >
> > > That said, I can modify the code of "MyKOBolt" to use same
> > > Integer.parseInt or Integer.parseLong trick, but it's the first
> > > time
> > > in my long history of upgrades that I have seen such runtime
> > > incompatibility.
> > >
> > > Maybe "MyKOBolt" was badly written since the beginning and I have
> > > just
> > > hit the punishment for it: is there a documentation which clarifies
> > > datatypes of the Map argument of prepare() method?
> > >
> > > Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
> > > "MyOKBolt" derives from BaseBasicBolt => could this have any impact
> > > on
> > > this finding ?
> > >
> > > Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK
> > > 17.0.8.1.
> > >
> > > Kind regards,
> > > Alexandre
> > >
> > > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > > <li...@gmail.com> a écrit :
> > > >
> > > > Thanks Richard.
> > > >
> > > > Tried the latest snapshot with StormCrawler both in local and
> > > > deployed mode
> > > > and did not find any issues.
> > > > Will try it on a topology generating WARC files next week to
> > > > check that the
> > > > dependency changes on Hadoop have not broken anything.
> > > >
> > > > Have a good week end
> > > >
> > > > Julien
> > > >
> > > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <rz...@apache.org>
> > > > wrote:
> > > >
> > > > > Short update. Sorted out an issue with the tar.gz/zip with
> > > > > Julien today
> > > > > and re-uploaded them to the nightlies area.
> > > > >
> > > > > This new bundle works as expected in my deployment but happy to
> > > > > receive
> > > > > additional feedback before getting up a first release candidate
> > > > > ;-)
> > > > >
> > > > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard
> > > > > Zowalla:
> > > > > > Hi Alexandre,
> > > > > >
> > > > > > we are not in a hurry here :) - take as much as time you
> > > > > > need.
> > > > > >
> > > > > > Gruß
> > > > > > Richard
> > > > > >
> > > > > >
> > > > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > > > > > Vermeerbergen:
> > > > > > > Hello Richard,
> > > > > > >
> > > > > > > Okay, I'm more than happy to do that with my pre-production
> > > > > > > cluster
> > > > > > > (~10 topologies) using the binary artifacts.
> > > > > > > Would it be OK if I can use up to end of this week so that
> > > > > > > I'll be
> > > > > > > able to have enough time to check all potential issues that
> > > > > > > this
> > > > > > > upgrade bring?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Alexandre
> > > > > > >
> > > > > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla
> > > > > > > <rz...@apache.org> a
> > > > > > > écrit :
> > > > > > > >
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > before starting a release of 2.6.0-SNAPSHOT, I would like
> > > > > > > > to
> > > > > > > > receive
> > > > > > > > some feedback on the current SNAPSHOT build.
> > > > > > > >
> > > > > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > > > > > > >
> > > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > > >
> > > > > > > > It is available via the ASF Snapshot repository:
> > > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > > >
> > > > > > > > You can consume by adding
> > > > > > > >
> > > > > > > > <repository>
> > > > > > > > <id>apache.snapshots</id>
> > > > > > > > <name>Apache Snapshot Repository</name>
> > > > > > > > <url>https://repository.apache.org/snapshots</url>
> > > > > > > > <releases>
> > > > > > > > <enabled>false</enabled>
> > > > > > > > </releases>
> > > > > > > > </repository>
> > > > > > > >
> > > > > > > > to your project pom. As we do not deploy SNAPSHOTS
> > > > > > > > automatically,
> > > > > > > > it
> > > > > > > > should be easy to just consume the latest SNAPSHOT.
> > > > > > > >
> > > > > > > > The packaged binaries are available at
> > > > > > > > nightlies.apache.org:
> > > > > > > >
> > > > > > > >
> > > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > > >
> > > > > > > > If you have some minutes left: Please test and report any
> > > > > > > > issues
> > > > > > > > with
> > > > > > > > this binaries, so we can fix before attempting to
> > > > > > > > release.
> > > > > > > >
> > > > > > > > The most significant changes are in the are of
> > > > > > > > hadoop/hbase/hdfs
> > > > > > > > as
> > > > > > > > we
> > > > > > > > upgraded from 2.x to 3.x - our own test coverage within
> > > > > > > > the build
> > > > > > > > looks
> > > > > > > > good but would be nice to get some real world use-case
> > > > > > > > feedback
> > > > > > > > on
> > > > > > > > this. In addition, we had quite a lof of 3rd party
> > > > > > > > dependency
> > > > > > > > upgrades.
> > > > > > > >
> > > > > > > > In addition, it contains the pruning of external modules
> > > > > > > > as
> > > > > > > > listed
> > > > > > > > in
> > > > > > > >
> > > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > > >
> > > > > > > > A summaryin Jira is here:
> > > > > > > >
> > > > > > > >
> > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > > >
> > > > > > > > Gruß
> > > > > > > > Richard
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > >
> > > > *Open Source Solutions for Text Engineering*
> > > >
> > > > http://www.digitalpebble.com
> > > > http://digitalpebble.blogspot.com/
> > > > #digitalpebble <http://twitter.com/digitalpebble>
>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Richard Zowalla <rz...@apache.org>.
Hi,
thanks for the feedback.
The main difference to 2.5.0 is, that we switched the JSON
implementation in Utils, which relied on a
super old JSON library. What is happening here is, that the new JSON
library (net.minidev) behaves differently.
We can see it by writing some simple code without Storm (The related
code line with the parsing is taken from Storm Utils)
Map<String, Long> conf = new HashMap<>();
conf.put("test", 100000000L);
Map<String, Object> deserTopoConf = (Map<String, Object>)
JSONValue.parseWithException(JSONValue.toJSONString(conf));
for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
System.out.println(e.getValue().getClass().getSimpleName());
}
This will print "Integer". Using "10000000000000000L" as a value will
result in "Long".
The main issue derives from the fact, that we serialize the config map
into a JSON String and parse the result.
Based on the String representation, "100000000L" will still fit into an
Integer, so the parser will not use Long ;-)
If the number is big enough, the parser will switch to a Long.
The old library (> 10 years old now):
deserTopoConf = (Map<String, Object>)
org.json.simple.JSONValue.parseWithException(org.json.simple.JSONValue.
toJSONString(conf));
for(Map.Entry<String, Object> e : deserTopoConf.entrySet()) {
System.out.println(e.getValue().getClass().getSimpleName());
}
This will print "Long".
The difference between these libraries is, that the super old
org.json.simple just always returns Long as the type.
If can eventually tune the new JSON parser to something like:
deserTopoConf = (Map<String, Object>) new
JSONParser(JSONParser.BIG_DIGIT_UNRESTRICTED).parse(JSONValue.toJSONStr
ing(conf), JSONValue.defaultReader.DEFAULT);
This will result in "Long" but might have other side-effects.
From my point of view, the newer JSON library works as expected
(although it might break backward compatibility).
Unchecked casts are always evil, imho. In your specific case, it might
be more save to cast to "Number" and than invoke "longValue()" instead.
Nevertheless, we might being able to restore the old behaviour by
tuning the JSONParser configuration.
Thoughts?
Gruß
Richard
Am Freitag, dem 03.11.2023 um 18:32 +0100 schrieb Alexandre
Vermeerbergen:
> On a side note, the Long value that "MyKOBolt" is set by a
> DefaultTopologyConfig to a value with is a java.lang.Long instance...
> hence it's surprising to see it internally converted into a
> java.lang.Integer in the Map passed to prepare() method...
>
>
> Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
> <av...@gmail.com> a écrit :
> >
> > Hello,
> >
> > I found a breaking change using Apache Storm 2.6.0 snapshot build
> > dated 20231103 and without changing my topologies' code (keeping
> > the
> > same big jars).
> >
> > This seems to be tied to a datatype change in Map argument of
> > prepare() method of Bolt.
> >
> > For example, this Bolt is KO at runtime:
> >
> > public class MyKOBolt extends BaseRichBolt {
> >
> > @Override
> > public void prepare(Map stormConf, TopologyContext context,
> > OutputCollector collector) {
> > cacheMaxSize = (Long)
> > stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> > cacheTtlMin = (Long)
> > stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
> >
> > as it throws this error:
> > 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> > Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
> > __system:[-1]
> > 2023-11-03 16:42:20.193 o.a.s.u.Utils
> > Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop
> > died!
> > java.lang.ClassCastException: java.lang.Integer incompatible with
> > java.lang.Long
> > at com.acme.storm.alerting.MyKOBolt
> > .prepare(EvaluationBolt.java:71)
> > ~[stormjar.jar:?]
> > at
> > org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:
> > 128)
> > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > at
> > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > 138)
> > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > at
> > org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:
> > 54)
> > ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > at org.apache.storm.utils.Utils$1.run(Utils.java:393)
> > [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> > at java.lang.Thread.run(Thread.java:857) [?:?]
> >
> > whereas this bolt is fine with Storm 2.6.0:
> >
> > public class MyOKBolt extends BaseBasicBolt {
> >
> > @Override
> > public void prepare(final Map stormConf, final TopologyContext
> > context) {
> > super.prepare(stormConf, context);
> >
> > this.redisPort =
> > Integer.parseInt((String)
> > stormConf.get(ConfKeys.REDIS_PORT));
> >
> > That said, I can modify the code of "MyKOBolt" to use same
> > Integer.parseInt or Integer.parseLong trick, but it's the first
> > time
> > in my long history of upgrades that I have seen such runtime
> > incompatibility.
> >
> > Maybe "MyKOBolt" was badly written since the beginning and I have
> > just
> > hit the punishment for it: is there a documentation which clarifies
> > datatypes of the Map argument of prepare() method?
> >
> > Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
> > "MyOKBolt" derives from BaseBasicBolt => could this have any impact
> > on
> > this finding ?
> >
> > Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK
> > 17.0.8.1.
> >
> > Kind regards,
> > Alexandre
> >
> > Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> > <li...@gmail.com> a écrit :
> > >
> > > Thanks Richard.
> > >
> > > Tried the latest snapshot with StormCrawler both in local and
> > > deployed mode
> > > and did not find any issues.
> > > Will try it on a topology generating WARC files next week to
> > > check that the
> > > dependency changes on Hadoop have not broken anything.
> > >
> > > Have a good week end
> > >
> > > Julien
> > >
> > > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <rz...@apache.org>
> > > wrote:
> > >
> > > > Short update. Sorted out an issue with the tar.gz/zip with
> > > > Julien today
> > > > and re-uploaded them to the nightlies area.
> > > >
> > > > This new bundle works as expected in my deployment but happy to
> > > > receive
> > > > additional feedback before getting up a first release candidate
> > > > ;-)
> > > >
> > > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard
> > > > Zowalla:
> > > > > Hi Alexandre,
> > > > >
> > > > > we are not in a hurry here :) - take as much as time you
> > > > > need.
> > > > >
> > > > > Gruß
> > > > > Richard
> > > > >
> > > > >
> > > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > > > > Vermeerbergen:
> > > > > > Hello Richard,
> > > > > >
> > > > > > Okay, I'm more than happy to do that with my pre-production
> > > > > > cluster
> > > > > > (~10 topologies) using the binary artifacts.
> > > > > > Would it be OK if I can use up to end of this week so that
> > > > > > I'll be
> > > > > > able to have enough time to check all potential issues that
> > > > > > this
> > > > > > upgrade bring?
> > > > > >
> > > > > > Thanks,
> > > > > > Alexandre
> > > > > >
> > > > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla
> > > > > > <rz...@apache.org> a
> > > > > > écrit :
> > > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > before starting a release of 2.6.0-SNAPSHOT, I would like
> > > > > > > to
> > > > > > > receive
> > > > > > > some feedback on the current SNAPSHOT build.
> > > > > > >
> > > > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > > > > > >
> > > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > > >
> > > > > > > It is available via the ASF Snapshot repository:
> > > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > > >
> > > > > > > You can consume by adding
> > > > > > >
> > > > > > > <repository>
> > > > > > > <id>apache.snapshots</id>
> > > > > > > <name>Apache Snapshot Repository</name>
> > > > > > > <url>https://repository.apache.org/snapshots</url>
> > > > > > > <releases>
> > > > > > > <enabled>false</enabled>
> > > > > > > </releases>
> > > > > > > </repository>
> > > > > > >
> > > > > > > to your project pom. As we do not deploy SNAPSHOTS
> > > > > > > automatically,
> > > > > > > it
> > > > > > > should be easy to just consume the latest SNAPSHOT.
> > > > > > >
> > > > > > > The packaged binaries are available at
> > > > > > > nightlies.apache.org:
> > > > > > >
> > > > > > >
> > > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > > >
> > > > > > > If you have some minutes left: Please test and report any
> > > > > > > issues
> > > > > > > with
> > > > > > > this binaries, so we can fix before attempting to
> > > > > > > release.
> > > > > > >
> > > > > > > The most significant changes are in the are of
> > > > > > > hadoop/hbase/hdfs
> > > > > > > as
> > > > > > > we
> > > > > > > upgraded from 2.x to 3.x - our own test coverage within
> > > > > > > the build
> > > > > > > looks
> > > > > > > good but would be nice to get some real world use-case
> > > > > > > feedback
> > > > > > > on
> > > > > > > this. In addition, we had quite a lof of 3rd party
> > > > > > > dependency
> > > > > > > upgrades.
> > > > > > >
> > > > > > > In addition, it contains the pruning of external modules
> > > > > > > as
> > > > > > > listed
> > > > > > > in
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > > >
> > > > > > > A summaryin Jira is here:
> > > > > > >
> > > > > > >
> > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > > >
> > > > > > > Gruß
> > > > > > > Richard
> > > > > > >
> > > > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > >
> > > *Open Source Solutions for Text Engineering*
> > >
> > > http://www.digitalpebble.com
> > > http://digitalpebble.blogspot.com/
> > > #digitalpebble <http://twitter.com/digitalpebble>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
On a side note, the Long value that "MyKOBolt" is set by a
DefaultTopologyConfig to a value with is a java.lang.Long instance...
hence it's surprising to see it internally converted into a
java.lang.Integer in the Map passed to prepare() method...
Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerbergen
<av...@gmail.com> a écrit :
>
> Hello,
>
> I found a breaking change using Apache Storm 2.6.0 snapshot build
> dated 20231103 and without changing my topologies' code (keeping the
> same big jars).
>
> This seems to be tied to a datatype change in Map argument of
> prepare() method of Bolt.
>
> For example, this Bolt is KO at runtime:
>
> public class MyKOBolt extends BaseRichBolt {
>
> @Override
> public void prepare(Map stormConf, TopologyContext context,
> OutputCollector collector) {
> cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
> cacheTtlMin = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
>
> as it throws this error:
> 2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
> Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
> __system:[-1]
> 2023-11-03 16:42:20.193 o.a.s.u.Utils
> Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop died!
> java.lang.ClassCastException: java.lang.Integer incompatible with java.lang.Long
> at com.acme.storm.alerting.MyKOBolt .prepare(EvaluationBolt.java:71)
> ~[stormjar.jar:?]
> at org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:128)
> ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:138)
> ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:54)
> ~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> at org.apache.storm.utils.Utils$1.run(Utils.java:393)
> [storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
> at java.lang.Thread.run(Thread.java:857) [?:?]
>
> whereas this bolt is fine with Storm 2.6.0:
>
> public class MyOKBolt extends BaseBasicBolt {
>
> @Override
> public void prepare(final Map stormConf, final TopologyContext context) {
> super.prepare(stormConf, context);
>
> this.redisPort = Integer.parseInt((String)
> stormConf.get(ConfKeys.REDIS_PORT));
>
> That said, I can modify the code of "MyKOBolt" to use same
> Integer.parseInt or Integer.parseLong trick, but it's the first time
> in my long history of upgrades that I have seen such runtime
> incompatibility.
>
> Maybe "MyKOBolt" was badly written since the beginning and I have just
> hit the punishment for it: is there a documentation which clarifies
> datatypes of the Map argument of prepare() method?
>
> Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
> "MyOKBolt" derives from BaseBasicBolt => could this have any impact on
> this finding ?
>
> Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK 17.0.8.1.
>
> Kind regards,
> Alexandre
>
> Le ven. 3 nov. 2023 à 15:16, Julien Nioche
> <li...@gmail.com> a écrit :
> >
> > Thanks Richard.
> >
> > Tried the latest snapshot with StormCrawler both in local and deployed mode
> > and did not find any issues.
> > Will try it on a topology generating WARC files next week to check that the
> > dependency changes on Hadoop have not broken anything.
> >
> > Have a good week end
> >
> > Julien
> >
> > On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <rz...@apache.org> wrote:
> >
> > > Short update. Sorted out an issue with the tar.gz/zip with Julien today
> > > and re-uploaded them to the nightlies area.
> > >
> > > This new bundle works as expected in my deployment but happy to receive
> > > additional feedback before getting up a first release candidate ;-)
> > >
> > > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard Zowalla:
> > > > Hi Alexandre,
> > > >
> > > > we are not in a hurry here :) - take as much as time you need.
> > > >
> > > > Gruß
> > > > Richard
> > > >
> > > >
> > > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > > > Vermeerbergen:
> > > > > Hello Richard,
> > > > >
> > > > > Okay, I'm more than happy to do that with my pre-production cluster
> > > > > (~10 topologies) using the binary artifacts.
> > > > > Would it be OK if I can use up to end of this week so that I'll be
> > > > > able to have enough time to check all potential issues that this
> > > > > upgrade bring?
> > > > >
> > > > > Thanks,
> > > > > Alexandre
> > > > >
> > > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla <rz...@apache.org> a
> > > > > écrit :
> > > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > before starting a release of 2.6.0-SNAPSHOT, I would like to
> > > > > > receive
> > > > > > some feedback on the current SNAPSHOT build.
> > > > > >
> > > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > > > > >
> > > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > > >
> > > > > > It is available via the ASF Snapshot repository:
> > > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > > >
> > > > > > You can consume by adding
> > > > > >
> > > > > > <repository>
> > > > > > <id>apache.snapshots</id>
> > > > > > <name>Apache Snapshot Repository</name>
> > > > > > <url>https://repository.apache.org/snapshots</url>
> > > > > > <releases>
> > > > > > <enabled>false</enabled>
> > > > > > </releases>
> > > > > > </repository>
> > > > > >
> > > > > > to your project pom. As we do not deploy SNAPSHOTS automatically,
> > > > > > it
> > > > > > should be easy to just consume the latest SNAPSHOT.
> > > > > >
> > > > > > The packaged binaries are available at nightlies.apache.org:
> > > > > >
> > > > > >
> > > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > > >
> > > > > > If you have some minutes left: Please test and report any issues
> > > > > > with
> > > > > > this binaries, so we can fix before attempting to release.
> > > > > >
> > > > > > The most significant changes are in the are of hadoop/hbase/hdfs
> > > > > > as
> > > > > > we
> > > > > > upgraded from 2.x to 3.x - our own test coverage within the build
> > > > > > looks
> > > > > > good but would be nice to get some real world use-case feedback
> > > > > > on
> > > > > > this. In addition, we had quite a lof of 3rd party dependency
> > > > > > upgrades.
> > > > > >
> > > > > > In addition, it contains the pruning of external modules as
> > > > > > listed
> > > > > > in
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > > >
> > > > > > A summaryin Jira is here:
> > > > > >
> > > > > >
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > > >
> > > > > > Gruß
> > > > > > Richard
> > > > > >
> > > > > >
> > > >
> > >
> > >
> >
> > --
> >
> > *Open Source Solutions for Text Engineering*
> >
> > http://www.digitalpebble.com
> > http://digitalpebble.blogspot.com/
> > #digitalpebble <http://twitter.com/digitalpebble>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Alexandre Vermeerbergen <av...@gmail.com>.
Hello,
I found a breaking change using Apache Storm 2.6.0 snapshot build
dated 20231103 and without changing my topologies' code (keeping the
same big jars).
This seems to be tied to a datatype change in Map argument of
prepare() method of Bolt.
For example, this Bolt is KO at runtime:
public class MyKOBolt extends BaseRichBolt {
@Override
public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
cacheTtlMin = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
as it throws this error:
2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
__system:[-1]
2023-11-03 16:42:20.193 o.a.s.u.Utils
Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop died!
java.lang.ClassCastException: java.lang.Integer incompatible with java.lang.Long
at com.acme.storm.alerting.MyKOBolt .prepare(EvaluationBolt.java:71)
~[stormjar.jar:?]
at org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:128)
~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:138)
~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:54)
~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
at org.apache.storm.utils.Utils$1.run(Utils.java:393)
[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
at java.lang.Thread.run(Thread.java:857) [?:?]
whereas this bolt is fine with Storm 2.6.0:
public class MyOKBolt extends BaseBasicBolt {
@Override
public void prepare(final Map stormConf, final TopologyContext context) {
super.prepare(stormConf, context);
this.redisPort = Integer.parseInt((String)
stormConf.get(ConfKeys.REDIS_PORT));
That said, I can modify the code of "MyKOBolt" to use same
Integer.parseInt or Integer.parseLong trick, but it's the first time
in my long history of upgrades that I have seen such runtime
incompatibility.
Maybe "MyKOBolt" was badly written since the beginning and I have just
hit the punishment for it: is there a documentation which clarifies
datatypes of the Map argument of prepare() method?
Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
"MyOKBolt" derives from BaseBasicBolt => could this have any impact on
this finding ?
Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK 17.0.8.1.
Kind regards,
Alexandre
Le ven. 3 nov. 2023 à 15:16, Julien Nioche
<li...@gmail.com> a écrit :
>
> Thanks Richard.
>
> Tried the latest snapshot with StormCrawler both in local and deployed mode
> and did not find any issues.
> Will try it on a topology generating WARC files next week to check that the
> dependency changes on Hadoop have not broken anything.
>
> Have a good week end
>
> Julien
>
> On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <rz...@apache.org> wrote:
>
> > Short update. Sorted out an issue with the tar.gz/zip with Julien today
> > and re-uploaded them to the nightlies area.
> >
> > This new bundle works as expected in my deployment but happy to receive
> > additional feedback before getting up a first release candidate ;-)
> >
> > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard Zowalla:
> > > Hi Alexandre,
> > >
> > > we are not in a hurry here :) - take as much as time you need.
> > >
> > > Gruß
> > > Richard
> > >
> > >
> > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > > Vermeerbergen:
> > > > Hello Richard,
> > > >
> > > > Okay, I'm more than happy to do that with my pre-production cluster
> > > > (~10 topologies) using the binary artifacts.
> > > > Would it be OK if I can use up to end of this week so that I'll be
> > > > able to have enough time to check all potential issues that this
> > > > upgrade bring?
> > > >
> > > > Thanks,
> > > > Alexandre
> > > >
> > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla <rz...@apache.org> a
> > > > écrit :
> > > > >
> > > > > Hi all,
> > > > >
> > > > > before starting a release of 2.6.0-SNAPSHOT, I would like to
> > > > > receive
> > > > > some feedback on the current SNAPSHOT build.
> > > > >
> > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > > > >
> > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > >
> > > > > It is available via the ASF Snapshot repository:
> > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > >
> > > > > You can consume by adding
> > > > >
> > > > > <repository>
> > > > > <id>apache.snapshots</id>
> > > > > <name>Apache Snapshot Repository</name>
> > > > > <url>https://repository.apache.org/snapshots</url>
> > > > > <releases>
> > > > > <enabled>false</enabled>
> > > > > </releases>
> > > > > </repository>
> > > > >
> > > > > to your project pom. As we do not deploy SNAPSHOTS automatically,
> > > > > it
> > > > > should be easy to just consume the latest SNAPSHOT.
> > > > >
> > > > > The packaged binaries are available at nightlies.apache.org:
> > > > >
> > > > >
> > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > >
> > > > > If you have some minutes left: Please test and report any issues
> > > > > with
> > > > > this binaries, so we can fix before attempting to release.
> > > > >
> > > > > The most significant changes are in the are of hadoop/hbase/hdfs
> > > > > as
> > > > > we
> > > > > upgraded from 2.x to 3.x - our own test coverage within the build
> > > > > looks
> > > > > good but would be nice to get some real world use-case feedback
> > > > > on
> > > > > this. In addition, we had quite a lof of 3rd party dependency
> > > > > upgrades.
> > > > >
> > > > > In addition, it contains the pruning of external modules as
> > > > > listed
> > > > > in
> > > > >
> > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > >
> > > > > A summaryin Jira is here:
> > > > >
> > > > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > >
> > > > > Gruß
> > > > > Richard
> > > > >
> > > > >
> > >
> >
> >
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble <http://twitter.com/digitalpebble>
Re: [HELP NEEDED] Please test 2.6.0-SNAPSHOT
Posted by Julien Nioche <li...@gmail.com>.
Thanks Richard.
Tried the latest snapshot with StormCrawler both in local and deployed mode
and did not find any issues.
Will try it on a topology generating WARC files next week to check that the
dependency changes on Hadoop have not broken anything.
Have a good week end
Julien
On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <rz...@apache.org> wrote:
> Short update. Sorted out an issue with the tar.gz/zip with Julien today
> and re-uploaded them to the nightlies area.
>
> This new bundle works as expected in my deployment but happy to receive
> additional feedback before getting up a first release candidate ;-)
>
> Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard Zowalla:
> > Hi Alexandre,
> >
> > we are not in a hurry here :) - take as much as time you need.
> >
> > Gruß
> > Richard
> >
> >
> > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > Vermeerbergen:
> > > Hello Richard,
> > >
> > > Okay, I'm more than happy to do that with my pre-production cluster
> > > (~10 topologies) using the binary artifacts.
> > > Would it be OK if I can use up to end of this week so that I'll be
> > > able to have enough time to check all potential issues that this
> > > upgrade bring?
> > >
> > > Thanks,
> > > Alexandre
> > >
> > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla <rz...@apache.org> a
> > > écrit :
> > > >
> > > > Hi all,
> > > >
> > > > before starting a release of 2.6.0-SNAPSHOT, I would like to
> > > > receive
> > > > some feedback on the current SNAPSHOT build.
> > > >
> > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > > >
> https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > >
> > > > It is available via the ASF Snapshot repository:
> > > > https://repository.apache.org/content/repositories/snapshots/
> > > >
> > > > You can consume by adding
> > > >
> > > > <repository>
> > > > <id>apache.snapshots</id>
> > > > <name>Apache Snapshot Repository</name>
> > > > <url>https://repository.apache.org/snapshots</url>
> > > > <releases>
> > > > <enabled>false</enabled>
> > > > </releases>
> > > > </repository>
> > > >
> > > > to your project pom. As we do not deploy SNAPSHOTS automatically,
> > > > it
> > > > should be easy to just consume the latest SNAPSHOT.
> > > >
> > > > The packaged binaries are available at nightlies.apache.org:
> > > >
> > > >
> https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > >
> > > > If you have some minutes left: Please test and report any issues
> > > > with
> > > > this binaries, so we can fix before attempting to release.
> > > >
> > > > The most significant changes are in the are of hadoop/hbase/hdfs
> > > > as
> > > > we
> > > > upgraded from 2.x to 3.x - our own test coverage within the build
> > > > looks
> > > > good but would be nice to get some real world use-case feedback
> > > > on
> > > > this. In addition, we had quite a lof of 3rd party dependency
> > > > upgrades.
> > > >
> > > > In addition, it contains the pruning of external modules as
> > > > listed
> > > > in
> > > >
> > > > https://issues.apache.org/jira/browse/STORM-3988
> > > >
> > > > A summaryin Jira is here:
> > > >
> > > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > >
> > > > Gruß
> > > > Richard
> > > >
> > > >
> >
>
>
--
*Open Source Solutions for Text Engineering*
http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble <http://twitter.com/digitalpebble>