You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Arne Degenring <ar...@luxonit.com> on 2018/03/02 14:44:31 UTC
NiFI Maven Plugin - Timestamped Snapshot Version Problem
Hi,
We have seen a strange problem on NiFi 1.4.0 where custom processors could suddenly not be started, because of incompatibility with custom services:
2018-03-02 13:40:35,490 ERROR [main] o.apache.nifi.controller.FlowController Unable to start …[id=5d57d39a-015c-1000-ffff-ffffd654d90b] due to java.lang.IllegalStateException: Processor … is not in a valid state due to … is invalid because … - 1.4-SNAPSHOT from … is not compatible with … - 1.4-SNAPSHOT …]
It seems that the root cause was related to:
2018-03-02 13:39:55,086 WARN [main] org.apache.nifi.nar.NarClassLoaders While loading …:…:1.5-SNAPSHOT' unable to locate exact NAR dependency '…:…:1.4-20180302.111133-16'. Only found one possible match …:….:1.4-SNAPSHOT'. Continuing...
It turned out that our various custom NARs were built on different built agents.
Maven has the (ugly) behaviour that when a snapshot version is present in the local repository after local build, the version number will be e.g. 1.4-SNAPSHOT. However if it is downloaded from a remote repository, the version number includes a timestamp such as 1.4-20180302.111133-16.
So the manifest of the depending NARs contained:
Nar-Dependency-Version: 1.4-20180302.111133-16
while the other NAR file declared:
Nar-Version: 1.4-SNAPSHOT
Not sure if NiFi 1.5 would behave differently when loading such NAR files, but I would recommend to anyway fix the NiFi Maven Plugin:
https://github.com/apache/nifi-maven/blob/master/src/main/java/org/apache/nifi/NarMojo.java
Replace line 705:
narDependency = new NarDependency(artifact.getGroupId(), artifact.getArtifactId(), artifact.getVersion());
with:
narDependency = new NarDependency(artifact.getGroupId(), artifact.getArtifactId(), artifact.getBaseVersion());
Explanation: artifact.getBaseVersion() will always consistenly return -SNAPSHOT, even in case the artifact was a timestamped snapshot downloaded from a remote repo.
Thanks
Arne
Re: NiFI Maven Plugin - Timestamped Snapshot Version Problem
Posted by Arne Degenring <ar...@luxonit.com>.
Hi Bryan,
done, please feel free to amend:
https://issues.apache.org/jira/browse/NIFI-4930 <https://issues.apache.org/jira/browse/NIFI-4930>
Thanks
Arne
> On 2. Mar 2018, at 19:41, Bryan Bende <bb...@gmail.com> wrote:
>
> Hi Arne,
>
> Thanks for the detailed explanation, that is an interesting scenario,
> but I can see why it would be problematic.
>
> Can you file a NiFi JIRA with that write up and tag the component as
> "Nar-Maven-Plugin" ?
>
> Thanks,
>
> Bryan
>
>
> On Fri, Mar 2, 2018 at 12:02 PM, Arne Degenring
> <ar...@luxonit.com> wrote:
>> Hi Mike,
>>
>> No, we are running standard NiFi 1.4.0 and are using the standard NiFi Maven
>> plugin 1.2.0.
>>
>> We just have implemented a couple of custom processor and service NARs.
>>
>> Thanks
>> Arne
>>
>> On 2. Mar 2018, at 17:54, Mike Thomsen <mi...@gmail.com> wrote:
>>
>> Are you by any chance running a custom build of NiFi?
>>
>> On Fri, Mar 2, 2018 at 9:44 AM, Arne Degenring <ar...@luxonit.com>
>> wrote:
>>>
>>> Hi,
>>>
>>>
>>>
>>> We have seen a strange problem on NiFi 1.4.0 where custom processors could
>>> suddenly not be started, because of incompatibility with custom services:
>>>
>>>
>>>
>>> 2018-03-02 13:40:35,490 ERROR [main]
>>> o.apache.nifi.controller.FlowController Unable to start
>>> …[id=5d57d39a-015c-1000-ffff-ffffd654d90b] due to
>>> java.lang.IllegalStateException: Processor … is not in a valid state due to
>>> … is invalid because … - 1.4-SNAPSHOT from … is not compatible with … -
>>> 1.4-SNAPSHOT …]
>>>
>>>
>>>
>>> It seems that the root cause was related to:
>>>
>>>
>>>
>>> 2018-03-02 13:39:55,086 WARN [main] org.apache.nifi.nar.NarClassLoaders
>>> While loading …:…:1.5-SNAPSHOT' unable to locate exact NAR dependency
>>> '…:…:1.4-20180302.111133-16'. Only found one possible match
>>> …:….:1.4-SNAPSHOT'. Continuing...
>>>
>>>
>>>
>>> It turned out that our various custom NARs were built on different built
>>> agents.
>>>
>>>
>>>
>>> Maven has the (ugly) behaviour that when a snapshot version is present in
>>> the local repository after local build, the version number will be e.g.
>>> 1.4-SNAPSHOT. However if it is downloaded from a remote repository, the
>>> version number includes a timestamp such as 1.4-20180302.111133-16.
>>>
>>>
>>>
>>> So the manifest of the depending NARs contained:
>>>
>>>
>>>
>>> Nar-Dependency-Version: 1.4-20180302.111133-16
>>>
>>>
>>>
>>> while the other NAR file declared:
>>>
>>>
>>>
>>> Nar-Version: 1.4-SNAPSHOT
>>>
>>>
>>>
>>> Not sure if NiFi 1.5 would behave differently when loading such NAR files,
>>> but I would recommend to anyway fix the NiFi Maven Plugin:
>>>
>>>
>>>
>>>
>>> https://github.com/apache/nifi-maven/blob/master/src/main/java/org/apache/nifi/NarMojo.java
>>>
>>>
>>>
>>> Replace line 705:
>>> narDependency = new NarDependency(artifact.getGroupId(),
>>> artifact.getArtifactId(), artifact.getVersion());
>>>
>>>
>>>
>>> with:
>>> narDependency = new NarDependency(artifact.getGroupId(),
>>> artifact.getArtifactId(), artifact.getBaseVersion());
>>>
>>>
>>>
>>> Explanation: artifact.getBaseVersion() will always consistenly return
>>> -SNAPSHOT, even in case the artifact was a timestamped snapshot downloaded
>>> from a remote repo.
>>>
>>>
>>>
>>> Thanks
>>> Arne
>>>
>>>
>>>
>>>
>>
>>
>>
Re: NiFI Maven Plugin - Timestamped Snapshot Version Problem
Posted by Bryan Bende <bb...@gmail.com>.
Hi Arne,
Thanks for the detailed explanation, that is an interesting scenario,
but I can see why it would be problematic.
Can you file a NiFi JIRA with that write up and tag the component as
"Nar-Maven-Plugin" ?
Thanks,
Bryan
On Fri, Mar 2, 2018 at 12:02 PM, Arne Degenring
<ar...@luxonit.com> wrote:
> Hi Mike,
>
> No, we are running standard NiFi 1.4.0 and are using the standard NiFi Maven
> plugin 1.2.0.
>
> We just have implemented a couple of custom processor and service NARs.
>
> Thanks
> Arne
>
> On 2. Mar 2018, at 17:54, Mike Thomsen <mi...@gmail.com> wrote:
>
> Are you by any chance running a custom build of NiFi?
>
> On Fri, Mar 2, 2018 at 9:44 AM, Arne Degenring <ar...@luxonit.com>
> wrote:
>>
>> Hi,
>>
>>
>>
>> We have seen a strange problem on NiFi 1.4.0 where custom processors could
>> suddenly not be started, because of incompatibility with custom services:
>>
>>
>>
>> 2018-03-02 13:40:35,490 ERROR [main]
>> o.apache.nifi.controller.FlowController Unable to start
>> …[id=5d57d39a-015c-1000-ffff-ffffd654d90b] due to
>> java.lang.IllegalStateException: Processor … is not in a valid state due to
>> … is invalid because … - 1.4-SNAPSHOT from … is not compatible with … -
>> 1.4-SNAPSHOT …]
>>
>>
>>
>> It seems that the root cause was related to:
>>
>>
>>
>> 2018-03-02 13:39:55,086 WARN [main] org.apache.nifi.nar.NarClassLoaders
>> While loading …:…:1.5-SNAPSHOT' unable to locate exact NAR dependency
>> '…:…:1.4-20180302.111133-16'. Only found one possible match
>> …:….:1.4-SNAPSHOT'. Continuing...
>>
>>
>>
>> It turned out that our various custom NARs were built on different built
>> agents.
>>
>>
>>
>> Maven has the (ugly) behaviour that when a snapshot version is present in
>> the local repository after local build, the version number will be e.g.
>> 1.4-SNAPSHOT. However if it is downloaded from a remote repository, the
>> version number includes a timestamp such as 1.4-20180302.111133-16.
>>
>>
>>
>> So the manifest of the depending NARs contained:
>>
>>
>>
>> Nar-Dependency-Version: 1.4-20180302.111133-16
>>
>>
>>
>> while the other NAR file declared:
>>
>>
>>
>> Nar-Version: 1.4-SNAPSHOT
>>
>>
>>
>> Not sure if NiFi 1.5 would behave differently when loading such NAR files,
>> but I would recommend to anyway fix the NiFi Maven Plugin:
>>
>>
>>
>>
>> https://github.com/apache/nifi-maven/blob/master/src/main/java/org/apache/nifi/NarMojo.java
>>
>>
>>
>> Replace line 705:
>> narDependency = new NarDependency(artifact.getGroupId(),
>> artifact.getArtifactId(), artifact.getVersion());
>>
>>
>>
>> with:
>> narDependency = new NarDependency(artifact.getGroupId(),
>> artifact.getArtifactId(), artifact.getBaseVersion());
>>
>>
>>
>> Explanation: artifact.getBaseVersion() will always consistenly return
>> -SNAPSHOT, even in case the artifact was a timestamped snapshot downloaded
>> from a remote repo.
>>
>>
>>
>> Thanks
>> Arne
>>
>>
>>
>>
>
>
>
Re: NiFI Maven Plugin - Timestamped Snapshot Version Problem
Posted by Arne Degenring <ar...@luxonit.com>.
Hi Mike,
No, we are running standard NiFi 1.4.0 and are using the standard NiFi Maven plugin 1.2.0.
We just have implemented a couple of custom processor and service NARs.
Thanks
Arne
> On 2. Mar 2018, at 17:54, Mike Thomsen <mi...@gmail.com> wrote:
>
> Are you by any chance running a custom build of NiFi?
>
> On Fri, Mar 2, 2018 at 9:44 AM, Arne Degenring <arne.degenring@luxonit.com <ma...@luxonit.com>> wrote:
> Hi,
>
> We have seen a strange problem on NiFi 1.4.0 where custom processors could suddenly not be started, because of incompatibility with custom services:
>
> 2018-03-02 13:40:35,490 ERROR [main] o.apache.nifi.controller.FlowController Unable to start …[id=5d57d39a-015c-1000-ffff-ffffd654d90b] due to java.lang.IllegalStateException: Processor … is not in a valid state due to … is invalid because … - 1.4-SNAPSHOT from … is not compatible with … - 1.4-SNAPSHOT …]
>
> It seems that the root cause was related to:
>
> 2018-03-02 13:39:55,086 WARN [main] org.apache.nifi.nar.NarClassLoaders While loading …:…:1.5-SNAPSHOT' unable to locate exact NAR dependency '…:…:1.4-20180302.111133-16'. Only found one possible match …:….:1.4-SNAPSHOT'. Continuing...
>
> It turned out that our various custom NARs were built on different built agents.
>
> Maven has the (ugly) behaviour that when a snapshot version is present in the local repository after local build, the version number will be e.g. 1.4-SNAPSHOT. However if it is downloaded from a remote repository, the version number includes a timestamp such as 1.4-20180302.111133-16.
>
> So the manifest of the depending NARs contained:
>
> Nar-Dependency-Version: 1.4-20180302.111133-16
>
> while the other NAR file declared:
>
> Nar-Version: 1.4-SNAPSHOT
>
> Not sure if NiFi 1.5 would behave differently when loading such NAR files, but I would recommend to anyway fix the NiFi Maven Plugin:
>
> https://github.com/apache/nifi-maven/blob/master/src/main/java/org/apache/nifi/NarMojo.java <https://github.com/apache/nifi-maven/blob/master/src/main/java/org/apache/nifi/NarMojo.java>
>
> Replace line 705:
> narDependency = new NarDependency(artifact.getGroupId(), artifact.getArtifactId(), artifact.getVersion());
>
> with:
> narDependency = new NarDependency(artifact.getGroupId(), artifact.getArtifactId(), artifact.getBaseVersion());
>
> Explanation: artifact.getBaseVersion() will always consistenly return -SNAPSHOT, even in case the artifact was a timestamped snapshot downloaded from a remote repo.
>
> Thanks
> Arne
>
>
>
Re: NiFI Maven Plugin - Timestamped Snapshot Version Problem
Posted by Mike Thomsen <mi...@gmail.com>.
Are you by any chance running a custom build of NiFi?
On Fri, Mar 2, 2018 at 9:44 AM, Arne Degenring <ar...@luxonit.com>
wrote:
> Hi,
>
>
>
> We have seen a strange problem on NiFi 1.4.0 where custom processors could
> suddenly not be started, because of incompatibility with custom services:
>
>
>
> 2018-03-02 13:40:35,490 ERROR [main] o.apache.nifi.controller.FlowController
> Unable to start …[id=5d57d39a-015c-1000-ffff-ffffd654d90b] due to
> java.lang.IllegalStateException: Processor … is not in a valid state due
> to … is invalid because … - 1.4-SNAPSHOT from … is not compatible with …
> - 1.4-SNAPSHOT …]
>
>
>
> It seems that the root cause was related to:
>
>
>
> 2018-03-02 13:39:55,086 WARN [main] org.apache.nifi.nar.NarClassLoaders
> While loading …:…:1.5-SNAPSHOT' unable to locate exact NAR dependency
> '…:…:1.4-20180302.111133-16'. Only found one possible match
> …:….:1.4-SNAPSHOT'. Continuing...
>
>
>
> It turned out that our various custom NARs were built on different built
> agents.
>
>
>
> Maven has the (ugly) behaviour that when a snapshot version is present in
> the local repository after local build, the version number will be e.g.
> 1.4-SNAPSHOT. However if it is downloaded from a remote repository, the
> version number includes a timestamp such as 1.4-20180302.111133-16.
>
>
>
> So the manifest of the depending NARs contained:
>
>
>
> Nar-Dependency-Version: 1.4-20180302.111133-16
>
>
>
> while the other NAR file declared:
>
>
>
> Nar-Version: 1.4-SNAPSHOT
>
>
>
> Not sure if NiFi 1.5 would behave differently when loading such NAR files,
> but I would recommend to anyway fix the NiFi Maven Plugin:
>
>
>
> https://github.com/apache/nifi-maven/blob/master/src/
> main/java/org/apache/nifi/NarMojo.java
>
>
>
> Replace line 705:
>
> narDependency = new NarDependency(artifact.getGroupId(),
> artifact.getArtifactId(), artifact.getVersion());
>
>
>
> with:
>
> narDependency = new NarDependency(artifact.getGroupId(),
> artifact.getArtifactId(), artifact.getBaseVersion());
>
>
>
> Explanation: artifact.getBaseVersion() will always consistenly return
> -SNAPSHOT, even in case the artifact was a timestamped snapshot downloaded
> from a remote repo.
>
>
>
> Thanks
>
> Arne
>
>
>
>
>