You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@nifi.apache.org by "GAQUIERE, Olivier" <ol...@capgemini.com> on 2020/06/19 08:51:17 UTC

NIFI JMS consuler : AMQP 1.0 trouble

Hi all

I try to use NIFI to consume AMQP 1.0 messages from a ActiveMQ Topic.
Did anyone manage to implement the JMS consumer to consume AMQP messages ?
If yes, which client librairies did you use ?

Thanks in advance

Regards

Olivier
This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

RE: NIFI JMS consuler : AMQP 1.0 trouble

Posted by "GAQUIERE, Olivier" <ol...@capgemini.com>.

Hi Peter

Thanks for your answer.
I tried your solution and I got the following error :

ConsumeJMS[id=c7e90ec4-0172-1000-ed11-6dd4c691e385] Failed to initialize JMS Connection Factory: org.apache.nifi.processor.exception.ProcessException: Could not obtain JMS Connection Factory from JNDI

Do you have an idea ?
Could you give me the exact configuration of your JMS consumer ?

Thanks in advance

Olivier GAQUIERE

Architect Solution Managing | CSD-Practice | Space/Public/ATM
Capgemini France - Toulouse
Tel.: +33 582 28 34 61
[cid:image001.png@01D6465E.863AA9F0]
Si vous recevez ce courriel pendant vos temps non travaillés, il ne requiert
ni prise de connaissance ni réponse immédiate de votre part.

De : Peter Turcsanyi <tu...@apache.org>
Envoyé : vendredi 19 juin 2020 14:59
À : users@nifi.apache.org
Objet : Re: NIFI JMS consuler : AMQP 1.0 trouble

Hi Olivier,

You can use Apache Qpid Proton AMQP client library [1] as recommended in the ActiveMQ AMQP configuration docs [2].
You'll want to download Qpid JMS (AMQP 1.0) and use the jars from the lib folder of the tarball.
The JNDI Initial Context Factory Class needs to be configured: org.apache.qpid.jms.jndi.JmsInitialContextFactory


Regards,
Peter

[1] https://qpid.apache.org/proton/
[2] https://activemq.apache.org/amqp

On Fri, Jun 19, 2020 at 10:51 AM GAQUIERE, Olivier <ol...@capgemini.com>> wrote:
Hi all

I try to use NIFI to consume AMQP 1.0 messages from a ActiveMQ Topic.
Did anyone manage to implement the JMS consumer to consume AMQP messages ?
If yes, which client librairies did you use ?

Thanks in advance

Regards

Olivier
This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

RE: NIFI JMS consuler : AMQP 1.0 trouble

Posted by "GAQUIERE, Olivier" <ol...@capgemini.com>.

Hi Peter

Thanks very much for your help !!!
It perfectly works.

Have a nice day

Olivier

De : Peter Turcsanyi <tu...@apache.org>
Envoyé : lundi 22 juin 2020 23:28
À : users@nifi.apache.org
Objet : Re: NIFI JMS consuler : AMQP 1.0 trouble

Olivier,

The problematic property is the JNDI Provider URL. It seems the amqp:// url cannot be specified here directly (though I believe I managed to do this earlier somehow, but I cannot reproduce it now).

The Qpid JMS documentation [1] says that you need to create a properties file (eg. jndi.properties but it can have any arbitrary name):
##
connectionfactory.ConnectionFactory = amqp://localhost:5672
##

and then reference this file in the JNDI Provider URL property via file:// url or OS filesystem path.

It works for me properly. My processor config:

JNDI Initial Context Factory Class: org.apache.qpid.jms.jndi.JmsInitialContextFactory
JNDI Provider URL: ${NIFI_HOME}/ext/qpid/jndi.properties
JNDI Name of the Connection Factory: ConnectionFactory
JNDI / JMS Client Libraries: ${NIFI_HOME}/ext/qpid/lib

Please note that you must configure the same name in the JNDI Name of the Connection Factory property as in the properties file.

Hope it will work for you too.

Best,
Peter

[1] http://qpid.apache.org/releases/qpid-jms-0.52.0/docs/index.html

On Mon, Jun 22, 2020 at 2:56 PM Mark Payne <ma...@hotmail.com>> wrote:
Olivier,

I don’t know much of anything about qpid. But I did notice that the JNDI Name that you provided is topicConnectionFactry rather than topicConnectionFactory. I.e., “Factory” is misspelled. Would recommend you double-check that.

Thanks
-Mark

On Jun 22, 2020, at 5:02 AM, GAQUIERE, Olivier <ol...@capgemini.com>> wrote:

Hi again Peter

Here is my processor configuration

<image002.png>

Olivier GAQUIERE

Architect Solution Managing | CSD-Practice | Space/Public/ATM
Capgemini France - Toulouse
Tel.: +33 582 28 34 61
<image001.png>
Si vous recevez ce courriel pendant vos temps non travaillés, il ne requiert
ni prise de connaissance ni réponse immédiate de votre part.

De : Peter Turcsanyi <tu...@apache.org>>
Envoyé : vendredi 19 juin 2020 14:59
À : users@nifi.apache.org<ma...@nifi.apache.org>
Objet : Re: NIFI JMS consuler : AMQP 1.0 trouble

Hi Olivier,

You can use Apache Qpid Proton AMQP client library [1] as recommended in the ActiveMQ AMQP configuration docs [2].
You'll want to download Qpid JMS (AMQP 1.0) and use the jars from the lib folder of the tarball.
The JNDI Initial Context Factory Class needs to be configured: org.apache.qpid.jms.jndi.JmsInitialContextFactory

Regards,
Peter

[1] https://qpid.apache.org/proton/
[2] https://activemq.apache.org/amqp

On Fri, Jun 19, 2020 at 10:51 AM GAQUIERE, Olivier <ol...@capgemini.com>> wrote:
Hi all

I try to use NIFI to consume AMQP 1.0 messages from a ActiveMQ Topic.
Did anyone manage to implement the JMS consumer to consume AMQP messages ?
If yes, which client librairies did you use ?

Thanks in advance

Regards

Olivier
This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

Re: NIFI JMS consuler : AMQP 1.0 trouble

Posted by Peter Turcsanyi <tu...@apache.org>.

Olivier,

The problematic property is the JNDI Provider URL. It seems the amqp:// url
cannot be specified here directly (though I believe I managed to do this
earlier somehow, but I cannot reproduce it now).

The Qpid JMS documentation [1] says that you need to create a properties
file (eg. jndi.properties but it can have any arbitrary name):
##
connectionfactory.ConnectionFactory = amqp://localhost:5672
##

and then reference this file in the JNDI Provider URL property via file://
url or OS filesystem path.

It works for me properly. My processor config:

JNDI Initial Context Factory
Class: org.apache.qpid.jms.jndi.JmsInitialContextFactory
JNDI Provider URL: ${NIFI_HOME}/ext/qpid/jndi.properties
JNDI Name of the Connection Factory: ConnectionFactory
JNDI / JMS Client Libraries: ${NIFI_HOME}/ext/qpid/lib

Please note that you must configure the same name in the JNDI Name of the
Connection Factory property as in the properties file.

Hope it will work for you too.

Best,
Peter

[1] http://qpid.apache.org/releases/qpid-jms-0.52.0/docs/index.html

On Mon, Jun 22, 2020 at 2:56 PM Mark Payne <ma...@hotmail.com> wrote:

> Olivier,
>
> I don’t know much of anything about qpid. But I did notice that the JNDI
> Name that you provided is topicConnectionFactry rather than
> topicConnectionFactory. I.e., “Factory” is misspelled. Would recommend you
> double-check that.
>
> Thanks
> -Mark
>
> On Jun 22, 2020, at 5:02 AM, GAQUIERE, Olivier <
> olivier.gaquiere@capgemini.com> wrote:
>
> Hi again Peter
>
> Here is my processor configuration
>
> <image002.png>
>
> *Olivier GAQUIERE*
>
> Architect Solution Managing | CSD-Practice | Space/Public/ATM
> Capgemini France - Toulouse
> Tel.: *+33 582 28 34 61*
> <image001.png>
> Si vous recevez ce courriel pendant vos temps non travaillés, il ne
> requiert
> ni prise de connaissance ni réponse immédiate de votre part.
>
> *De :* Peter Turcsanyi <tu...@apache.org>
> *Envoyé :* vendredi 19 juin 2020 14:59
> *À :* users@nifi.apache.org
> *Objet :* Re: NIFI JMS consuler : AMQP 1.0 trouble
>
> Hi Olivier,
>
> You can use Apache Qpid Proton AMQP client library [1] as recommended in
> the ActiveMQ AMQP configuration docs [2].
> You'll want to download Qpid JMS (AMQP 1.0) and use the jars from the lib
> folder of the tarball.
> The JNDI Initial Context Factory Class needs to be configured:
> org.apache.qpid.jms.jndi.JmsInitialContextFactory
>
>
> Regards,
> Peter
>
> [1] https://qpid.apache.org/proton/
> [2] https://activemq.apache.org/amqp
>
> On Fri, Jun 19, 2020 at 10:51 AM GAQUIERE, Olivier <
> olivier.gaquiere@capgemini.com> wrote:
>
> Hi all
>
> I try to use NIFI to consume AMQP 1.0 messages from a ActiveMQ Topic.
> Did anyone manage to implement the JMS consumer to consume AMQP messages ?
> If yes, which client librairies did you use ?
>
> Thanks in advance
>
> Regards
>
> Olivier
> This message contains information that may be privileged or confidential
> and is the property of the Capgemini Group. It is intended only for the
> person to whom it is addressed. If you are not the intended recipient, you
> are not authorized to read, print, retain, copy, disseminate, distribute,
> or use this message or any part thereof. If you receive this message in
> error, please notify the sender immediately and delete all copies of this
> message.
>
>
>

Re: NIFI JMS consuler : AMQP 1.0 trouble

Posted by Mark Payne <ma...@hotmail.com>.

Olivier,

Thanks
-Mark

On Jun 22, 2020, at 5:02 AM, GAQUIERE, Olivier <ol...@capgemini.com>> wrote:

Hi again Peter

Here is my processor configuration

<image002.png>

Olivier GAQUIERE

De : Peter Turcsanyi <tu...@apache.org>>
Envoyé : vendredi 19 juin 2020 14:59
À : users@nifi.apache.org<ma...@nifi.apache.org>
Objet : Re: NIFI JMS consuler : AMQP 1.0 trouble

Hi Olivier,

Regards,
Peter

[1] https://qpid.apache.org/proton/
[2] https://activemq.apache.org/amqp

On Fri, Jun 19, 2020 at 10:51 AM GAQUIERE, Olivier <ol...@capgemini.com>> wrote:
Hi all

I try to use NIFI to consume AMQP 1.0 messages from a ActiveMQ Topic.
Did anyone manage to implement the JMS consumer to consume AMQP messages ?
If yes, which client librairies did you use ?

Thanks in advance

Regards

RE: NIFI JMS consuler : AMQP 1.0 trouble

Posted by "GAQUIERE, Olivier" <ol...@capgemini.com>.

Hi again Peter

Here is my processor configuration

[cid:image002.png@01D64884.9914F9E0]

Olivier GAQUIERE

Architect Solution Managing | CSD-Practice | Space/Public/ATM
Capgemini France - Toulouse
Tel.: +33 582 28 34 61
[cid:image001.png@01D64884.6C4FA450]
Si vous recevez ce courriel pendant vos temps non travaillés, il ne requiert
ni prise de connaissance ni réponse immédiate de votre part.

De : Peter Turcsanyi <tu...@apache.org>
Envoyé : vendredi 19 juin 2020 14:59
À : users@nifi.apache.org
Objet : Re: NIFI JMS consuler : AMQP 1.0 trouble

Hi Olivier,

You can use Apache Qpid Proton AMQP client library [1] as recommended in the ActiveMQ AMQP configuration docs [2].
You'll want to download Qpid JMS (AMQP 1.0) and use the jars from the lib folder of the tarball.
The JNDI Initial Context Factory Class needs to be configured: org.apache.qpid.jms.jndi.JmsInitialContextFactory


Regards,
Peter

[1] https://qpid.apache.org/proton/
[2] https://activemq.apache.org/amqp

On Fri, Jun 19, 2020 at 10:51 AM GAQUIERE, Olivier <ol...@capgemini.com>> wrote:
Hi all

I try to use NIFI to consume AMQP 1.0 messages from a ActiveMQ Topic.
Did anyone manage to implement the JMS consumer to consume AMQP messages ?
If yes, which client librairies did you use ?

Thanks in advance

Regards

Olivier
This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

Re: NIFI JMS consuler : AMQP 1.0 trouble

Posted by Peter Turcsanyi <tu...@apache.org>.

Hi Olivier,

You can use Apache Qpid Proton AMQP client library [1] as recommended in
the ActiveMQ AMQP configuration docs [2].
You'll want to download Qpid JMS (AMQP 1.0) and use the jars from the lib
folder of the tarball.
The JNDI Initial Context Factory Class needs to be configured:
org.apache.qpid.jms.jndi.JmsInitialContextFactory


Regards,
Peter

[1] https://qpid.apache.org/proton/
[2] https://activemq.apache.org/amqp

On Fri, Jun 19, 2020 at 10:51 AM GAQUIERE, Olivier <
olivier.gaquiere@capgemini.com> wrote:

> Hi all
>
>
>
> I try to use NIFI to consume AMQP 1.0 messages from a ActiveMQ Topic.
>
> Did anyone manage to implement the JMS consumer to consume AMQP messages ?
>
> If yes, which client librairies did you use ?
>
>
>
> Thanks in advance
>
>
>
> Regards
>
>
>
> Olivier
> This message contains information that may be privileged or confidential
> and is the property of the Capgemini Group. It is intended only for the
> person to whom it is addressed. If you are not the intended recipient, you
> are not authorized to read, print, retain, copy, disseminate, distribute,
> or use this message or any part thereof. If you receive this message in
> error, please notify the sender immediately and delete all copies of this
> message.
>