You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Toivo Adams <to...@gmail.com> on 2015/05/17 12:52:15 UTC
JMS vendors
Currently only Active MQ is supported?
How other vendors JMS implementation can be used?
For example SonicMQ
JMS is fairly well standardized, so most of the send/receive code is vendor
independent.
But often connection factories have more or less vendor specific parts.
And of course JMS client libraries are vendor specific, sometimes 4-6
proprietary jar files.
Some of them are not open source.
Thanks
Toivo
--
View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/JMS-vendors-tp1558.html
Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
Re: JMS vendors
Posted by Bryan Bende <bb...@gmail.com>.
I know it is not part of this JMS solution, but +1 for pulling BinFiles
into the api nar.
On Mon, May 18, 2015 at 11:34 AM, Mark Payne <ma...@hotmail.com> wrote:
> Hmm, I like where your head is at, Adam.
>
> If we use a Controller Service to provide the ConnectionFactory, then each
> Controller Service can be configured with its own properties, which could
> be specific to the vendor. The JMS Providers would be discovered
> automatically by the framework because they are just Controller Services.
>
> From the perspective of the PropertyDescriptor that exposes the provider,
> it's as simple as adding:
> ".identifiesControllerService(JmsConnectionFactoryService.class)" and that
> will automatically populate with the appropriate services. And with 0.1.0
> out, the services are all configured in the UI.
>
> I like it.
>
> -Mark
>
> ----------------------------------------
> > Date: Mon, 18 May 2015 11:21:36 -0400
> > Subject: Re: JMS vendors
> > From: adam@adamtaft.com
> > To: dev@nifi.incubator.apache.org
> >
> > Just a suggestion and a nod to Joshua Bloch's Item 16, Favor Composition
> > over Inheritance.
> >
> > This might arguably be a better design if the processor didn't have to be
> > extended at all. Instead, let the JMS processor pull in a
> > ConnectionFactory via the Java ServiceLoader or from a NiFi service. We
> > would provide a standard interface which the processor would use to find
> > valid implementations. Vendor specific JMS setup could be isolated as
> > individual jars or service nars and loaded when needed.
> >
> > To make this really nice, the processor would provide a
> PropertyDescriptor
> > with a set of allowable values coming from the service loader. When a
> user
> > configures the JMS, he would be able to have a dropdown of available JMS
> > providers, such as ActiveMQ, SonicMQ, Tibco, etc. (similar to SSL Context
> > Service configuration).
> >
> > In general, I think it would be much more powerful to create final
> > processors of certain families and let them defer to a pluggable lookup
> > service rather than having to extend an abstract processor.
> >
> > Adam
> >
> >
> > On Mon, May 18, 2015 at 11:01 AM, Mark Payne <ma...@hotmail.com>
> wrote:
> >
> >> Mike,
> >>
> >> I remember a while back a handful of us discussing renaming
> >> nifi-standard-services-api-nar to something more generic like
> >> nifi-standard-component-api-nar or something like that, specifically for
> >> this reason. I'd be very hesitant to ask people to inherit
> >> nifi-standard-nar, just because there are a lot of dependencies that
> would
> >> be pulled in, so conflict resolution could potentially get hairy.
> >>
> >> We should also probably pull BinFiles into the api nar, and refactor
> >> MergeContent to have an AbstractMergeContent processor. This allows a
> >> developer to very easily extend AbstractMergeContent and provide their
> own
> >> Merge Formats, without dealing with the complexity of the merging code.
> But
> >> these are separate tickets for sure.
> >>
> >>
> >>
> >> ----------------------------------------
> >>> Date: Mon, 18 May 2015 10:51:13 -0400
> >>> Subject: Re: JMS vendors
> >>> From: moser.mw@gmail.com
> >>> To: dev@nifi.incubator.apache.org
> >>>
> >>> There is no JIRA ticket yet. Perhaps we can agree on an approach before
> >>> writing the ticket.
> >>>
> >>> Mark, I was thinking the custom nar would inherit from the
> >>> nifi-standard-nar instead of the standard-services-api-nar. A processor
> >>> isn't a service, so I didn't even think of standard-services-api-nar.
> >> Then
> >>> just "class MyPutJMS extends PutJMS" inside my custom nar and go.
> >>>
> >>> I'm sure there are all kinds of approaches we could take here, so if
> the
> >>> community wants to implement something fancier, that's cool.
> >>>
> >>> -- Mike
> >>>
> >>>
> >>> On Mon, May 18, 2015 at 10:20 AM, Mark Payne <ma...@hotmail.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> I absolutely agree. I was conceptualizing it working a bit differently
> >> in
> >>>> my head, but I think the solution that you laid out here is better
> than
> >>>> what I was picturing. Excellent.
> >>>>
> >>>> We will have to be a bit careful here, though. We will need the
> abstract
> >>>> processor to be pulled into the standard-services-api-nar, I think, so
> >> that
> >>>> others can easily extend it. But we don't want to pull any libraries
> >> into
> >>>> that nar that aren't absolutely critical, because a lot of nars will
> be
> >>>> pulling that one in as the parent.
> >>>>
> >>>> Thanks
> >>>> -Mark
> >>>>
> >>>>
> >>>> ----------------------------------------
> >>>>> Date: Mon, 18 May 2015 10:01:41 -0400
> >>>>> Subject: Re: JMS vendors
> >>>>> From: moser.mw@gmail.com
> >>>>> To: dev@nifi.incubator.apache.org
> >>>>>
> >>>>> Having extended GetJMS*/PutJMS* myself to use another JMS vendor, I
> >> think
> >>>>> we can make this much easier than it currently is.
> >>>>>
> >>>>> Right now, the provider's ConnectionFactory creation is buried in the
> >>>> NiFi
> >>>>> JmsFactory class which the GetJMS*/PutJMS* processors use. This
> >>>> JmsFactory
> >>>>> class doesn't implement an interface or extend a base class, so it
> >> cannot
> >>>>> be easily replaced or extended. I ended up copying it and the
> >>>>> GetJMS*/PutJMS* source into my own project in order to extend them.
> Not
> >>>>> ideal.
> >>>>>
> >>>>> If we can move provider
> >>
> >> ConnectionFactory creation into the processors
> >>>>> themselves, then doing this would be as simple as extending the
> >>>>> GetJMS*/PutJMS* processors and overriding the ConnectionFactory
> >> creation
> >>>>> method.
> >>>>>
> >>>>> This is on my list of things that I wanted to work on and contribute,
> >> but
> >>>>> haven't allocated time for.
> >>>>>
> >>>>> -- Mike
> >>>>>
> >>>>>
> >>>>> On Mon, May 18, 2015 at 6:35 AM, Toivo Adams <to...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>>> Joe,
> >>>>>>
> >>>>>> You are right, creating special GetJms*/PutJms* version for other
> >>>> vendor is
> >>>>>> not difficult.
> >>>>>> And yes more specifically I have interested to have only SonicMQ
> >> support
> >>>>>> because its used heavily.
> >>>>>>
> >>>>>> I was hoping to have some general solution which helps more easily
> to
> >>>> start
> >>>>>> using any JMS vendor.
> >>>>>>
> >>>>>> When each user create its own GetJms*/PutJms* version, such version
> is
> >>>> much
> >>>>>> less battle tested than Nifi standard processors (much smaller user
> >>>> base).
> >>>>>>
> >>>>>> But creating some kind of general JMS solution requires much more
> >> effort
> >>>>>> than separate version for one vendor.
> >>>>>> So it's boils down how big is the demand.
> >>>>>> It seems there are not (yet?) many different JMS vendor users in
> >>>> NiFiland?
> >>>>>>
> >>>>>>
> >>>>>> Thanks
> >>>>>> Toivo
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> View this message in context:
> >>>>>>
> >>>>
> >>
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/JMS-vendors-tp1558p1567.html
> >>>>>> Sent from the Apache NiFi (incubating) Developer List mailing list
> >>>> archive
> >>>>>> at Nabble.com.
> >>>>>>
> >>>>
> >>>>
> >>
> >>
>
>
Re: JMS vendors
Posted by Toivo Adams <to...@gmail.com>.
Michael Moser
I am glad to heard you have already thought how to have multi vendor JMS
support.
And you right, even fancier solution is always possible.
But question remains – who has time to implement it?
For example.
I have used many years ago product called HermesJMS
http://www.hermesjms.com/confluence/display/HJMS/Home
As far I remember Hermes was capable to support any JMS vendors.
No custom code was required. Only some configuration and vendor client
libraries must be made available.
I don't how this was possible and I am not sure is it feasible in NiFiland.
Thanks
toivo
--
View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/JMS-vendors-tp1558p1583.html
Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
Re: JMS vendors
Posted by Michael Moser <mo...@gmail.com>.
I also love the JmsConnectionFactoryService idea, so I created the JIRA
ticket NIFI-614 [1].
-- Mike
[1] https://issues.apache.org/jira/browse/NIFI-614
On Mon, May 18, 2015 at 11:34 AM, Mark Payne <ma...@hotmail.com> wrote:
> Hmm, I like where your head is at, Adam.
>
> If we use a Controller Service to provide the ConnectionFactory, then each
> Controller Service can be configured with its own properties, which could
> be specific to the vendor. The JMS Providers would be discovered
> automatically by the framework because they are just Controller Services.
>
> From the perspective of the PropertyDescriptor that exposes the provider,
> it's as simple as adding:
> ".identifiesControllerService(JmsConnectionFactoryService.class)" and that
> will automatically populate with the appropriate services. And with 0.1.0
> out, the services are all configured in the UI.
>
> I like it.
>
> -Mark
>
> ----------------------------------------
> > Date: Mon, 18 May 2015 11:21:36 -0400
> > Subject: Re: JMS vendors
> > From: adam@adamtaft.com
> > To: dev@nifi.incubator.apache.org
> >
> > Just a suggestion and a nod to Joshua Bloch's Item 16, Favor Composition
> > over Inheritance.
> >
> > This might arguably be a better design if the processor didn't have to be
> > extended at all. Instead, let the JMS processor pull in a
> > ConnectionFactory via the Java ServiceLoader or from a NiFi service. We
> > would provide a standard interface which the processor would use to find
> > valid implementations. Vendor specific JMS setup could be isolated as
> > individual jars or service nars and loaded when needed.
> >
> > To make this really nice, the processor would provide a
> PropertyDescriptor
> > with a set of allowable values coming from the service loader. When a
> user
> > configures the JMS, he would be able to have a dropdown of available JMS
> > providers, such as ActiveMQ, SonicMQ, Tibco, etc. (similar to SSL Context
> > Service configuration).
> >
> > In general, I think it would be much more powerful to create final
> > processors of certain families and let them defer to a pluggable lookup
> > service rather than having to extend an abstract processor.
> >
> > Adam
> >
> >
> > On Mon, May 18, 2015 at 11:01 AM, Mark Payne <ma...@hotmail.com>
> wrote:
> >
> >> Mike,
> >>
> >> I remember a while back a handful of us discussing renaming
> >> nifi-standard-services-api-nar to something more generic like
> >> nifi-standard-component-api-nar or something like that, specifically for
> >> this reason. I'd be very hesitant to ask people to inherit
> >> nifi-standard-nar, just because there are a lot of dependencies that
> would
> >> be pulled in, so conflict resolution could potentially get hairy.
> >>
> >> We should also probably pull BinFiles into the api nar, and refactor
> >> MergeContent to have an AbstractMergeContent processor. This allows a
> >> developer to very easily extend AbstractMergeContent and provide their
> own
> >> Merge Formats, without dealing with the complexity of the merging code.
> But
> >> these are separate tickets for sure.
> >>
> >>
> >>
> >> ----------------------------------------
> >>> Date: Mon, 18 May 2015 10:51:13 -0400
> >>> Subject: Re: JMS vendors
> >>> From: moser.mw@gmail.com
> >>> To: dev@nifi.incubator.apache.org
> >>>
> >>> There is no JIRA ticket yet. Perhaps we can agree on an approach before
> >>> writing the ticket.
> >>>
> >>> Mark, I was thinking the custom nar would inherit from the
> >>> nifi-standard-nar instead of the standard-services-api-nar. A processor
> >>> isn't a service, so I didn't even think of standard-services-api-nar.
> >> Then
> >>> just "class MyPutJMS extends PutJMS" inside my custom nar and go.
> >>>
> >>> I'm sure there are all kinds of approaches we could take here, so if
> the
> >>> community wants to implement something fancier, that's cool.
> >>>
> >>> -- Mike
> >>>
> >>>
> >>> On Mon, May 18, 2015 at 10:20 AM, Mark Payne <ma...@hotmail.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> I absolutely agree. I was conceptualizing it working a bit differently
> >> in
> >>>> my head, but I think the solution that you laid out here is better
> than
> >>>> what I was picturing. Excellent.
> >>>>
> >>>> We will have to be a bit careful here, though. We will need the
> abstract
> >>>> processor to be pulled into the standard-services-api-nar, I think, so
> >> that
> >>>> others can easily extend it. But we don't want to pull any libraries
> >> into
> >>>> that nar that aren't absolutely critical, because a lot of nars will
> be
> >>>> pulling that one in as the parent.
> >>>>
> >>>> Thanks
> >>>> -Mark
> >>>>
> >>>>
> >>>> ----------------------------------------
> >>>>> Date: Mon, 18 May 2015 10:01:41 -0400
> >>>>> Subject: Re: JMS vendors
> >>>>> From: moser.mw@gmail.com
> >>>>> To: dev@nifi.incubator.apache.org
> >>>>>
> >>>>> Having extended GetJMS*/PutJMS* myself to use another JMS vendor, I
> >> think
> >>>>> we can make this much easier than it currently is.
> >>>>>
> >>>>> Right now, the provider's ConnectionFactory creation is buried in the
> >>>> NiFi
> >>>>> JmsFactory class which the GetJMS*/PutJMS* processors use. This
> >>>> JmsFactory
> >>>>> class doesn't implement an interface or extend a base class, so it
> >> cannot
> >>>>> be easily replaced or extended. I ended up copying it and the
> >>>>> GetJMS*/PutJMS* source into my own project in order to extend them.
> Not
> >>>>> ideal.
> >>>>>
> >>>>> If we can move provider
> >>
> >> ConnectionFactory creation into the processors
> >>>>> themselves, then doing this would be as simple as extending the
> >>>>> GetJMS*/PutJMS* processors and overriding the ConnectionFactory
> >> creation
> >>>>> method.
> >>>>>
> >>>>> This is on my list of things that I wanted to work on and contribute,
> >> but
> >>>>> haven't allocated time for.
> >>>>>
> >>>>> -- Mike
> >>>>>
> >>>>>
> >>>>> On Mon, May 18, 2015 at 6:35 AM, Toivo Adams <to...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>>> Joe,
> >>>>>>
> >>>>>> You are right, creating special GetJms*/PutJms* version for other
> >>>> vendor is
> >>>>>> not difficult.
> >>>>>> And yes more specifically I have interested to have only SonicMQ
> >> support
> >>>>>> because its used heavily.
> >>>>>>
> >>>>>> I was hoping to have some general solution which helps more easily
> to
> >>>> start
> >>>>>> using any JMS vendor.
> >>>>>>
> >>>>>> When each user create its own GetJms*/PutJms* version, such version
> is
> >>>> much
> >>>>>> less battle tested than Nifi standard processors (much smaller user
> >>>> base).
> >>>>>>
> >>>>>> But creating some kind of general JMS solution requires much more
> >> effort
> >>>>>> than separate version for one vendor.
> >>>>>> So it's boils down how big is the demand.
> >>>>>> It seems there are not (yet?) many different JMS vendor users in
> >>>> NiFiland?
> >>>>>>
> >>>>>>
> >>>>>> Thanks
> >>>>>> Toivo
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> View this message in context:
> >>>>>>
> >>>>
> >>
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/JMS-vendors-tp1558p1567.html
> >>>>>> Sent from the Apache NiFi (incubating) Developer List mailing list
> >>>> archive
> >>>>>> at Nabble.com.
> >>>>>>
> >>>>
> >>>>
> >>
> >>
>
>
RE: JMS vendors
Posted by Mark Payne <ma...@hotmail.com>.
Hmm, I like where your head is at, Adam.
If we use a Controller Service to provide the ConnectionFactory, then each Controller Service can be configured with its own properties, which could be specific to the vendor. The JMS Providers would be discovered automatically by the framework because they are just Controller Services.
From the perspective of the PropertyDescriptor that exposes the provider, it's as simple as adding: ".identifiesControllerService(JmsConnectionFactoryService.class)" and that will automatically populate with the appropriate services. And with 0.1.0 out, the services are all configured in the UI.
I like it.
-Mark
----------------------------------------
> Date: Mon, 18 May 2015 11:21:36 -0400
> Subject: Re: JMS vendors
> From: adam@adamtaft.com
> To: dev@nifi.incubator.apache.org
>
> Just a suggestion and a nod to Joshua Bloch's Item 16, Favor Composition
> over Inheritance.
>
> This might arguably be a better design if the processor didn't have to be
> extended at all. Instead, let the JMS processor pull in a
> ConnectionFactory via the Java ServiceLoader or from a NiFi service. We
> would provide a standard interface which the processor would use to find
> valid implementations. Vendor specific JMS setup could be isolated as
> individual jars or service nars and loaded when needed.
>
> To make this really nice, the processor would provide a PropertyDescriptor
> with a set of allowable values coming from the service loader. When a user
> configures the JMS, he would be able to have a dropdown of available JMS
> providers, such as ActiveMQ, SonicMQ, Tibco, etc. (similar to SSL Context
> Service configuration).
>
> In general, I think it would be much more powerful to create final
> processors of certain families and let them defer to a pluggable lookup
> service rather than having to extend an abstract processor.
>
> Adam
>
>
> On Mon, May 18, 2015 at 11:01 AM, Mark Payne <ma...@hotmail.com> wrote:
>
>> Mike,
>>
>> I remember a while back a handful of us discussing renaming
>> nifi-standard-services-api-nar to something more generic like
>> nifi-standard-component-api-nar or something like that, specifically for
>> this reason. I'd be very hesitant to ask people to inherit
>> nifi-standard-nar, just because there are a lot of dependencies that would
>> be pulled in, so conflict resolution could potentially get hairy.
>>
>> We should also probably pull BinFiles into the api nar, and refactor
>> MergeContent to have an AbstractMergeContent processor. This allows a
>> developer to very easily extend AbstractMergeContent and provide their own
>> Merge Formats, without dealing with the complexity of the merging code. But
>> these are separate tickets for sure.
>>
>>
>>
>> ----------------------------------------
>>> Date: Mon, 18 May 2015 10:51:13 -0400
>>> Subject: Re: JMS vendors
>>> From: moser.mw@gmail.com
>>> To: dev@nifi.incubator.apache.org
>>>
>>> There is no JIRA ticket yet. Perhaps we can agree on an approach before
>>> writing the ticket.
>>>
>>> Mark, I was thinking the custom nar would inherit from the
>>> nifi-standard-nar instead of the standard-services-api-nar. A processor
>>> isn't a service, so I didn't even think of standard-services-api-nar.
>> Then
>>> just "class MyPutJMS extends PutJMS" inside my custom nar and go.
>>>
>>> I'm sure there are all kinds of approaches we could take here, so if the
>>> community wants to implement something fancier, that's cool.
>>>
>>> -- Mike
>>>
>>>
>>> On Mon, May 18, 2015 at 10:20 AM, Mark Payne <ma...@hotmail.com>
>> wrote:
>>>
>>>> Mike,
>>>>
>>>> I absolutely agree. I was conceptualizing it working a bit differently
>> in
>>>> my head, but I think the solution that you laid out here is better than
>>>> what I was picturing. Excellent.
>>>>
>>>> We will have to be a bit careful here, though. We will need the abstract
>>>> processor to be pulled into the standard-services-api-nar, I think, so
>> that
>>>> others can easily extend it. But we don't want to pull any libraries
>> into
>>>> that nar that aren't absolutely critical, because a lot of nars will be
>>>> pulling that one in as the parent.
>>>>
>>>> Thanks
>>>> -Mark
>>>>
>>>>
>>>> ----------------------------------------
>>>>> Date: Mon, 18 May 2015 10:01:41 -0400
>>>>> Subject: Re: JMS vendors
>>>>> From: moser.mw@gmail.com
>>>>> To: dev@nifi.incubator.apache.org
>>>>>
>>>>> Having extended GetJMS*/PutJMS* myself to use another JMS vendor, I
>> think
>>>>> we can make this much easier than it currently is.
>>>>>
>>>>> Right now, the provider's ConnectionFactory creation is buried in the
>>>> NiFi
>>>>> JmsFactory class which the GetJMS*/PutJMS* processors use. This
>>>> JmsFactory
>>>>> class doesn't implement an interface or extend a base class, so it
>> cannot
>>>>> be easily replaced or extended. I ended up copying it and the
>>>>> GetJMS*/PutJMS* source into my own project in order to extend them. Not
>>>>> ideal.
>>>>>
>>>>> If we can move provider
>>
>> ConnectionFactory creation into the processors
>>>>> themselves, then doing this would be as simple as extending the
>>>>> GetJMS*/PutJMS* processors and overriding the ConnectionFactory
>> creation
>>>>> method.
>>>>>
>>>>> This is on my list of things that I wanted to work on and contribute,
>> but
>>>>> haven't allocated time for.
>>>>>
>>>>> -- Mike
>>>>>
>>>>>
>>>>> On Mon, May 18, 2015 at 6:35 AM, Toivo Adams <to...@gmail.com>
>>>> wrote:
>>>>>
>>>>>> Joe,
>>>>>>
>>>>>> You are right, creating special GetJms*/PutJms* version for other
>>>> vendor is
>>>>>> not difficult.
>>>>>> And yes more specifically I have interested to have only SonicMQ
>> support
>>>>>> because its used heavily.
>>>>>>
>>>>>> I was hoping to have some general solution which helps more easily to
>>>> start
>>>>>> using any JMS vendor.
>>>>>>
>>>>>> When each user create its own GetJms*/PutJms* version, such version is
>>>> much
>>>>>> less battle tested than Nifi standard processors (much smaller user
>>>> base).
>>>>>>
>>>>>> But creating some kind of general JMS solution requires much more
>> effort
>>>>>> than separate version for one vendor.
>>>>>> So it's boils down how big is the demand.
>>>>>> It seems there are not (yet?) many different JMS vendor users in
>>>> NiFiland?
>>>>>>
>>>>>>
>>>>>> Thanks
>>>>>> Toivo
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>>
>>>>
>> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/JMS-vendors-tp1558p1567.html
>>>>>> Sent from the Apache NiFi (incubating) Developer List mailing list
>>>> archive
>>>>>> at Nabble.com.
>>>>>>
>>>>
>>>>
>>
>>
Re: JMS vendors
Posted by Adam Taft <ad...@adamtaft.com>.
Just a suggestion and a nod to Joshua Bloch's Item 16, Favor Composition
over Inheritance.
This might arguably be a better design if the processor didn't have to be
extended at all. Instead, let the JMS processor pull in a
ConnectionFactory via the Java ServiceLoader or from a NiFi service. We
would provide a standard interface which the processor would use to find
valid implementations. Vendor specific JMS setup could be isolated as
individual jars or service nars and loaded when needed.
To make this really nice, the processor would provide a PropertyDescriptor
with a set of allowable values coming from the service loader. When a user
configures the JMS, he would be able to have a dropdown of available JMS
providers, such as ActiveMQ, SonicMQ, Tibco, etc. (similar to SSL Context
Service configuration).
In general, I think it would be much more powerful to create final
processors of certain families and let them defer to a pluggable lookup
service rather than having to extend an abstract processor.
Adam
On Mon, May 18, 2015 at 11:01 AM, Mark Payne <ma...@hotmail.com> wrote:
> Mike,
>
> I remember a while back a handful of us discussing renaming
> nifi-standard-services-api-nar to something more generic like
> nifi-standard-component-api-nar or something like that, specifically for
> this reason. I'd be very hesitant to ask people to inherit
> nifi-standard-nar, just because there are a lot of dependencies that would
> be pulled in, so conflict resolution could potentially get hairy.
>
> We should also probably pull BinFiles into the api nar, and refactor
> MergeContent to have an AbstractMergeContent processor. This allows a
> developer to very easily extend AbstractMergeContent and provide their own
> Merge Formats, without dealing with the complexity of the merging code. But
> these are separate tickets for sure.
>
>
>
> ----------------------------------------
> > Date: Mon, 18 May 2015 10:51:13 -0400
> > Subject: Re: JMS vendors
> > From: moser.mw@gmail.com
> > To: dev@nifi.incubator.apache.org
> >
> > There is no JIRA ticket yet. Perhaps we can agree on an approach before
> > writing the ticket.
> >
> > Mark, I was thinking the custom nar would inherit from the
> > nifi-standard-nar instead of the standard-services-api-nar. A processor
> > isn't a service, so I didn't even think of standard-services-api-nar.
> Then
> > just "class MyPutJMS extends PutJMS" inside my custom nar and go.
> >
> > I'm sure there are all kinds of approaches we could take here, so if the
> > community wants to implement something fancier, that's cool.
> >
> > -- Mike
> >
> >
> > On Mon, May 18, 2015 at 10:20 AM, Mark Payne <ma...@hotmail.com>
> wrote:
> >
> >> Mike,
> >>
> >> I absolutely agree. I was conceptualizing it working a bit differently
> in
> >> my head, but I think the solution that you laid out here is better than
> >> what I was picturing. Excellent.
> >>
> >> We will have to be a bit careful here, though. We will need the abstract
> >> processor to be pulled into the standard-services-api-nar, I think, so
> that
> >> others can easily extend it. But we don't want to pull any libraries
> into
> >> that nar that aren't absolutely critical, because a lot of nars will be
> >> pulling that one in as the parent.
> >>
> >> Thanks
> >> -Mark
> >>
> >>
> >> ----------------------------------------
> >>> Date: Mon, 18 May 2015 10:01:41 -0400
> >>> Subject: Re: JMS vendors
> >>> From: moser.mw@gmail.com
> >>> To: dev@nifi.incubator.apache.org
> >>>
> >>> Having extended GetJMS*/PutJMS* myself to use another JMS vendor, I
> think
> >>> we can make this much easier than it currently is.
> >>>
> >>> Right now, the provider's ConnectionFactory creation is buried in the
> >> NiFi
> >>> JmsFactory class which the GetJMS*/PutJMS* processors use. This
> >> JmsFactory
> >>> class doesn't implement an interface or extend a base class, so it
> cannot
> >>> be easily replaced or extended. I ended up copying it and the
> >>> GetJMS*/PutJMS* source into my own project in order to extend them. Not
> >>> ideal.
> >>>
> >>> If we can move provider
>
> ConnectionFactory creation into the processors
> >>> themselves, then doing this would be as simple as extending the
> >>> GetJMS*/PutJMS* processors and overriding the ConnectionFactory
> creation
> >>> method.
> >>>
> >>> This is on my list of things that I wanted to work on and contribute,
> but
> >>> haven't allocated time for.
> >>>
> >>> -- Mike
> >>>
> >>>
> >>> On Mon, May 18, 2015 at 6:35 AM, Toivo Adams <to...@gmail.com>
> >> wrote:
> >>>
> >>>> Joe,
> >>>>
> >>>> You are right, creating special GetJms*/PutJms* version for other
> >> vendor is
> >>>> not difficult.
> >>>> And yes more specifically I have interested to have only SonicMQ
> support
> >>>> because its used heavily.
> >>>>
> >>>> I was hoping to have some general solution which helps more easily to
> >> start
> >>>> using any JMS vendor.
> >>>>
> >>>> When each user create its own GetJms*/PutJms* version, such version is
> >> much
> >>>> less battle tested than Nifi standard processors (much smaller user
> >> base).
> >>>>
> >>>> But creating some kind of general JMS solution requires much more
> effort
> >>>> than separate version for one vendor.
> >>>> So it's boils down how big is the demand.
> >>>> It seems there are not (yet?) many different JMS vendor users in
> >> NiFiland?
> >>>>
> >>>>
> >>>> Thanks
> >>>> Toivo
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> View this message in context:
> >>>>
> >>
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/JMS-vendors-tp1558p1567.html
> >>>> Sent from the Apache NiFi (incubating) Developer List mailing list
> >> archive
> >>>> at Nabble.com.
> >>>>
> >>
> >>
>
>
RE: JMS vendors
Posted by Mark Payne <ma...@hotmail.com>.
Mike,
I remember a while back a handful of us discussing renaming nifi-standard-services-api-nar to something more generic like nifi-standard-component-api-nar or something like that, specifically for this reason. I'd be very hesitant to ask people to inherit nifi-standard-nar, just because there are a lot of dependencies that would be pulled in, so conflict resolution could potentially get hairy.
We should also probably pull BinFiles into the api nar, and refactor MergeContent to have an AbstractMergeContent processor. This allows a developer to very easily extend AbstractMergeContent and provide their own Merge Formats, without dealing with the complexity of the merging code. But these are separate tickets for sure.
----------------------------------------
> Date: Mon, 18 May 2015 10:51:13 -0400
> Subject: Re: JMS vendors
> From: moser.mw@gmail.com
> To: dev@nifi.incubator.apache.org
>
> There is no JIRA ticket yet. Perhaps we can agree on an approach before
> writing the ticket.
>
> Mark, I was thinking the custom nar would inherit from the
> nifi-standard-nar instead of the standard-services-api-nar. A processor
> isn't a service, so I didn't even think of standard-services-api-nar. Then
> just "class MyPutJMS extends PutJMS" inside my custom nar and go.
>
> I'm sure there are all kinds of approaches we could take here, so if the
> community wants to implement something fancier, that's cool.
>
> -- Mike
>
>
> On Mon, May 18, 2015 at 10:20 AM, Mark Payne <ma...@hotmail.com> wrote:
>
>> Mike,
>>
>> I absolutely agree. I was conceptualizing it working a bit differently in
>> my head, but I think the solution that you laid out here is better than
>> what I was picturing. Excellent.
>>
>> We will have to be a bit careful here, though. We will need the abstract
>> processor to be pulled into the standard-services-api-nar, I think, so that
>> others can easily extend it. But we don't want to pull any libraries into
>> that nar that aren't absolutely critical, because a lot of nars will be
>> pulling that one in as the parent.
>>
>> Thanks
>> -Mark
>>
>>
>> ----------------------------------------
>>> Date: Mon, 18 May 2015 10:01:41 -0400
>>> Subject: Re: JMS vendors
>>> From: moser.mw@gmail.com
>>> To: dev@nifi.incubator.apache.org
>>>
>>> Having extended GetJMS*/PutJMS* myself to use another JMS vendor, I think
>>> we can make this much easier than it currently is.
>>>
>>> Right now, the provider's ConnectionFactory creation is buried in the
>> NiFi
>>> JmsFactory class which the GetJMS*/PutJMS* processors use. This
>> JmsFactory
>>> class doesn't implement an interface or extend a base class, so it cannot
>>> be easily replaced or extended. I ended up copying it and the
>>> GetJMS*/PutJMS* source into my own project in order to extend them. Not
>>> ideal.
>>>
>>> If we can move provider ConnectionFactory creation into the processors
>>> themselves, then doing this would be as simple as extending the
>>> GetJMS*/PutJMS* processors and overriding the ConnectionFactory creation
>>> method.
>>>
>>> This is on my list of things that I wanted to work on and contribute, but
>>> haven't allocated time for.
>>>
>>> -- Mike
>>>
>>>
>>> On Mon, May 18, 2015 at 6:35 AM, Toivo Adams <to...@gmail.com>
>> wrote:
>>>
>>>> Joe,
>>>>
>>>> You are right, creating special GetJms*/PutJms* version for other
>> vendor is
>>>> not difficult.
>>>> And yes more specifically I have interested to have only SonicMQ support
>>>> because its used heavily.
>>>>
>>>> I was hoping to have some general solution which helps more easily to
>> start
>>>> using any JMS vendor.
>>>>
>>>> When each user create its own GetJms*/PutJms* version, such version is
>> much
>>>> less battle tested than Nifi standard processors (much smaller user
>> base).
>>>>
>>>> But creating some kind of general JMS solution requires much more effort
>>>> than separate version for one vendor.
>>>> So it's boils down how big is the demand.
>>>> It seems there are not (yet?) many different JMS vendor users in
>> NiFiland?
>>>>
>>>>
>>>> Thanks
>>>> Toivo
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>>
>> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/JMS-vendors-tp1558p1567.html
>>>> Sent from the Apache NiFi (incubating) Developer List mailing list
>> archive
>>>> at Nabble.com.
>>>>
>>
>>
Re: JMS vendors
Posted by Michael Moser <mo...@gmail.com>.
There is no JIRA ticket yet. Perhaps we can agree on an approach before
writing the ticket.
Mark, I was thinking the custom nar would inherit from the
nifi-standard-nar instead of the standard-services-api-nar. A processor
isn't a service, so I didn't even think of standard-services-api-nar. Then
just "class MyPutJMS extends PutJMS" inside my custom nar and go.
I'm sure there are all kinds of approaches we could take here, so if the
community wants to implement something fancier, that's cool.
-- Mike
On Mon, May 18, 2015 at 10:20 AM, Mark Payne <ma...@hotmail.com> wrote:
> Mike,
>
> I absolutely agree. I was conceptualizing it working a bit differently in
> my head, but I think the solution that you laid out here is better than
> what I was picturing. Excellent.
>
> We will have to be a bit careful here, though. We will need the abstract
> processor to be pulled into the standard-services-api-nar, I think, so that
> others can easily extend it. But we don't want to pull any libraries into
> that nar that aren't absolutely critical, because a lot of nars will be
> pulling that one in as the parent.
>
> Thanks
> -Mark
>
>
> ----------------------------------------
> > Date: Mon, 18 May 2015 10:01:41 -0400
> > Subject: Re: JMS vendors
> > From: moser.mw@gmail.com
> > To: dev@nifi.incubator.apache.org
> >
> > Having extended GetJMS*/PutJMS* myself to use another JMS vendor, I think
> > we can make this much easier than it currently is.
> >
> > Right now, the provider's ConnectionFactory creation is buried in the
> NiFi
> > JmsFactory class which the GetJMS*/PutJMS* processors use. This
> JmsFactory
> > class doesn't implement an interface or extend a base class, so it cannot
> > be easily replaced or extended. I ended up copying it and the
> > GetJMS*/PutJMS* source into my own project in order to extend them. Not
> > ideal.
> >
> > If we can move provider ConnectionFactory creation into the processors
> > themselves, then doing this would be as simple as extending the
> > GetJMS*/PutJMS* processors and overriding the ConnectionFactory creation
> > method.
> >
> > This is on my list of things that I wanted to work on and contribute, but
> > haven't allocated time for.
> >
> > -- Mike
> >
> >
> > On Mon, May 18, 2015 at 6:35 AM, Toivo Adams <to...@gmail.com>
> wrote:
> >
> >> Joe,
> >>
> >> You are right, creating special GetJms*/PutJms* version for other
> vendor is
> >> not difficult.
> >> And yes more specifically I have interested to have only SonicMQ support
> >> because its used heavily.
> >>
> >> I was hoping to have some general solution which helps more easily to
> start
> >> using any JMS vendor.
> >>
> >> When each user create its own GetJms*/PutJms* version, such version is
> much
> >> less battle tested than Nifi standard processors (much smaller user
> base).
> >>
> >> But creating some kind of general JMS solution requires much more effort
> >> than separate version for one vendor.
> >> So it's boils down how big is the demand.
> >> It seems there are not (yet?) many different JMS vendor users in
> NiFiland?
> >>
> >>
> >> Thanks
> >> Toivo
> >>
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >>
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/JMS-vendors-tp1558p1567.html
> >> Sent from the Apache NiFi (incubating) Developer List mailing list
> archive
> >> at Nabble.com.
> >>
>
>
RE: JMS vendors
Posted by Mark Payne <ma...@hotmail.com>.
Mike,
I absolutely agree. I was conceptualizing it working a bit differently in my head, but I think the solution that you laid out here is better than what I was picturing. Excellent.
We will have to be a bit careful here, though. We will need the abstract processor to be pulled into the standard-services-api-nar, I think, so that others can easily extend it. But we don't want to pull any libraries into that nar that aren't absolutely critical, because a lot of nars will be pulling that one in as the parent.
Thanks
-Mark
----------------------------------------
> Date: Mon, 18 May 2015 10:01:41 -0400
> Subject: Re: JMS vendors
> From: moser.mw@gmail.com
> To: dev@nifi.incubator.apache.org
>
> Having extended GetJMS*/PutJMS* myself to use another JMS vendor, I think
> we can make this much easier than it currently is.
>
> Right now, the provider's ConnectionFactory creation is buried in the NiFi
> JmsFactory class which the GetJMS*/PutJMS* processors use. This JmsFactory
> class doesn't implement an interface or extend a base class, so it cannot
> be easily replaced or extended. I ended up copying it and the
> GetJMS*/PutJMS* source into my own project in order to extend them. Not
> ideal.
>
> If we can move provider ConnectionFactory creation into the processors
> themselves, then doing this would be as simple as extending the
> GetJMS*/PutJMS* processors and overriding the ConnectionFactory creation
> method.
>
> This is on my list of things that I wanted to work on and contribute, but
> haven't allocated time for.
>
> -- Mike
>
>
> On Mon, May 18, 2015 at 6:35 AM, Toivo Adams <to...@gmail.com> wrote:
>
>> Joe,
>>
>> You are right, creating special GetJms*/PutJms* version for other vendor is
>> not difficult.
>> And yes more specifically I have interested to have only SonicMQ support
>> because its used heavily.
>>
>> I was hoping to have some general solution which helps more easily to start
>> using any JMS vendor.
>>
>> When each user create its own GetJms*/PutJms* version, such version is much
>> less battle tested than Nifi standard processors (much smaller user base).
>>
>> But creating some kind of general JMS solution requires much more effort
>> than separate version for one vendor.
>> So it's boils down how big is the demand.
>> It seems there are not (yet?) many different JMS vendor users in NiFiland?
>>
>>
>> Thanks
>> Toivo
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/JMS-vendors-tp1558p1567.html
>> Sent from the Apache NiFi (incubating) Developer List mailing list archive
>> at Nabble.com.
>>
Re: JMS vendors
Posted by Mike Drob <ma...@cloudera.com>.
Mike,
This is a solid idea. Is there a JIRA for tracking it?
Mike
On Mon, May 18, 2015 at 9:01 AM, Michael Moser <mo...@gmail.com> wrote:
> Having extended GetJMS*/PutJMS* myself to use another JMS vendor, I think
> we can make this much easier than it currently is.
>
> Right now, the provider's ConnectionFactory creation is buried in the NiFi
> JmsFactory class which the GetJMS*/PutJMS* processors use. This JmsFactory
> class doesn't implement an interface or extend a base class, so it cannot
> be easily replaced or extended. I ended up copying it and the
> GetJMS*/PutJMS* source into my own project in order to extend them. Not
> ideal.
>
> If we can move provider ConnectionFactory creation into the processors
> themselves, then doing this would be as simple as extending the
> GetJMS*/PutJMS* processors and overriding the ConnectionFactory creation
> method.
>
> This is on my list of things that I wanted to work on and contribute, but
> haven't allocated time for.
>
> -- Mike
>
>
> On Mon, May 18, 2015 at 6:35 AM, Toivo Adams <to...@gmail.com>
> wrote:
>
> > Joe,
> >
> > You are right, creating special GetJms*/PutJms* version for other vendor
> is
> > not difficult.
> > And yes more specifically I have interested to have only SonicMQ support
> > because its used heavily.
> >
> > I was hoping to have some general solution which helps more easily to
> start
> > using any JMS vendor.
> >
> > When each user create its own GetJms*/PutJms* version, such version is
> much
> > less battle tested than Nifi standard processors (much smaller user
> base).
> >
> > But creating some kind of general JMS solution requires much more effort
> > than separate version for one vendor.
> > So it's boils down how big is the demand.
> > It seems there are not (yet?) many different JMS vendor users in
> NiFiland?
> >
> >
> > Thanks
> > Toivo
> >
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/JMS-vendors-tp1558p1567.html
> > Sent from the Apache NiFi (incubating) Developer List mailing list
> archive
> > at Nabble.com.
> >
>
Re: JMS vendors
Posted by Michael Moser <mo...@gmail.com>.
Having extended GetJMS*/PutJMS* myself to use another JMS vendor, I think
we can make this much easier than it currently is.
Right now, the provider's ConnectionFactory creation is buried in the NiFi
JmsFactory class which the GetJMS*/PutJMS* processors use. This JmsFactory
class doesn't implement an interface or extend a base class, so it cannot
be easily replaced or extended. I ended up copying it and the
GetJMS*/PutJMS* source into my own project in order to extend them. Not
ideal.
If we can move provider ConnectionFactory creation into the processors
themselves, then doing this would be as simple as extending the
GetJMS*/PutJMS* processors and overriding the ConnectionFactory creation
method.
This is on my list of things that I wanted to work on and contribute, but
haven't allocated time for.
-- Mike
On Mon, May 18, 2015 at 6:35 AM, Toivo Adams <to...@gmail.com> wrote:
> Joe,
>
> You are right, creating special GetJms*/PutJms* version for other vendor is
> not difficult.
> And yes more specifically I have interested to have only SonicMQ support
> because its used heavily.
>
> I was hoping to have some general solution which helps more easily to start
> using any JMS vendor.
>
> When each user create its own GetJms*/PutJms* version, such version is much
> less battle tested than Nifi standard processors (much smaller user base).
>
> But creating some kind of general JMS solution requires much more effort
> than separate version for one vendor.
> So it's boils down how big is the demand.
> It seems there are not (yet?) many different JMS vendor users in NiFiland?
>
>
> Thanks
> Toivo
>
>
>
>
> --
> View this message in context:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/JMS-vendors-tp1558p1567.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive
> at Nabble.com.
>
Re: JMS vendors
Posted by Toivo Adams <to...@gmail.com>.
Joe,
You are right, creating special GetJms*/PutJms* version for other vendor is
not difficult.
And yes more specifically I have interested to have only SonicMQ support
because its used heavily.
I was hoping to have some general solution which helps more easily to start
using any JMS vendor.
When each user create its own GetJms*/PutJms* version, such version is much
less battle tested than Nifi standard processors (much smaller user base).
But creating some kind of general JMS solution requires much more effort
than separate version for one vendor.
So it's boils down how big is the demand.
It seems there are not (yet?) many different JMS vendor users in NiFiland?
Thanks
Toivo
--
View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/JMS-vendors-tp1558p1567.html
Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
Re: JMS vendors
Posted by Joe Witt <jo...@gmail.com>.
Toivo
It is rather easy to support additional brokers. However the issue has
been their fitness with asl v2. Besides soniq what else would you like to
see support for?
Absent being able to include in a release we could provide a blog
explaining how to support any jms broker you like by extending the existing
processor perhaps.
Thanks
Joe
On May 17, 2015 7:11 AM, "Toivo Adams" <to...@gmail.com> wrote:
> Currently only Active MQ is supported?
>
> How other vendors JMS implementation can be used?
> For example SonicMQ
>
> JMS is fairly well standardized, so most of the send/receive code is vendor
> independent.
>
> But often connection factories have more or less vendor specific parts.
>
> And of course JMS client libraries are vendor specific, sometimes 4-6
> proprietary jar files.
> Some of them are not open source.
>
> Thanks
> Toivo
>
>
>
>
> --
> View this message in context:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/JMS-vendors-tp1558.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive
> at Nabble.com.
>