You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com> on 2016/09/14 13:28:11 UTC
ConsumeJMS and Acknowledgement Mode
Hey folks,
The GetJMSQueue processor allows configuration of the Acknowledgment Mode.
Acknowledgement Mode
Client Acknowledge
· Client Acknowledge
· Auto Acknowledge
The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.
Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS processor?
The reason I ask is that I seem to be losing messages on NiFi restart.
Thanks,
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
[cid:image001.png@01D20E6A.4EDB8FE0]
Re: ConsumeJMS and Acknowledgement Mode
Posted by Oleg Zhurakousky <oz...@hortonworks.com>.
Good point about lambda. Let me see if I can refactor it a bit.
And sure, please share your patch. Would love to compare.
Cheers
Oleg
> On Sep 15, 2016, at 11:10 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>
> Oleg,
>
> Thanks very much. Yes, I fully agree with you.
>
> I saw your PR and noted that it’s for 1.x. Since it uses lambdas the patch is not going to work for the 0.x branch. I don’t know what your plan for 0.8 is but I have a much less sophisticated patch for the for the 0.x branch. Should I share?
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
> On 9/15/16, 9:45 AM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
>
> Chris
>
> PR was submitted to the JIRA. Basically as I’ve commented in the JIRA I now do not believe we should be exposing any QOS options to the end user and instead implicitly ack messages only after they’ve been consumed by NiFi (i.e., transferred to a success relationship). Basically a classic chain of custody pattern.
> Let me know if you agree since adding ACK mode options would be a simply change, although at the moment I am reluctant to do so (see my explanation in JIRA)
>
> Cheers
> Oleg
>
>> On Sep 14, 2016, at 10:13 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>>
>> Will do. Thanks!
>>
>> Chris McDermott
>>
>> Remote Business Analytics
>> STaTS/StoreFront Remote
>> HPE Storage
>> Hewlett Packard Enterprise
>> Mobile: +1 978-697-5315
>>
>>
>>
>>
>> On 9/14/16, 10:10 AM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
>>
>> Chris
>>
>> The “AUTO_ACKNOWLEDGE” (actual value 1) is the default. So although it is not being set explicitly it is set.
>>
>> We can definitely expose the other two (client ack and dups ok).
>> Please raise JIRA.
>> Cheers
>> Oleg
>>
>> On Sep 14, 2016, at 10:04 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
>>
>> Looking at the code the ACK mode is not being set.
>>
>> /**
>> * This method essentially performs initialization of this Processor by
>> * obtaining an instance of the {@link ConnectionFactory} from the
>> * {@link JMSConnectionFactoryProvider} (ControllerService) and performing a
>> * series of {@link ConnectionFactory} adaptations which eventually results
>> * in an instance of the {@link CachingConnectionFactory} used to construct
>> * {@link JmsTemplate} used by this Processor.
>> */
>> private void buildTargetResource(ProcessContext context) {
>> if (this.targetResource == null) {
>> JMSConnectionFactoryProviderDefinition cfProvider = context.getProperty(CF_SERVICE).asControllerService(JMSConnectionFactoryProviderDefinition.class);
>> ConnectionFactory connectionFactory = cfProvider.getConnectionFactory();
>>
>> UserCredentialsConnectionFactoryAdapter cfCredentialsAdapter = new UserCredentialsConnectionFactoryAdapter();
>> cfCredentialsAdapter.setTargetConnectionFactory(connectionFactory);
>> cfCredentialsAdapter.setUsername(context.getProperty(USER).getValue());
>> cfCredentialsAdapter.setPassword(context.getProperty(PASSWORD).getValue());
>>
>> this.cachingConnectionFactory = new CachingConnectionFactory(cfCredentialsAdapter);
>> this.cachingConnectionFactory.setSessionCacheSize(Integer.parseInt(context.getProperty(SESSION_CACHE_SIZE).getValue()));
>>
>> JmsTemplate jmsTemplate = new JmsTemplate();
>> jmsTemplate.setConnectionFactory(this.cachingConnectionFactory);
>> jmsTemplate.setDefaultDestinationName(context.getProperty(DESTINATION).getValue());
>> jmsTemplate.setPubSubDomain(TOPIC.equals(context.getProperty(DESTINATION_TYPE).getValue()));
>>
>> // set of properties that may be good candidates for exposure via configuration
>> jmsTemplate.setReceiveTimeout(10000);
>>
>> this.targetResource = this.finishBuildingTargetResource(jmsTemplate);
>> }
>>
>> The JavaDoc for JMSTemplate says.
>>
>> Default settings for JMS Sessions are "not transacted" and "auto-acknowledge". As defined by the Java EE specification, the transaction and acknowledgement parameters are ignored when a JMS Session is created inside an active transaction, no matter if a JTA transaction or a Spring-managed transaction. To configure them for native JMS usage, specify appropriate values for the "sessionTransacted" and "sessionAcknowledgeMode" bean properties.
>>
>> If I understand this correctly it means that ConsumeJMS is always using auto-acknowledge which would explain my lost files on NiFi restart.
>>
>> Do you concur?
>>
>> Chris McDermott
>>
>> Remote Business Analytics
>> STaTS/StoreFront Remote
>> HPE Storage
>> Hewlett Packard Enterprise
>> Mobile: +1 978-697-5315
>>
>>
>>
>>
>> On 9/14/16, 9:57 AM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com>> wrote:
>>
>> Yes.
>>
>> Chris McDermott
>>
>> Remote Business Analytics
>> STaTS/StoreFront Remote
>> HPE Storage
>> Hewlett Packard Enterprise
>> Mobile: +1 978-697-5315
>>
>>
>>
>>
>> On 9/14/16, 9:54 AM, "Oleg Zhurakousky" <oz...@hortonworks.com>> wrote:
>>
>> So you’re asking about ConsumeJMS?
>>
>> On Sep 14, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
>>
>> Hi Oleg,
>>
>> Sorry, I wasn’t more clear. I should have stated that ConsumeJMS does not expose a setting for ACK mode. Screen shot attached. So my question is which ACK mode does it use?
>>
>> Thanks,
>>
>> Chris McDermott
>>
>> Remote Business Analytics
>> STaTS/StoreFront Remote
>> HPE Storage
>> Hewlett Packard Enterprise
>> Mobile: +1 978-697-5315
>>
>>
>>
>>
>> On 9/14/16, 9:41 AM, "Oleg Zhurakousky" <oz...@hortonworks.com>> wrote:
>>
>> Chris
>>
>> FWIW, the Get/PutJMS* are effectively deprecated processors since they only support one JMS provider (ActiiveMQ) and should not be used. Instead we recommend Publish/ConsumeJMS pair.
>>
>> As to your question, which specific ACK mode you are inquiring about? As the doc currently states, the auto-ACK may loose messages. Is that what you are using?
>>
>> Cheers
>> Oleg
>>
>> On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
>>
>>
>> Hey folks,
>>
>> The GetJMSQueue processor allows configuration of the Acknowledgment Mode.
>>
>> Acknowledgement Mode
>>
>> Client Acknowledge
>>
>> • Client Acknowledge
>> • Auto Acknowledge
>>
>> The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.
>>
>>
>> Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS processor?
>>
>> The reason I ask is that I seem to be losing messages on NiFi restart.
>>
>> Thanks,
>> Chris McDermott
>>
>> Remote Business Analytics
>> STaTS/StoreFront Remote
>> HPE Storage
>> Hewlett Packard Enterprise
>> Mobile: +1 978-697-5315
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>
>
>
Re: ConsumeJMS and Acknowledgement Mode
Posted by "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com>.
Oleg,
Thanks very much. Yes, I fully agree with you.
I saw your PR and noted that it’s for 1.x. Since it uses lambdas the patch is not going to work for the 0.x branch. I don’t know what your plan for 0.8 is but I have a much less sophisticated patch for the for the 0.x branch. Should I share?
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
On 9/15/16, 9:45 AM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
Chris
PR was submitted to the JIRA. Basically as I’ve commented in the JIRA I now do not believe we should be exposing any QOS options to the end user and instead implicitly ack messages only after they’ve been consumed by NiFi (i.e., transferred to a success relationship). Basically a classic chain of custody pattern.
Let me know if you agree since adding ACK mode options would be a simply change, although at the moment I am reluctant to do so (see my explanation in JIRA)
Cheers
Oleg
> On Sep 14, 2016, at 10:13 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>
> Will do. Thanks!
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
> On 9/14/16, 10:10 AM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
>
> Chris
>
> The “AUTO_ACKNOWLEDGE” (actual value 1) is the default. So although it is not being set explicitly it is set.
>
> We can definitely expose the other two (client ack and dups ok).
> Please raise JIRA.
> Cheers
> Oleg
>
> On Sep 14, 2016, at 10:04 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
>
> Looking at the code the ACK mode is not being set.
>
> /**
> * This method essentially performs initialization of this Processor by
> * obtaining an instance of the {@link ConnectionFactory} from the
> * {@link JMSConnectionFactoryProvider} (ControllerService) and performing a
> * series of {@link ConnectionFactory} adaptations which eventually results
> * in an instance of the {@link CachingConnectionFactory} used to construct
> * {@link JmsTemplate} used by this Processor.
> */
> private void buildTargetResource(ProcessContext context) {
> if (this.targetResource == null) {
> JMSConnectionFactoryProviderDefinition cfProvider = context.getProperty(CF_SERVICE).asControllerService(JMSConnectionFactoryProviderDefinition.class);
> ConnectionFactory connectionFactory = cfProvider.getConnectionFactory();
>
> UserCredentialsConnectionFactoryAdapter cfCredentialsAdapter = new UserCredentialsConnectionFactoryAdapter();
> cfCredentialsAdapter.setTargetConnectionFactory(connectionFactory);
> cfCredentialsAdapter.setUsername(context.getProperty(USER).getValue());
> cfCredentialsAdapter.setPassword(context.getProperty(PASSWORD).getValue());
>
> this.cachingConnectionFactory = new CachingConnectionFactory(cfCredentialsAdapter);
> this.cachingConnectionFactory.setSessionCacheSize(Integer.parseInt(context.getProperty(SESSION_CACHE_SIZE).getValue()));
>
> JmsTemplate jmsTemplate = new JmsTemplate();
> jmsTemplate.setConnectionFactory(this.cachingConnectionFactory);
> jmsTemplate.setDefaultDestinationName(context.getProperty(DESTINATION).getValue());
> jmsTemplate.setPubSubDomain(TOPIC.equals(context.getProperty(DESTINATION_TYPE).getValue()));
>
> // set of properties that may be good candidates for exposure via configuration
> jmsTemplate.setReceiveTimeout(10000);
>
> this.targetResource = this.finishBuildingTargetResource(jmsTemplate);
> }
>
> The JavaDoc for JMSTemplate says.
>
> Default settings for JMS Sessions are "not transacted" and "auto-acknowledge". As defined by the Java EE specification, the transaction and acknowledgement parameters are ignored when a JMS Session is created inside an active transaction, no matter if a JTA transaction or a Spring-managed transaction. To configure them for native JMS usage, specify appropriate values for the "sessionTransacted" and "sessionAcknowledgeMode" bean properties.
>
> If I understand this correctly it means that ConsumeJMS is always using auto-acknowledge which would explain my lost files on NiFi restart.
>
> Do you concur?
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
> On 9/14/16, 9:57 AM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com>> wrote:
>
> Yes.
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
> On 9/14/16, 9:54 AM, "Oleg Zhurakousky" <oz...@hortonworks.com>> wrote:
>
> So you’re asking about ConsumeJMS?
>
> On Sep 14, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
>
> Hi Oleg,
>
> Sorry, I wasn’t more clear. I should have stated that ConsumeJMS does not expose a setting for ACK mode. Screen shot attached. So my question is which ACK mode does it use?
>
> Thanks,
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
> On 9/14/16, 9:41 AM, "Oleg Zhurakousky" <oz...@hortonworks.com>> wrote:
>
> Chris
>
> FWIW, the Get/PutJMS* are effectively deprecated processors since they only support one JMS provider (ActiiveMQ) and should not be used. Instead we recommend Publish/ConsumeJMS pair.
>
> As to your question, which specific ACK mode you are inquiring about? As the doc currently states, the auto-ACK may loose messages. Is that what you are using?
>
> Cheers
> Oleg
>
> On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
>
>
> Hey folks,
>
> The GetJMSQueue processor allows configuration of the Acknowledgment Mode.
>
> Acknowledgement Mode
>
> Client Acknowledge
>
> • Client Acknowledge
> • Auto Acknowledge
>
> The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.
>
>
> Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS processor?
>
> The reason I ask is that I seem to be losing messages on NiFi restart.
>
> Thanks,
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
Re: ConsumeJMS and Acknowledgement Mode
Posted by Oleg Zhurakousky <oz...@hortonworks.com>.
Chris
PR was submitted to the JIRA. Basically as I’ve commented in the JIRA I now do not believe we should be exposing any QOS options to the end user and instead implicitly ack messages only after they’ve been consumed by NiFi (i.e., transferred to a success relationship). Basically a classic chain of custody pattern.
Let me know if you agree since adding ACK mode options would be a simply change, although at the moment I am reluctant to do so (see my explanation in JIRA)
Cheers
Oleg
> On Sep 14, 2016, at 10:13 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>
> Will do. Thanks!
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
> On 9/14/16, 10:10 AM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
>
> Chris
>
> The “AUTO_ACKNOWLEDGE” (actual value 1) is the default. So although it is not being set explicitly it is set.
>
> We can definitely expose the other two (client ack and dups ok).
> Please raise JIRA.
> Cheers
> Oleg
>
> On Sep 14, 2016, at 10:04 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
>
> Looking at the code the ACK mode is not being set.
>
> /**
> * This method essentially performs initialization of this Processor by
> * obtaining an instance of the {@link ConnectionFactory} from the
> * {@link JMSConnectionFactoryProvider} (ControllerService) and performing a
> * series of {@link ConnectionFactory} adaptations which eventually results
> * in an instance of the {@link CachingConnectionFactory} used to construct
> * {@link JmsTemplate} used by this Processor.
> */
> private void buildTargetResource(ProcessContext context) {
> if (this.targetResource == null) {
> JMSConnectionFactoryProviderDefinition cfProvider = context.getProperty(CF_SERVICE).asControllerService(JMSConnectionFactoryProviderDefinition.class);
> ConnectionFactory connectionFactory = cfProvider.getConnectionFactory();
>
> UserCredentialsConnectionFactoryAdapter cfCredentialsAdapter = new UserCredentialsConnectionFactoryAdapter();
> cfCredentialsAdapter.setTargetConnectionFactory(connectionFactory);
> cfCredentialsAdapter.setUsername(context.getProperty(USER).getValue());
> cfCredentialsAdapter.setPassword(context.getProperty(PASSWORD).getValue());
>
> this.cachingConnectionFactory = new CachingConnectionFactory(cfCredentialsAdapter);
> this.cachingConnectionFactory.setSessionCacheSize(Integer.parseInt(context.getProperty(SESSION_CACHE_SIZE).getValue()));
>
> JmsTemplate jmsTemplate = new JmsTemplate();
> jmsTemplate.setConnectionFactory(this.cachingConnectionFactory);
> jmsTemplate.setDefaultDestinationName(context.getProperty(DESTINATION).getValue());
> jmsTemplate.setPubSubDomain(TOPIC.equals(context.getProperty(DESTINATION_TYPE).getValue()));
>
> // set of properties that may be good candidates for exposure via configuration
> jmsTemplate.setReceiveTimeout(10000);
>
> this.targetResource = this.finishBuildingTargetResource(jmsTemplate);
> }
>
> The JavaDoc for JMSTemplate says.
>
> Default settings for JMS Sessions are "not transacted" and "auto-acknowledge". As defined by the Java EE specification, the transaction and acknowledgement parameters are ignored when a JMS Session is created inside an active transaction, no matter if a JTA transaction or a Spring-managed transaction. To configure them for native JMS usage, specify appropriate values for the "sessionTransacted" and "sessionAcknowledgeMode" bean properties.
>
> If I understand this correctly it means that ConsumeJMS is always using auto-acknowledge which would explain my lost files on NiFi restart.
>
> Do you concur?
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
> On 9/14/16, 9:57 AM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com>> wrote:
>
> Yes.
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
> On 9/14/16, 9:54 AM, "Oleg Zhurakousky" <oz...@hortonworks.com>> wrote:
>
> So you’re asking about ConsumeJMS?
>
> On Sep 14, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
>
> Hi Oleg,
>
> Sorry, I wasn’t more clear. I should have stated that ConsumeJMS does not expose a setting for ACK mode. Screen shot attached. So my question is which ACK mode does it use?
>
> Thanks,
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
> On 9/14/16, 9:41 AM, "Oleg Zhurakousky" <oz...@hortonworks.com>> wrote:
>
> Chris
>
> FWIW, the Get/PutJMS* are effectively deprecated processors since they only support one JMS provider (ActiiveMQ) and should not be used. Instead we recommend Publish/ConsumeJMS pair.
>
> As to your question, which specific ACK mode you are inquiring about? As the doc currently states, the auto-ACK may loose messages. Is that what you are using?
>
> Cheers
> Oleg
>
> On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
>
>
> Hey folks,
>
> The GetJMSQueue processor allows configuration of the Acknowledgment Mode.
>
> Acknowledgement Mode
>
> Client Acknowledge
>
> • Client Acknowledge
> • Auto Acknowledge
>
> The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.
>
>
> Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS processor?
>
> The reason I ask is that I seem to be losing messages on NiFi restart.
>
> Thanks,
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
Re: ConsumeJMS and Acknowledgement Mode
Posted by "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com>.
Will do. Thanks!
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
On 9/14/16, 10:10 AM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
Chris
The “AUTO_ACKNOWLEDGE” (actual value 1) is the default. So although it is not being set explicitly it is set.
We can definitely expose the other two (client ack and dups ok).
Please raise JIRA.
Cheers
Oleg
On Sep 14, 2016, at 10:04 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
Looking at the code the ACK mode is not being set.
/**
* This method essentially performs initialization of this Processor by
* obtaining an instance of the {@link ConnectionFactory} from the
* {@link JMSConnectionFactoryProvider} (ControllerService) and performing a
* series of {@link ConnectionFactory} adaptations which eventually results
* in an instance of the {@link CachingConnectionFactory} used to construct
* {@link JmsTemplate} used by this Processor.
*/
private void buildTargetResource(ProcessContext context) {
if (this.targetResource == null) {
JMSConnectionFactoryProviderDefinition cfProvider = context.getProperty(CF_SERVICE).asControllerService(JMSConnectionFactoryProviderDefinition.class);
ConnectionFactory connectionFactory = cfProvider.getConnectionFactory();
UserCredentialsConnectionFactoryAdapter cfCredentialsAdapter = new UserCredentialsConnectionFactoryAdapter();
cfCredentialsAdapter.setTargetConnectionFactory(connectionFactory);
cfCredentialsAdapter.setUsername(context.getProperty(USER).getValue());
cfCredentialsAdapter.setPassword(context.getProperty(PASSWORD).getValue());
this.cachingConnectionFactory = new CachingConnectionFactory(cfCredentialsAdapter);
this.cachingConnectionFactory.setSessionCacheSize(Integer.parseInt(context.getProperty(SESSION_CACHE_SIZE).getValue()));
JmsTemplate jmsTemplate = new JmsTemplate();
jmsTemplate.setConnectionFactory(this.cachingConnectionFactory);
jmsTemplate.setDefaultDestinationName(context.getProperty(DESTINATION).getValue());
jmsTemplate.setPubSubDomain(TOPIC.equals(context.getProperty(DESTINATION_TYPE).getValue()));
// set of properties that may be good candidates for exposure via configuration
jmsTemplate.setReceiveTimeout(10000);
this.targetResource = this.finishBuildingTargetResource(jmsTemplate);
}
The JavaDoc for JMSTemplate says.
Default settings for JMS Sessions are "not transacted" and "auto-acknowledge". As defined by the Java EE specification, the transaction and acknowledgement parameters are ignored when a JMS Session is created inside an active transaction, no matter if a JTA transaction or a Spring-managed transaction. To configure them for native JMS usage, specify appropriate values for the "sessionTransacted" and "sessionAcknowledgeMode" bean properties.
If I understand this correctly it means that ConsumeJMS is always using auto-acknowledge which would explain my lost files on NiFi restart.
Do you concur?
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
On 9/14/16, 9:57 AM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com>> wrote:
Yes.
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
On 9/14/16, 9:54 AM, "Oleg Zhurakousky" <oz...@hortonworks.com>> wrote:
So you’re asking about ConsumeJMS?
On Sep 14, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
Hi Oleg,
Sorry, I wasn’t more clear. I should have stated that ConsumeJMS does not expose a setting for ACK mode. Screen shot attached. So my question is which ACK mode does it use?
Thanks,
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
On 9/14/16, 9:41 AM, "Oleg Zhurakousky" <oz...@hortonworks.com>> wrote:
Chris
FWIW, the Get/PutJMS* are effectively deprecated processors since they only support one JMS provider (ActiiveMQ) and should not be used. Instead we recommend Publish/ConsumeJMS pair.
As to your question, which specific ACK mode you are inquiring about? As the doc currently states, the auto-ACK may loose messages. Is that what you are using?
Cheers
Oleg
On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
Hey folks,
The GetJMSQueue processor allows configuration of the Acknowledgment Mode.
Acknowledgement Mode
Client Acknowledge
• Client Acknowledge
• Auto Acknowledge
The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.
Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS processor?
The reason I ask is that I seem to be losing messages on NiFi restart.
Thanks,
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
Re: ConsumeJMS and Acknowledgement Mode
Posted by Oleg Zhurakousky <oz...@hortonworks.com>.
Chris
The “AUTO_ACKNOWLEDGE” (actual value 1) is the default. So although it is not being set explicitly it is set.
We can definitely expose the other two (client ack and dups ok).
Please raise JIRA.
Cheers
Oleg
On Sep 14, 2016, at 10:04 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
Looking at the code the ACK mode is not being set.
/**
* This method essentially performs initialization of this Processor by
* obtaining an instance of the {@link ConnectionFactory} from the
* {@link JMSConnectionFactoryProvider} (ControllerService) and performing a
* series of {@link ConnectionFactory} adaptations which eventually results
* in an instance of the {@link CachingConnectionFactory} used to construct
* {@link JmsTemplate} used by this Processor.
*/
private void buildTargetResource(ProcessContext context) {
if (this.targetResource == null) {
JMSConnectionFactoryProviderDefinition cfProvider = context.getProperty(CF_SERVICE).asControllerService(JMSConnectionFactoryProviderDefinition.class);
ConnectionFactory connectionFactory = cfProvider.getConnectionFactory();
UserCredentialsConnectionFactoryAdapter cfCredentialsAdapter = new UserCredentialsConnectionFactoryAdapter();
cfCredentialsAdapter.setTargetConnectionFactory(connectionFactory);
cfCredentialsAdapter.setUsername(context.getProperty(USER).getValue());
cfCredentialsAdapter.setPassword(context.getProperty(PASSWORD).getValue());
this.cachingConnectionFactory = new CachingConnectionFactory(cfCredentialsAdapter);
this.cachingConnectionFactory.setSessionCacheSize(Integer.parseInt(context.getProperty(SESSION_CACHE_SIZE).getValue()));
JmsTemplate jmsTemplate = new JmsTemplate();
jmsTemplate.setConnectionFactory(this.cachingConnectionFactory);
jmsTemplate.setDefaultDestinationName(context.getProperty(DESTINATION).getValue());
jmsTemplate.setPubSubDomain(TOPIC.equals(context.getProperty(DESTINATION_TYPE).getValue()));
// set of properties that may be good candidates for exposure via configuration
jmsTemplate.setReceiveTimeout(10000);
this.targetResource = this.finishBuildingTargetResource(jmsTemplate);
}
The JavaDoc for JMSTemplate says.
Default settings for JMS Sessions are "not transacted" and "auto-acknowledge". As defined by the Java EE specification, the transaction and acknowledgement parameters are ignored when a JMS Session is created inside an active transaction, no matter if a JTA transaction or a Spring-managed transaction. To configure them for native JMS usage, specify appropriate values for the "sessionTransacted" and "sessionAcknowledgeMode" bean properties.
If I understand this correctly it means that ConsumeJMS is always using auto-acknowledge which would explain my lost files on NiFi restart.
Do you concur?
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
On 9/14/16, 9:57 AM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com>> wrote:
Yes.
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
On 9/14/16, 9:54 AM, "Oleg Zhurakousky" <oz...@hortonworks.com>> wrote:
So you’re asking about ConsumeJMS?
On Sep 14, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
Hi Oleg,
Sorry, I wasn’t more clear. I should have stated that ConsumeJMS does not expose a setting for ACK mode. Screen shot attached. So my question is which ACK mode does it use?
Thanks,
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
On 9/14/16, 9:41 AM, "Oleg Zhurakousky" <oz...@hortonworks.com>> wrote:
Chris
FWIW, the Get/PutJMS* are effectively deprecated processors since they only support one JMS provider (ActiiveMQ) and should not be used. Instead we recommend Publish/ConsumeJMS pair.
As to your question, which specific ACK mode you are inquiring about? As the doc currently states, the auto-ACK may loose messages. Is that what you are using?
Cheers
Oleg
On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
Hey folks,
The GetJMSQueue processor allows configuration of the Acknowledgment Mode.
Acknowledgement Mode
Client Acknowledge
• Client Acknowledge
• Auto Acknowledge
The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.
Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS processor?
The reason I ask is that I seem to be losing messages on NiFi restart.
Thanks,
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
Re: ConsumeJMS and Acknowledgement Mode
Posted by "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com>.
Looking at the code the ACK mode is not being set.
/**
* This method essentially performs initialization of this Processor by
* obtaining an instance of the {@link ConnectionFactory} from the
* {@link JMSConnectionFactoryProvider} (ControllerService) and performing a
* series of {@link ConnectionFactory} adaptations which eventually results
* in an instance of the {@link CachingConnectionFactory} used to construct
* {@link JmsTemplate} used by this Processor.
*/
private void buildTargetResource(ProcessContext context) {
if (this.targetResource == null) {
JMSConnectionFactoryProviderDefinition cfProvider = context.getProperty(CF_SERVICE).asControllerService(JMSConnectionFactoryProviderDefinition.class);
ConnectionFactory connectionFactory = cfProvider.getConnectionFactory();
UserCredentialsConnectionFactoryAdapter cfCredentialsAdapter = new UserCredentialsConnectionFactoryAdapter();
cfCredentialsAdapter.setTargetConnectionFactory(connectionFactory);
cfCredentialsAdapter.setUsername(context.getProperty(USER).getValue());
cfCredentialsAdapter.setPassword(context.getProperty(PASSWORD).getValue());
this.cachingConnectionFactory = new CachingConnectionFactory(cfCredentialsAdapter);
this.cachingConnectionFactory.setSessionCacheSize(Integer.parseInt(context.getProperty(SESSION_CACHE_SIZE).getValue()));
JmsTemplate jmsTemplate = new JmsTemplate();
jmsTemplate.setConnectionFactory(this.cachingConnectionFactory);
jmsTemplate.setDefaultDestinationName(context.getProperty(DESTINATION).getValue());
jmsTemplate.setPubSubDomain(TOPIC.equals(context.getProperty(DESTINATION_TYPE).getValue()));
// set of properties that may be good candidates for exposure via configuration
jmsTemplate.setReceiveTimeout(10000);
this.targetResource = this.finishBuildingTargetResource(jmsTemplate);
}
The JavaDoc for JMSTemplate says.
Default settings for JMS Sessions are "not transacted" and "auto-acknowledge". As defined by the Java EE specification, the transaction and acknowledgement parameters are ignored when a JMS Session is created inside an active transaction, no matter if a JTA transaction or a Spring-managed transaction. To configure them for native JMS usage, specify appropriate values for the "sessionTransacted" and "sessionAcknowledgeMode" bean properties.
If I understand this correctly it means that ConsumeJMS is always using auto-acknowledge which would explain my lost files on NiFi restart.
Do you concur?
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
On 9/14/16, 9:57 AM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com> wrote:
Yes.
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
On 9/14/16, 9:54 AM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
So you’re asking about ConsumeJMS?
> On Sep 14, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>
> Hi Oleg,
>
> Sorry, I wasn’t more clear. I should have stated that ConsumeJMS does not expose a setting for ACK mode. Screen shot attached. So my question is which ACK mode does it use?
>
> Thanks,
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
> On 9/14/16, 9:41 AM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
>
> Chris
>
> FWIW, the Get/PutJMS* are effectively deprecated processors since they only support one JMS provider (ActiiveMQ) and should not be used. Instead we recommend Publish/ConsumeJMS pair.
>
> As to your question, which specific ACK mode you are inquiring about? As the doc currently states, the auto-ACK may loose messages. Is that what you are using?
>
> Cheers
> Oleg
>
> On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
>
>
> Hey folks,
>
> The GetJMSQueue processor allows configuration of the Acknowledgment Mode.
>
> Acknowledgement Mode
>
> Client Acknowledge
>
> • Client Acknowledge
> • Auto Acknowledge
>
> The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.
>
>
> Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS processor?
>
> The reason I ask is that I seem to be losing messages on NiFi restart.
>
> Thanks,
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
>
Re: ConsumeJMS and Acknowledgement Mode
Posted by Oleg Zhurakousky <oz...@hortonworks.com>.
Ok, sorry, got confused a bit.
Indeed ConsumeJMS is using auto-ack at the moment, hence can suffer from the same problem as GetJMS.
However, the designed of this new JMS support definitely provides for exposure of ACK and other properties to the end user. Please raise JIRA and we’ll take care of it.
Cheers
Oleg
> On Sep 14, 2016, at 9:57 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>
> Yes.
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
> On 9/14/16, 9:54 AM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
>
> So you’re asking about ConsumeJMS?
>
>> On Sep 14, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>>
>> Hi Oleg,
>>
>> Sorry, I wasn’t more clear. I should have stated that ConsumeJMS does not expose a setting for ACK mode. Screen shot attached. So my question is which ACK mode does it use?
>>
>> Thanks,
>>
>> Chris McDermott
>>
>> Remote Business Analytics
>> STaTS/StoreFront Remote
>> HPE Storage
>> Hewlett Packard Enterprise
>> Mobile: +1 978-697-5315
>>
>>
>>
>>
>> On 9/14/16, 9:41 AM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
>>
>> Chris
>>
>> FWIW, the Get/PutJMS* are effectively deprecated processors since they only support one JMS provider (ActiiveMQ) and should not be used. Instead we recommend Publish/ConsumeJMS pair.
>>
>> As to your question, which specific ACK mode you are inquiring about? As the doc currently states, the auto-ACK may loose messages. Is that what you are using?
>>
>> Cheers
>> Oleg
>>
>> On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
>>
>>
>> Hey folks,
>>
>> The GetJMSQueue processor allows configuration of the Acknowledgment Mode.
>>
>> Acknowledgement Mode
>>
>> Client Acknowledge
>>
>> • Client Acknowledge
>> • Auto Acknowledge
>>
>> The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.
>>
>>
>> Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS processor?
>>
>> The reason I ask is that I seem to be losing messages on NiFi restart.
>>
>> Thanks,
>> Chris McDermott
>>
>> Remote Business Analytics
>> STaTS/StoreFront Remote
>> HPE Storage
>> Hewlett Packard Enterprise
>> Mobile: +1 978-697-5315
>>
>>
>>
>>
>>
>
>
>
>
Re: ConsumeJMS and Acknowledgement Mode
Posted by "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com>.
Yes.
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
On 9/14/16, 9:54 AM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
So you’re asking about ConsumeJMS?
> On Sep 14, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>
> Hi Oleg,
>
> Sorry, I wasn’t more clear. I should have stated that ConsumeJMS does not expose a setting for ACK mode. Screen shot attached. So my question is which ACK mode does it use?
>
> Thanks,
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
> On 9/14/16, 9:41 AM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
>
> Chris
>
> FWIW, the Get/PutJMS* are effectively deprecated processors since they only support one JMS provider (ActiiveMQ) and should not be used. Instead we recommend Publish/ConsumeJMS pair.
>
> As to your question, which specific ACK mode you are inquiring about? As the doc currently states, the auto-ACK may loose messages. Is that what you are using?
>
> Cheers
> Oleg
>
> On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
>
>
> Hey folks,
>
> The GetJMSQueue processor allows configuration of the Acknowledgment Mode.
>
> Acknowledgement Mode
>
> Client Acknowledge
>
> • Client Acknowledge
> • Auto Acknowledge
>
> The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.
>
>
> Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS processor?
>
> The reason I ask is that I seem to be losing messages on NiFi restart.
>
> Thanks,
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
>
Re: ConsumeJMS and Acknowledgement Mode
Posted by Oleg Zhurakousky <oz...@hortonworks.com>.
So you’re asking about ConsumeJMS?
> On Sep 14, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>
> Hi Oleg,
>
> Sorry, I wasn’t more clear. I should have stated that ConsumeJMS does not expose a setting for ACK mode. Screen shot attached. So my question is which ACK mode does it use?
>
> Thanks,
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
> On 9/14/16, 9:41 AM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
>
> Chris
>
> FWIW, the Get/PutJMS* are effectively deprecated processors since they only support one JMS provider (ActiiveMQ) and should not be used. Instead we recommend Publish/ConsumeJMS pair.
>
> As to your question, which specific ACK mode you are inquiring about? As the doc currently states, the auto-ACK may loose messages. Is that what you are using?
>
> Cheers
> Oleg
>
> On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
>
>
> Hey folks,
>
> The GetJMSQueue processor allows configuration of the Acknowledgment Mode.
>
> Acknowledgement Mode
>
> Client Acknowledge
>
> • Client Acknowledge
> • Auto Acknowledge
>
> The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.
>
>
> Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS processor?
>
> The reason I ask is that I seem to be losing messages on NiFi restart.
>
> Thanks,
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
>
>
>
>
Re: ConsumeJMS and Acknowledgement Mode
Posted by "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com>.
Hi Oleg,
Sorry, I wasn’t more clear. I should have stated that ConsumeJMS does not expose a setting for ACK mode. Screen shot attached. So my question is which ACK mode does it use?
Thanks,
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
On 9/14/16, 9:41 AM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
Chris
FWIW, the Get/PutJMS* are effectively deprecated processors since they only support one JMS provider (ActiiveMQ) and should not be used. Instead we recommend Publish/ConsumeJMS pair.
As to your question, which specific ACK mode you are inquiring about? As the doc currently states, the auto-ACK may loose messages. Is that what you are using?
Cheers
Oleg
On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
Hey folks,
The GetJMSQueue processor allows configuration of the Acknowledgment Mode.
Acknowledgement Mode
Client Acknowledge
• Client Acknowledge
• Auto Acknowledge
The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.
Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS processor?
The reason I ask is that I seem to be losing messages on NiFi restart.
Thanks,
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
Re: ConsumeJMS and Acknowledgement Mode
Posted by Oleg Zhurakousky <oz...@hortonworks.com>.
Chris
FWIW, the Get/PutJMS* are effectively deprecated processors since they only support one JMS provider (ActiiveMQ) and should not be used. Instead we recommend Publish/ConsumeJMS pair.
As to your question, which specific ACK mode you are inquiring about? As the doc currently states, the auto-ACK may loose messages. Is that what you are using?
Cheers
Oleg
On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com>> wrote:
Hey folks,
The GetJMSQueue processor allows configuration of the Acknowledgment Mode.
Acknowledgement Mode
Client Acknowledge
• Client Acknowledge
• Auto Acknowledge
The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.
Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS processor?
The reason I ask is that I seem to be losing messages on NiFi restart.
Thanks,
Chris McDermott
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315