You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@activemq.apache.org by GitBox <gi...@apache.org> on 2023/01/05 21:04:13 UTC

[GitHub] [activemq] maxfortun opened a new pull request, #949: Track scheduler activity

maxfortun opened a new pull request, #949:
URL: https://github.com/apache/activemq/pull/949

   Augment scheduler to forward scheduled and dispatched messages to a destination, if such is configured.
   e.g.
   <broker ... schedulerSupport="true" schedulerActivityDestination="topic://ActiveMQ.Scheduler.Activity" >
   If schedulerActivityDestination is not set, current behavior is unaffected.
   Otherwise...
   When message is successfully scheduled scheduler should set property AMQ_SCHEDULER_ACTIVITY to SCHEDULED and forward scheduled message to the schedulerActivityDestination
   When message is successfully dispatched scheduler should set property AMQ_SCHEDULER_ACTIVITY to DISPATCHED and forward scheduled message to the schedulerActivityDestination


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] maxfortun commented on pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
maxfortun commented on PR #949:
URL: https://github.com/apache/activemq/pull/949#issuecomment-1374021941

   > @maxfortun are you able to share any more details about the use case? I'm pretty sure a standard plugin could support this without needing to modify broker internals or the current scheduler code.
   
   @mattrpav we are streaming messages and scheduling some of them for a delayed delivery. We sometimes need to cancel or replace scheduled messages with their latest versions. Currently the only way to interact with the scheduled messages is via a job id. The problem is that the job id is not available until the message is inside the scheduler. The only way to get the job id is to keep polling the broker for its scheduled jobs. Not only this is very inefficient and resource intensive when there are many scheduled messages, but also there is no way to react in event driven fashion to a message that was just scheduled until it was actually dispatched. By sending activity advisory messages with the job Id and group id I can have a pub-sub system that is always in sync with the scheduler and allows me to remove the scheduled jobs by mapping some attributes, like a group id to its job id, and ultimately cancel the job associated with the specific message. I am open to any other means of 
 achieving the same. Although, this feature seems like rather harmless, as it is fully backward compatible and does not get turned on unless explicitly configured.   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] mattrpav commented on pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
mattrpav commented on PR #949:
URL: https://github.com/apache/activemq/pull/949#issuecomment-1374064230

   > I also initially thought that the JobListener is the correct place for this feature, but since I am not familiar enough with the broker internals and maintainers philosophy I did not want to make any changes that are not backward compatible. If we change the JobListener interface then the upgrades for people who have any custom scheduler modules for the broker will no longer be a drop in easy-peasy process and would require quite a bit of work. Which may be ok, if that's how activemq is developed. Also, currently there is only 1 listener, and it is the `SchedulerBroker`, which is added and removed when broker starts and stops. I presume this is for the failover support. Using job listeners may make this feature more of a rearchitecture than a feature. I am not certain that this deserves so much effort. Thoughts?
   
   We can add a default no-op method to fix the upgrade compatibility issue.
   
   I think overall, adding a couple methods to existing internal interface is preferred over adding a new interface, config, and other internal changes. Also, the approach of only having one handler is limiting.
   
   Note-- the existing SchedulerBroker implements the JobListener only to have a handle to internal broker for publishing the message. It is an o-o hack of sorts to reduce the need to inject a BrokerService to the JobListener to start deliverying messages.
   
   Once the interface is updated, adding an AdvisoryJobListener would be trivial, and you could still add add'l listeners to do pub-sub stuff for your internal job track sync tool.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] mattrpav commented on pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
mattrpav commented on PR #949:
URL: https://github.com/apache/activemq/pull/949#issuecomment-1374104344

   Yeah, I just can't get there with the current approach. I think this could all be handled within the JobScheduler/JobSchedulerFacade and not have to bubble it all the way up to BrokerService.
   
   I'd vote -1 for this change as-is.
   
   Pinging @cshannon to see if he has any add'l thoughts.
   
   My suggested approaches:
   1. Extend current JobListener
   2. Extend advisory support to send an advisory message w/ the job id, message id and target destination (this would be add add'l broker filter methods, and then support in the AdvisoryBroker so it can be turned off when advisory support is disabled)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] mattrpav commented on pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
mattrpav commented on PR #949:
URL: https://github.com/apache/activemq/pull/949#issuecomment-1374040673

   @maxfortun sounds like the need is for additional extensions for registerJob (and for completeness, we should add unregisterJob). 
   
   There is already a JobListener interface that has an operation to catch when scheduled messages are fired. I'm wondering if we could extend that by adding the 2 add'l methods.
   
   Then from there, confirm wiring in a custom JobListener (aka a mini-plugin just for Job events).
   
   Thoughts?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] maxfortun commented on a diff in pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
maxfortun commented on code in PR #949:
URL: https://github.com/apache/activemq/pull/949#discussion_r1063715742


##########
activemq-broker/src/main/java/org/apache/activemq/broker/scheduler/SchedulerBroker.java:
##########
@@ -353,6 +355,12 @@ private void doSchedule(Message messageSend, Object cronValue, Object periodValu
 
         getInternalScheduler().schedule(jobId.toString(),
                 new ByteSequence(packet.data, packet.offset, packet.length), cronEntry, delay, period, repeat);
+
+        if(jobSchedulerActivityListener != null) {
+            msg.setProperty(ScheduledMessage.AMQ_SCHEDULED_ID, jobId.toString());
+            msg.setProperty(ScheduledMessage.AMQ_SCHEDULER_ACTIVITY, ScheduledMessage.AMQ_SCHEDULER_ACTIVITY_SCHEDULED);
+            jobSchedulerActivityListener.scheduled(this, msg);
+        }

Review Comment:
   @mattrpav this block is the main reason for a PR. As there is no way to get to the job id at the time of the scheduling.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] maxfortun commented on pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
maxfortun commented on PR #949:
URL: https://github.com/apache/activemq/pull/949#issuecomment-1374072705

   There is one other issue with this approach. In its current design the JobListener is blind to the message and its attributes, it treats it as a binary payload. It may be quite a lift to wire it for what I need to happen. Which is to forward a meaningful activity message. As far as multiple activity trackers are concerned, it is rather straightforward to implement a  class that wraps many trackers into one.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] mattrpav commented on a diff in pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
mattrpav commented on code in PR #949:
URL: https://github.com/apache/activemq/pull/949#discussion_r1062998856


##########
activemq-broker/src/main/java/org/apache/activemq/broker/BrokerService.java:
##########
@@ -231,6 +231,7 @@ public class BrokerService implements Service {
     private boolean forceStart = false;
     private IOExceptionHandler ioExceptionHandler;
     private boolean schedulerSupport = false;
+    private String schedulerActivityDestination = null;

Review Comment:
   Please add a default value from a constant



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] maxfortun commented on a diff in pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
maxfortun commented on code in PR #949:
URL: https://github.com/apache/activemq/pull/949#discussion_r1063713058


##########
activemq-broker/src/main/java/org/apache/activemq/broker/scheduler/SchedulerBroker.java:
##########
@@ -353,6 +355,12 @@ private void doSchedule(Message messageSend, Object cronValue, Object periodValu
 
         getInternalScheduler().schedule(jobId.toString(),
                 new ByteSequence(packet.data, packet.offset, packet.length), cronEntry, delay, period, repeat);
+

Review Comment:
   @mattrpav this block is the main reason for a PR. As there is no way to get to the job id at the time of the scheduling.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] maxfortun commented on a diff in pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
maxfortun commented on code in PR #949:
URL: https://github.com/apache/activemq/pull/949#discussion_r1063716430


##########
activemq-broker/src/main/java/org/apache/activemq/broker/scheduler/SchedulerBroker.java:
##########
@@ -418,6 +426,12 @@ public void scheduledJob(String id, ByteSequence job) {
             producerExchange.setMutable(true);
             producerExchange.setProducerState(new ProducerState(new ProducerInfo()));
             super.send(producerExchange, messageSend);
+
+            if(jobSchedulerActivityListener != null) {
+                Message msg = messageSend.copy();
+                msg.setProperty(ScheduledMessage.AMQ_SCHEDULER_ACTIVITY, ScheduledMessage.AMQ_SCHEDULER_ACTIVITY_DISPATCHED);
+                jobSchedulerActivityListener.dispatched(this, msg);
+            }

Review Comment:
   @mattrpav Once message is dispatched it has scheduledJobId property, so I can use that, but I figured it is nice to have a single source to monitor scheduler activity.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] maxfortun commented on pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
maxfortun commented on PR #949:
URL: https://github.com/apache/activemq/pull/949#issuecomment-1374092035

   Something like this:
   ```xml
           <jobSchedulerActivityListener>
               <bean xmlns="http://www.springframework.org/schema/beans" id="jobSchedulerMultipleActivityListeners"
                   class="org.apache.activemq.broker.scheduler.JobSchedulerMultipleActivityListeners">
                   <property name="listeners">
                       <list>
                           <bean xmlns="http://www.springframework.org/schema/beans"
                               id="jobSchedulerForwardingActivityListener"
                               class="org.apache.activemq.broker.scheduler.JobSchedulerForwardingActivityListener" />
                       </list>
                   </property>
               </bean>
           </jobSchedulerActivityListener>
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] maxfortun commented on a diff in pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
maxfortun commented on code in PR #949:
URL: https://github.com/apache/activemq/pull/949#discussion_r1063714013


##########
activemq-broker/src/main/java/org/apache/activemq/broker/scheduler/SchedulerBroker.java:
##########
@@ -418,6 +426,12 @@ public void scheduledJob(String id, ByteSequence job) {
             producerExchange.setMutable(true);
             producerExchange.setProducerState(new ProducerState(new ProducerInfo()));
             super.send(producerExchange, messageSend);
+

Review Comment:
   @mattrpav Once message is dispatched it has scheduledJobId property, so I can use that, but I figured it is nice to have a single source to monitor scheduler activity.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] mattrpav commented on pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
mattrpav commented on PR #949:
URL: https://github.com/apache/activemq/pull/949#issuecomment-1374009066

   @maxfortun are you able to share any more details about the use case? I'm pretty sure a standard plugin could support this without needing to modify broker internals or the current scheduler code.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] maxfortun commented on pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
maxfortun commented on PR #949:
URL: https://github.com/apache/activemq/pull/949#issuecomment-1374049087

   > @maxfortun sounds like the need is for additional extensions for registerJob (and for completeness, we should add unregisterJob).
   > 
   > There is already a JobListener interface that has an operation to catch when scheduled messages are fired. I'm wondering if we could extend that by adding the 2 add'l methods.
   > 
   > Then from there, confirm wiring in a custom JobListener (aka a mini-plugin just for Job events).
   > 
   > Thoughts?
   
   I also initially thought that the JobListener is the correct place for this feature, but since I am not familiar enough with the broker internals and maintainers philosophy I did not want to make any changes that are not backward compatible. If we change the JobListener interface then the upgrades for people who have any custom scheduler modules for the broker will no longer be a drop in easy-peasy process and would require quite a bit of work. Which may be ok, if that's how activemq is developed. Also, currently there is only 1 listener, and it is the `SchedulerBroker`, which is added and removed when broker starts and stops. I presume this is for the failover support. Using job listeners may make this feature more of a rearchitecture than a feature. I am not certain that this deserves so much effort. Thoughts?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] maxfortun commented on pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
maxfortun commented on PR #949:
URL: https://github.com/apache/activemq/pull/949#issuecomment-1381016813

   Closing this PR as a leaner approach was suggested. Submitting #950 as a proposal in continuation of this discussion.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [activemq] maxfortun closed pull request #949: Track scheduler activity

Posted by GitBox <gi...@apache.org>.
maxfortun closed pull request #949: Track scheduler activity
URL: https://github.com/apache/activemq/pull/949


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org