You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by Clebert Suconic <cl...@gmail.com> on 2024/11/20 03:55:47 UTC
[DISCUSS] Artemis: Address-setting # expiry-delay during expiry copy
Say you set the expiry delay on the address-settings.
address-settings.#.expiryDelay=5000
address-settings.#.expiryQueue=TheExpiryQueue
now, you send a message with an expiry delay as 2 seconds...
2 seconds from now, the message expiry will move the message TheExpiryQueue
The system is currently applying 5 seconds expiry on the message
routed to TheExpiryQueue. I even open a JIRA:
https://issues.apache.org/jira/browse/ARTEMIS-5167
I don't think this is correct, however an user could use the
unintended feature (aka bug) to set the expiration of the expired
message.
Should I avoid setting the expiry of the expiry? Does anyone see a
valid use case on such an unintended feature?
Currently AutoCreateExpiryResourcesTest::testWithJMSFQQN is failing
because of this.
--
Clebert Suconic
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@activemq.apache.org
For additional commands, e-mail: dev-help@activemq.apache.org
For further information, visit: https://activemq.apache.org/contact
Re: [DISCUSS] Artemis: Address-setting # expiry-delay during expiry copy
Posted by Clebert Suconic <cl...@gmail.com>.
I will keep current semantics. But add documentation and testing for
loops.
Clebert Suconic
On Wed, Nov 20, 2024 at 12:31 PM Clebert Suconic <cl...@gmail.com>
wrote:
> Regarding a loop, We do have a protection if the expiry queue is itself...
>
> It's just the expiry apply after the fact as you pointed out.
> Mr
>
> Someone could use that "feature" to say allow the message to disappear
> from the expiry queue after some time...
>
> or the same for the DLQ.. being a cleanup tool.
>
>
>
>
> I have a test failing that's failing because of this unintended
> configuration (AutoCreateExpiryResourcesTest::testWithJMSFQQN)
>
> And I could fix it (and add new tests covering the scenario) by not
> allowing the expiry of the expiry...
>
> Or we could embrace the scenario.. by also adding new tests validating
> the scenario of the expiry of the expiry.. or expiry of the DLQ.
>
>
>
> I was confused on which way to take... fixing seems breaking current
> semantics... (even though it was non intended semantics).
>
>
>
>
>
> Perhaps I could log an once log.info once expiry of an expiry is
> placed. or expiry of a DLQ also. As that would cover the unintended
> use.. while allowing intended cases.
>
> On Wed, Nov 20, 2024 at 11:42 AM Robbie Gemmell
> <ro...@gmail.com> wrote:
> >
> > You posted an example using catch-all wildcard config where the config
> > would have likely applied to both the original queue, and their expiry
> > queue, and in such a way it could cause re-expiry and would seem to
> > even create a loop as Domenico covered. I can certainly see argument
> > for that sometimes being unintended or unexpected. Though, I could
> > also see the view that it is just doing exactly what was configured in
> > the settings in the way the settings note they will.
> >
> > However what if someone instead explicitly configured [only] their
> > expiry queues settings to apply a certain additional expiration, e.g a
> > longer one than originally, and also e.g didnt have an expiry queue
> > set for their expiry queue? I could also see use cases like that,
> > where messages are originally expired into an expiry queue after a
> > shorter normal processing period, and then are allowed to sit there
> > for at most <longer time foo> for inspection etc before being expired
> > again into oblivion if not otherwise handled.
> >
> > Then much the same for the DLQ case. It similarly doesnt seem
> > unthinkable someone might want to (or may already) apply expiration to
> > those in some cases.
> >
> > Might not be the typical use cases, but it doesnt seem entirely unlikely.
> >
> > Would those still be possible with whatever you intend, or are you
> > just thinking on a blanket ignore of those settings for DLQ/expiry
> > queues?
> >
> >
> > On Wed, 20 Nov 2024 at 14:36, Clebert Suconic <cl...@gmail.com>
> wrote:
> > >
> > > I think it should not apply. But I wanted to check if anyone is
> against my
> > > opinion.
> > >
> > > Clebert Suconic
> > >
> > >
> > > On Wed, Nov 20, 2024 at 9:29 AM Clebert Suconic <
> clebert.suconic@gmail.com>
> > > wrote:
> > >
> > > > What about expiry of a DLQ?
> > > >
> > > > When moving a message to DLQ, the following code here would also
> > > > applyExpiration if addressSettings/expiry-delay is set on the DLQ
> > > > address:
> > > >
> > > >
> > > >
> https://github.com/apache/activemq-artemis/blob/b084eac71d168303019b799e0d267210c19677f7/artemis-server/src/main/java/org/apache/activemq/artemis/core/postoffice/impl/PostOfficeImpl.java#L1167-L1171
> > > >
> > > > On Wed, Nov 20, 2024 at 1:00 AM Domenico Francesco Bruscino
> > > > <br...@gmail.com> wrote:
> > > > >
> > > > > We should avoid the expiry of the expiry because it could cause
> the loss
> > > > of
> > > > > the original address/queue destination and infinite expiry loops.
> > > > >
> > > > > Domenico
> > > > >
> > > > > On Wed, 20 Nov 2024 at 04:56, Clebert Suconic <
> clebert.suconic@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Say you set the expiry delay on the address-settings.
> > > > > >
> > > > > >
> > > > > > address-settings.#.expiryDelay=5000
> > > > > > address-settings.#.expiryQueue=TheExpiryQueue
> > > > > >
> > > > > >
> > > > > > now, you send a message with an expiry delay as 2 seconds...
> > > > > >
> > > > > >
> > > > > > 2 seconds from now, the message expiry will move the message
> > > > TheExpiryQueue
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > The system is currently applying 5 seconds expiry on the message
> > > > > > routed to TheExpiryQueue. I even open a JIRA:
> > > > > > https://issues.apache.org/jira/browse/ARTEMIS-5167
> > > > > >
> > > > > >
> > > > > > I don't think this is correct, however an user could use the
> > > > > > unintended feature (aka bug) to set the expiration of the expired
> > > > > > message.
> > > > > >
> > > > > >
> > > > > > Should I avoid setting the expiry of the expiry? Does anyone see
> a
> > > > > > valid use case on such an unintended feature?
> > > > > >
> > > > > >
> > > > > > Currently AutoCreateExpiryResourcesTest::testWithJMSFQQN is
> failing
> > > > > > because of this.
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Clebert Suconic
> > > > > >
> > > > > >
> ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@activemq.apache.org
> > > > > > For additional commands, e-mail: dev-help@activemq.apache.org
> > > > > > For further information, visit:
> https://activemq.apache.org/contact
> > > > > >
> > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Clebert Suconic
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@activemq.apache.org
> > For additional commands, e-mail: dev-help@activemq.apache.org
> > For further information, visit: https://activemq.apache.org/contact
> >
> >
>
>
> --
> Clebert Suconic
>
Re: [DISCUSS] Artemis: Address-setting # expiry-delay during expiry copy
Posted by Clebert Suconic <cl...@gmail.com>.
Regarding a loop, We do have a protection if the expiry queue is itself...
It's just the expiry apply after the fact as you pointed out.
Someone could use that "feature" to say allow the message to disappear
from the expiry queue after some time...
or the same for the DLQ.. being a cleanup tool.
I have a test failing that's failing because of this unintended
configuration (AutoCreateExpiryResourcesTest::testWithJMSFQQN)
And I could fix it (and add new tests covering the scenario) by not
allowing the expiry of the expiry...
Or we could embrace the scenario.. by also adding new tests validating
the scenario of the expiry of the expiry.. or expiry of the DLQ.
I was confused on which way to take... fixing seems breaking current
semantics... (even though it was non intended semantics).
Perhaps I could log an once log.info once expiry of an expiry is
placed. or expiry of a DLQ also. As that would cover the unintended
use.. while allowing intended cases.
On Wed, Nov 20, 2024 at 11:42 AM Robbie Gemmell
<ro...@gmail.com> wrote:
>
> You posted an example using catch-all wildcard config where the config
> would have likely applied to both the original queue, and their expiry
> queue, and in such a way it could cause re-expiry and would seem to
> even create a loop as Domenico covered. I can certainly see argument
> for that sometimes being unintended or unexpected. Though, I could
> also see the view that it is just doing exactly what was configured in
> the settings in the way the settings note they will.
>
> However what if someone instead explicitly configured [only] their
> expiry queues settings to apply a certain additional expiration, e.g a
> longer one than originally, and also e.g didnt have an expiry queue
> set for their expiry queue? I could also see use cases like that,
> where messages are originally expired into an expiry queue after a
> shorter normal processing period, and then are allowed to sit there
> for at most <longer time foo> for inspection etc before being expired
> again into oblivion if not otherwise handled.
>
> Then much the same for the DLQ case. It similarly doesnt seem
> unthinkable someone might want to (or may already) apply expiration to
> those in some cases.
>
> Might not be the typical use cases, but it doesnt seem entirely unlikely.
>
> Would those still be possible with whatever you intend, or are you
> just thinking on a blanket ignore of those settings for DLQ/expiry
> queues?
>
>
> On Wed, 20 Nov 2024 at 14:36, Clebert Suconic <cl...@gmail.com> wrote:
> >
> > I think it should not apply. But I wanted to check if anyone is against my
> > opinion.
> >
> > Clebert Suconic
> >
> >
> > On Wed, Nov 20, 2024 at 9:29 AM Clebert Suconic <cl...@gmail.com>
> > wrote:
> >
> > > What about expiry of a DLQ?
> > >
> > > When moving a message to DLQ, the following code here would also
> > > applyExpiration if addressSettings/expiry-delay is set on the DLQ
> > > address:
> > >
> > >
> > > https://github.com/apache/activemq-artemis/blob/b084eac71d168303019b799e0d267210c19677f7/artemis-server/src/main/java/org/apache/activemq/artemis/core/postoffice/impl/PostOfficeImpl.java#L1167-L1171
> > >
> > > On Wed, Nov 20, 2024 at 1:00 AM Domenico Francesco Bruscino
> > > <br...@gmail.com> wrote:
> > > >
> > > > We should avoid the expiry of the expiry because it could cause the loss
> > > of
> > > > the original address/queue destination and infinite expiry loops.
> > > >
> > > > Domenico
> > > >
> > > > On Wed, 20 Nov 2024 at 04:56, Clebert Suconic <clebert.suconic@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > Say you set the expiry delay on the address-settings.
> > > > >
> > > > >
> > > > > address-settings.#.expiryDelay=5000
> > > > > address-settings.#.expiryQueue=TheExpiryQueue
> > > > >
> > > > >
> > > > > now, you send a message with an expiry delay as 2 seconds...
> > > > >
> > > > >
> > > > > 2 seconds from now, the message expiry will move the message
> > > TheExpiryQueue
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > The system is currently applying 5 seconds expiry on the message
> > > > > routed to TheExpiryQueue. I even open a JIRA:
> > > > > https://issues.apache.org/jira/browse/ARTEMIS-5167
> > > > >
> > > > >
> > > > > I don't think this is correct, however an user could use the
> > > > > unintended feature (aka bug) to set the expiration of the expired
> > > > > message.
> > > > >
> > > > >
> > > > > Should I avoid setting the expiry of the expiry? Does anyone see a
> > > > > valid use case on such an unintended feature?
> > > > >
> > > > >
> > > > > Currently AutoCreateExpiryResourcesTest::testWithJMSFQQN is failing
> > > > > because of this.
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Clebert Suconic
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@activemq.apache.org
> > > > > For additional commands, e-mail: dev-help@activemq.apache.org
> > > > > For further information, visit: https://activemq.apache.org/contact
> > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > > --
> > > Clebert Suconic
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@activemq.apache.org
> For additional commands, e-mail: dev-help@activemq.apache.org
> For further information, visit: https://activemq.apache.org/contact
>
>
--
Clebert Suconic
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@activemq.apache.org
For additional commands, e-mail: dev-help@activemq.apache.org
For further information, visit: https://activemq.apache.org/contact
Re: [DISCUSS] Artemis: Address-setting # expiry-delay during expiry copy
Posted by Robbie Gemmell <ro...@gmail.com>.
You posted an example using catch-all wildcard config where the config
would have likely applied to both the original queue, and their expiry
queue, and in such a way it could cause re-expiry and would seem to
even create a loop as Domenico covered. I can certainly see argument
for that sometimes being unintended or unexpected. Though, I could
also see the view that it is just doing exactly what was configured in
the settings in the way the settings note they will.
However what if someone instead explicitly configured [only] their
expiry queues settings to apply a certain additional expiration, e.g a
longer one than originally, and also e.g didnt have an expiry queue
set for their expiry queue? I could also see use cases like that,
where messages are originally expired into an expiry queue after a
shorter normal processing period, and then are allowed to sit there
for at most <longer time foo> for inspection etc before being expired
again into oblivion if not otherwise handled.
Then much the same for the DLQ case. It similarly doesnt seem
unthinkable someone might want to (or may already) apply expiration to
those in some cases.
Might not be the typical use cases, but it doesnt seem entirely unlikely.
Would those still be possible with whatever you intend, or are you
just thinking on a blanket ignore of those settings for DLQ/expiry
queues?
On Wed, 20 Nov 2024 at 14:36, Clebert Suconic <cl...@gmail.com> wrote:
>
> I think it should not apply. But I wanted to check if anyone is against my
> opinion.
>
> Clebert Suconic
>
>
> On Wed, Nov 20, 2024 at 9:29 AM Clebert Suconic <cl...@gmail.com>
> wrote:
>
> > What about expiry of a DLQ?
> >
> > When moving a message to DLQ, the following code here would also
> > applyExpiration if addressSettings/expiry-delay is set on the DLQ
> > address:
> >
> >
> > https://github.com/apache/activemq-artemis/blob/b084eac71d168303019b799e0d267210c19677f7/artemis-server/src/main/java/org/apache/activemq/artemis/core/postoffice/impl/PostOfficeImpl.java#L1167-L1171
> >
> > On Wed, Nov 20, 2024 at 1:00 AM Domenico Francesco Bruscino
> > <br...@gmail.com> wrote:
> > >
> > > We should avoid the expiry of the expiry because it could cause the loss
> > of
> > > the original address/queue destination and infinite expiry loops.
> > >
> > > Domenico
> > >
> > > On Wed, 20 Nov 2024 at 04:56, Clebert Suconic <clebert.suconic@gmail.com
> > >
> > > wrote:
> > >
> > > > Say you set the expiry delay on the address-settings.
> > > >
> > > >
> > > > address-settings.#.expiryDelay=5000
> > > > address-settings.#.expiryQueue=TheExpiryQueue
> > > >
> > > >
> > > > now, you send a message with an expiry delay as 2 seconds...
> > > >
> > > >
> > > > 2 seconds from now, the message expiry will move the message
> > TheExpiryQueue
> > > >
> > > >
> > > >
> > > >
> > > > The system is currently applying 5 seconds expiry on the message
> > > > routed to TheExpiryQueue. I even open a JIRA:
> > > > https://issues.apache.org/jira/browse/ARTEMIS-5167
> > > >
> > > >
> > > > I don't think this is correct, however an user could use the
> > > > unintended feature (aka bug) to set the expiration of the expired
> > > > message.
> > > >
> > > >
> > > > Should I avoid setting the expiry of the expiry? Does anyone see a
> > > > valid use case on such an unintended feature?
> > > >
> > > >
> > > > Currently AutoCreateExpiryResourcesTest::testWithJMSFQQN is failing
> > > > because of this.
> > > >
> > > >
> > > >
> > > > --
> > > > Clebert Suconic
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@activemq.apache.org
> > > > For additional commands, e-mail: dev-help@activemq.apache.org
> > > > For further information, visit: https://activemq.apache.org/contact
> > > >
> > > >
> > > >
> >
> >
> >
> > --
> > Clebert Suconic
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@activemq.apache.org
For additional commands, e-mail: dev-help@activemq.apache.org
For further information, visit: https://activemq.apache.org/contact
Re: [DISCUSS] Artemis: Address-setting # expiry-delay during expiry copy
Posted by Clebert Suconic <cl...@gmail.com>.
I think it should not apply. But I wanted to check if anyone is against my
opinion.
Clebert Suconic
On Wed, Nov 20, 2024 at 9:29 AM Clebert Suconic <cl...@gmail.com>
wrote:
> What about expiry of a DLQ?
>
> When moving a message to DLQ, the following code here would also
> applyExpiration if addressSettings/expiry-delay is set on the DLQ
> address:
>
>
> https://github.com/apache/activemq-artemis/blob/b084eac71d168303019b799e0d267210c19677f7/artemis-server/src/main/java/org/apache/activemq/artemis/core/postoffice/impl/PostOfficeImpl.java#L1167-L1171
>
> On Wed, Nov 20, 2024 at 1:00 AM Domenico Francesco Bruscino
> <br...@gmail.com> wrote:
> >
> > We should avoid the expiry of the expiry because it could cause the loss
> of
> > the original address/queue destination and infinite expiry loops.
> >
> > Domenico
> >
> > On Wed, 20 Nov 2024 at 04:56, Clebert Suconic <clebert.suconic@gmail.com
> >
> > wrote:
> >
> > > Say you set the expiry delay on the address-settings.
> > >
> > >
> > > address-settings.#.expiryDelay=5000
> > > address-settings.#.expiryQueue=TheExpiryQueue
> > >
> > >
> > > now, you send a message with an expiry delay as 2 seconds...
> > >
> > >
> > > 2 seconds from now, the message expiry will move the message
> TheExpiryQueue
> > >
> > >
> > >
> > >
> > > The system is currently applying 5 seconds expiry on the message
> > > routed to TheExpiryQueue. I even open a JIRA:
> > > https://issues.apache.org/jira/browse/ARTEMIS-5167
> > >
> > >
> > > I don't think this is correct, however an user could use the
> > > unintended feature (aka bug) to set the expiration of the expired
> > > message.
> > >
> > >
> > > Should I avoid setting the expiry of the expiry? Does anyone see a
> > > valid use case on such an unintended feature?
> > >
> > >
> > > Currently AutoCreateExpiryResourcesTest::testWithJMSFQQN is failing
> > > because of this.
> > >
> > >
> > >
> > > --
> > > Clebert Suconic
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@activemq.apache.org
> > > For additional commands, e-mail: dev-help@activemq.apache.org
> > > For further information, visit: https://activemq.apache.org/contact
> > >
> > >
> > >
>
>
>
> --
> Clebert Suconic
>
Re: [DISCUSS] Artemis: Address-setting # expiry-delay during expiry copy
Posted by Clebert Suconic <cl...@gmail.com>.
What about expiry of a DLQ?
When moving a message to DLQ, the following code here would also
applyExpiration if addressSettings/expiry-delay is set on the DLQ
address:
https://github.com/apache/activemq-artemis/blob/b084eac71d168303019b799e0d267210c19677f7/artemis-server/src/main/java/org/apache/activemq/artemis/core/postoffice/impl/PostOfficeImpl.java#L1167-L1171
On Wed, Nov 20, 2024 at 1:00 AM Domenico Francesco Bruscino
<br...@gmail.com> wrote:
>
> We should avoid the expiry of the expiry because it could cause the loss of
> the original address/queue destination and infinite expiry loops.
>
> Domenico
>
> On Wed, 20 Nov 2024 at 04:56, Clebert Suconic <cl...@gmail.com>
> wrote:
>
> > Say you set the expiry delay on the address-settings.
> >
> >
> > address-settings.#.expiryDelay=5000
> > address-settings.#.expiryQueue=TheExpiryQueue
> >
> >
> > now, you send a message with an expiry delay as 2 seconds...
> >
> >
> > 2 seconds from now, the message expiry will move the message TheExpiryQueue
> >
> >
> >
> >
> > The system is currently applying 5 seconds expiry on the message
> > routed to TheExpiryQueue. I even open a JIRA:
> > https://issues.apache.org/jira/browse/ARTEMIS-5167
> >
> >
> > I don't think this is correct, however an user could use the
> > unintended feature (aka bug) to set the expiration of the expired
> > message.
> >
> >
> > Should I avoid setting the expiry of the expiry? Does anyone see a
> > valid use case on such an unintended feature?
> >
> >
> > Currently AutoCreateExpiryResourcesTest::testWithJMSFQQN is failing
> > because of this.
> >
> >
> >
> > --
> > Clebert Suconic
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@activemq.apache.org
> > For additional commands, e-mail: dev-help@activemq.apache.org
> > For further information, visit: https://activemq.apache.org/contact
> >
> >
> >
--
Clebert Suconic
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@activemq.apache.org
For additional commands, e-mail: dev-help@activemq.apache.org
For further information, visit: https://activemq.apache.org/contact
Re: [DISCUSS] Artemis: Address-setting # expiry-delay during expiry copy
Posted by Domenico Francesco Bruscino <br...@gmail.com>.
We should avoid the expiry of the expiry because it could cause the loss of
the original address/queue destination and infinite expiry loops.
Domenico
On Wed, 20 Nov 2024 at 04:56, Clebert Suconic <cl...@gmail.com>
wrote:
> Say you set the expiry delay on the address-settings.
>
>
> address-settings.#.expiryDelay=5000
> address-settings.#.expiryQueue=TheExpiryQueue
>
>
> now, you send a message with an expiry delay as 2 seconds...
>
>
> 2 seconds from now, the message expiry will move the message TheExpiryQueue
>
>
>
>
> The system is currently applying 5 seconds expiry on the message
> routed to TheExpiryQueue. I even open a JIRA:
> https://issues.apache.org/jira/browse/ARTEMIS-5167
>
>
> I don't think this is correct, however an user could use the
> unintended feature (aka bug) to set the expiration of the expired
> message.
>
>
> Should I avoid setting the expiry of the expiry? Does anyone see a
> valid use case on such an unintended feature?
>
>
> Currently AutoCreateExpiryResourcesTest::testWithJMSFQQN is failing
> because of this.
>
>
>
> --
> Clebert Suconic
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@activemq.apache.org
> For additional commands, e-mail: dev-help@activemq.apache.org
> For further information, visit: https://activemq.apache.org/contact
>
>
>