You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@sling.apache.org by Stefan Seifert <ss...@pro-vision.de> on 2017/08/30 10:26:12 UTC

Sling Content Distribution: how to use "in-file" queue provider

this presentation [1] mentions that it's possible to switch the Sling Content Distribution from using a Sling Jobs-based queue to an "in-file" or "in-memory" queue which is much faster.

but i've not found any hint in the documentation what steps are needed to switch to this queue. can you give some hints?

is it recommended to use this alternative queue in production (if the possible loss of some events up to the last checkpoint is acceptable)? is this the implementation of it [2]? the javadoc states "queue not persisted on the repository and therefore not usable for production."

stefan

[1] https://docs.adobe.com/ddc/en/gems/Troubleshooting-Sling-Content-Distribution.html
[2] https://github.com/apache/sling/blob/trunk/contrib/extensions/distribution/core/src/main/java/org/apache/sling/distribution/queue/impl/simple/SimpleDistributionQueue.java

RE: Sling Content Distribution: how to use "in-file" queue provider

Posted by Stefan Seifert <ss...@pro-vision.de>.

thanks for the response! - we will keep this in mind.

stefan

>-----Original Message-----
>From: Timothee Maret [mailto:tmaret@apache.org]
>Sent: Monday, September 11, 2017 11:34 AM
>To: users@sling.apache.org
>Subject: Re: Sling Content Distribution: how to use "in-file" queue
>provider
>
>Hi,
>
>Sorry for late reply, I was on PTO. The In-file queue is only available for
>Forward agents.
>
>To enable it, you need to set the 'queue.provider' property to 'In-file' in
>your o.a.s.d.a.i.ForwardDistributionAgentFactory configuration.
>
>The checkpoint mechanism should be implemented by [0] despite the comments
>in SimpleDistributionQueue.
>
>I don't think there's yet a customer with this setting. It should really be
>used only if measurements prove that the queue is the bottleneck in your
>setup, that throughput is a real issue in your deployment, that your
>deployment can afford losing non checkpointed items and that you setup
>custom operations to backup the queue.
>
>Regards,
>
>Timothee
>
>[0]
>https://github.com/apache/sling/blob/trunk/contrib/extensions/distribution/
>core/src/main/java/org/apache/sling/distribution/queue/impl/simple/SimpleDi
>stributionQueueCheckpoint.java
>
>2017-08-30 12:26 GMT+02:00 Stefan Seifert <ss...@pro-vision.de>:
>
>> this presentation [1] mentions that it's possible to switch the Sling
>> Content Distribution from using a Sling Jobs-based queue to an "in-file"
>or
>> "in-memory" queue which is much faster.
>>
>> but i've not found any hint in the documentation what steps are needed to
>> switch to this queue. can you give some hints?
>>
>> is it recommended to use this alternative queue in production (if the
>> possible loss of some events up to the last checkpoint is acceptable)? is
>> this the implementation of it [2]? the javadoc states "queue not
>persisted
>> on the repository and therefore not usable for production."
>>
>> stefan
>>
>> [1] https://docs.adobe.com/ddc/en/gems/Troubleshooting-Sling-
>> Content-Distribution.html
>> [2] https://github.com/apache/sling/blob/trunk/contrib/
>> extensions/distribution/core/src/main/java/org/apache/
>> sling/distribution/queue/impl/simple/SimpleDistributionQueue.java
>>
>>
>>

Re: Sling Content Distribution: how to use "in-file" queue provider

Posted by Timothee Maret <tm...@apache.org>.

Hi,

Sorry for late reply, I was on PTO. The In-file queue is only available for
Forward agents.

To enable it, you need to set the 'queue.provider' property to 'In-file' in
your o.a.s.d.a.i.ForwardDistributionAgentFactory configuration.

The checkpoint mechanism should be implemented by [0] despite the comments
in SimpleDistributionQueue.

I don't think there's yet a customer with this setting. It should really be
used only if measurements prove that the queue is the bottleneck in your
setup, that throughput is a real issue in your deployment, that your
deployment can afford losing non checkpointed items and that you setup
custom operations to backup the queue.

Regards,

Timothee

[0]
https://github.com/apache/sling/blob/trunk/contrib/extensions/distribution/core/src/main/java/org/apache/sling/distribution/queue/impl/simple/SimpleDistributionQueueCheckpoint.java

2017-08-30 12:26 GMT+02:00 Stefan Seifert <ss...@pro-vision.de>:

> this presentation [1] mentions that it's possible to switch the Sling
> Content Distribution from using a Sling Jobs-based queue to an "in-file" or
> "in-memory" queue which is much faster.
>
> but i've not found any hint in the documentation what steps are needed to
> switch to this queue. can you give some hints?
>
> is it recommended to use this alternative queue in production (if the
> possible loss of some events up to the last checkpoint is acceptable)? is
> this the implementation of it [2]? the javadoc states "queue not persisted
> on the repository and therefore not usable for production."
>
> stefan
>
> [1] https://docs.adobe.com/ddc/en/gems/Troubleshooting-Sling-
> Content-Distribution.html
> [2] https://github.com/apache/sling/blob/trunk/contrib/
> extensions/distribution/core/src/main/java/org/apache/
> sling/distribution/queue/impl/simple/SimpleDistributionQueue.java
>
>
>