Sling job distribution across Cluster not sharing the same repo...

[Nono Junang <no...@insideaem.com>]

Hello there,
I am trying to test the distribution of sling jobs across clusters which do
NOT share the same repository.

I have set up 3 instances I1, I2 and I3 each alone in its cluster and I
have configured *org.apache.sling.discovery.oak.Config* on I2 and I3 to
have the topology connector url point to I1.

A job consumer for topic T1 is running on I1 and when I open
/system/console/slingevent on I2 or I3 I can see that the topic T1 is
displayed and listed under the sling id of I1. So topology is working.

When I start a job for example on I2 with topic T1, the job consumer for T1
on I1 is not executed.
But in the I2 repo I can see that a node is created for that job:
/var/eventing/jobs/assigned/*<I1-sling-id>*/*<T1>*/<year>/<month>/<day>/<hour>/<minute/<id>
and it looks like it is properly assigned to I1.
But as said the job consumer on I1 is not executed.

Could you please first confirm that job distribution across clusters that
do NOT share the same repository is supported? Reading through slides from
Carsten Ziegeler on slideshare, I had the impression that it is supported.
If supported could you please tell me what I am doing wrong/missing?

Regards,

-- 
Nono Junang
Technical Director

InsideAEM Limited
38 Waingaro Lane, RD 3
Kerikeri 0293

M +64 21 400 319
W www.insideaem.com

Re: Sling job distribution across Cluster not sharing the same repo...

[Carsten Ziegeler <cz...@apache.org>]

Hi,

the job implementation is assuming that all instances visible through 
discovery share the same repository. The distribution of jobs across 
instances works my moving them to

/var/eventing/jobs/assigned/*<sling-id>*/

and then the instance with that sling-id picks it up from the repository.

For your use case you would need to move the jobs stored at that place 
to the right repository - which comes with several challenges. It's 
probably easier to share the repository across all instances and form a 
single cluster.

Regards
Carsten

Am 13.04.2021 um 12:10 schrieb Nono Junang:
> Hello there,
> I am trying to test the distribution of sling jobs across clusters which do
> NOT share the same repository.
> 
> I have set up 3 instances I1, I2 and I3 each alone in its cluster and I
> have configured *org.apache.sling.discovery.oak.Config* on I2 and I3 to
> have the topology connector url point to I1.
> 
> A job consumer for topic T1 is running on I1 and when I open
> /system/console/slingevent on I2 or I3 I can see that the topic T1 is
> displayed and listed under the sling id of I1. So topology is working.
> 
> When I start a job for example on I2 with topic T1, the job consumer for T1
> on I1 is not executed.
> But in the I2 repo I can see that a node is created for that job:
> /var/eventing/jobs/assigned/*<I1-sling-id>*/*<T1>*/<year>/<month>/<day>/<hour>/<minute/<id>
> and it looks like it is properly assigned to I1.
> But as said the job consumer on I1 is not executed.
> 
> Could you please first confirm that job distribution across clusters that
> do NOT share the same repository is supported? Reading through slides from
> Carsten Ziegeler on slideshare, I had the impression that it is supported.
> If supported could you please tell me what I am doing wrong/missing?
> 
> Regards,
> 

-- 
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org