You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by Sham Hassan <sc...@gmail.com> on 2016/02/04 04:22:00 UTC
How to view /:async node properties without shutting down the system
Hello Team,
To find cause for why async indexing "is broken” we need [A]. In order
to get :async is costly noice because we need to get production system down
from business approval. Any low level api script you can share to get the
same without getting system down?
[A]
1. 5 minutes of thread dumps one every 2 seconds
2. debug level and logs for the appender:
1. org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate and
2. org.apache.jackrabbit.oak.plugins.index.IndexUpdate.
3. collect data from the IndexStatsMbean.
4. list of checkpoints using the CheckpointManager MBean.
5. using the oak-console what is under /:async node.
Thanks,
Sham
Re: How to view /:async node properties without shutting down the system
Posted by Sham Hassan <sc...@gmail.com>.
thanks alex. This is first time I am analyzing to find cause for why async
indexing "is broken”. Following some internal documentation. Let me
research more & get back with details.
On Thu, Feb 4, 2016 at 4:20 AM, Alex Parvulescu <al...@gmail.com>
wrote:
> Hi Sham,
>
> About the ":async" properties, all of the important bits are already
> exposed via the 'IndexStatsMBean' [0], of interest are
> 'getReferenceCheckpoint' - the current reference, 'getProcessedCheckpoint'
> - the in progress checkpoint, and possibly the 'getTemporaryCheckpoints' -
> stuff that is queued to be cleaned up in the future. The lease mechanics is
> not there but I don't think is needed.
>
> Basically if there's a checkpoint under the CheckpointManager that's not in
> either 'processed', 'referenced' or 'temp', it's probably garbage and needs
> to be cleared manually.
> Also, of note is that you might have more than a single 'IndexStatsMBean'
> (there's one per indexing lane, by default there's an 'async' one, but
> depending on setup, you might find others as well), so be sure to collect
> all existing instances when debugging.
>
> Were you looking for other information that's missing?
>
> best,
> alex
>
>
> [0]
>
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/api/jmx/IndexStatsMBean.java#L102
>
>
>
>
> On Thu, Feb 4, 2016 at 4:22 AM, Sham Hassan <sc...@gmail.com> wrote:
>
> > Hello Team,
> >
> > To find cause for why async indexing "is broken” we need [A]. In order
> > to get :async is costly noice because we need to get production system
> down
> > from business approval. Any low level api script you can share to get the
> > same without getting system down?
> >
> >
> > [A]
> >
> > 1. 5 minutes of thread dumps one every 2 seconds
> > 2. debug level and logs for the appender:
> > 1. org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate and
> > 2. org.apache.jackrabbit.oak.plugins.index.IndexUpdate.
> > 3. collect data from the IndexStatsMbean.
> > 4. list of checkpoints using the CheckpointManager MBean.
> > 5. using the oak-console what is under /:async node.
> >
> >
> > Thanks,
> > Sham
> >
>
Re: How to view /:async node properties without shutting down the system
Posted by Alex Parvulescu <al...@gmail.com>.
Hi Sham,
About the ":async" properties, all of the important bits are already
exposed via the 'IndexStatsMBean' [0], of interest are
'getReferenceCheckpoint' - the current reference, 'getProcessedCheckpoint'
- the in progress checkpoint, and possibly the 'getTemporaryCheckpoints' -
stuff that is queued to be cleaned up in the future. The lease mechanics is
not there but I don't think is needed.
Basically if there's a checkpoint under the CheckpointManager that's not in
either 'processed', 'referenced' or 'temp', it's probably garbage and needs
to be cleared manually.
Also, of note is that you might have more than a single 'IndexStatsMBean'
(there's one per indexing lane, by default there's an 'async' one, but
depending on setup, you might find others as well), so be sure to collect
all existing instances when debugging.
Were you looking for other information that's missing?
best,
alex
[0]
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/api/jmx/IndexStatsMBean.java#L102
On Thu, Feb 4, 2016 at 4:22 AM, Sham Hassan <sc...@gmail.com> wrote:
> Hello Team,
>
> To find cause for why async indexing "is broken” we need [A]. In order
> to get :async is costly noice because we need to get production system down
> from business approval. Any low level api script you can share to get the
> same without getting system down?
>
>
> [A]
>
> 1. 5 minutes of thread dumps one every 2 seconds
> 2. debug level and logs for the appender:
> 1. org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate and
> 2. org.apache.jackrabbit.oak.plugins.index.IndexUpdate.
> 3. collect data from the IndexStatsMbean.
> 4. list of checkpoints using the CheckpointManager MBean.
> 5. using the oak-console what is under /:async node.
>
>
> Thanks,
> Sham
>