You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@tika.apache.org by Subhajit Das <Su...@live.com> on 2021/03/15 08:06:19 UTC
Prometheus exporter for TikaServer
Hi,
Is there any Prometheus exporter functionally, on TikaServer?
Need details on status endpoint over Prometheus as well.
Then have to setup a Grafana dashboard for the same.
Thanks in advance.
RE: Prometheus exporter for TikaServer
Posted by Subhajit Das <su...@live.com>.
Hi,
Raised https://issues.apache.org/jira/browse/TIKA-3353 for out of the box Micrometer based monitoring system, exported to Prometheus metrics.
Will raise PR soon.
Thanks and Regards,
Subhajit Das
From: Tim Allison<ma...@apache.org>
Sent: 12 April 2021 06:30 PM
To: Subhajit Das<ma...@live.com>; <de...@tika.apache.org>
Subject: Re: Prometheus exporter for TikaServer
Thank you! I updated JIRA so that 1.26 is now "released". I'm also in the process of merging 2.0 into 2.0.0.
On Mon, Apr 12, 2021 at 8:20 AM Subhajit Das <su...@live.com>> wrote:
Hi Tim,
Updated documentation. V14
[cid:image002.png@01D73175.135B6390]
Thanks and Regards,
Subhajit Das
From: Tim Allison<ma...@apache.org>
Sent: 12 April 2021 03:44 AM
To: Subhajit Das<ma...@live.com>
Cc: user@tika.apache.org<ma...@tika.apache.org>
Subject: Re: Prometheus exporter for TikaServer
I may have just twiddled the permission bits correctly. Let me know if not. Thank you!!!
On Sun, Apr 11, 2021 at 5:38 PM Subhajit Das <su...@live.com>> wrote:
Hi Tim,
Thanks for merging the PR for this. But, CWIKI has to be updated for the same.
It seems that, there is even no mention for /status endpoint in https://cwiki.apache.org/confluence/display/TIKA/TikaServer.
I can make the necessary changes. Write access required for same.
subhajitdas298 is my username for cwiki.
Thanks and Regards,
Subhajit Das
From: Subhajit Das<ma...@live.com>
Sent: 17 March 2021 09:39 AM
To: user@tika.apache.org<ma...@tika.apache.org>; tallison@apache.org<ma...@apache.org>
Cc: user@tika.apache.org<ma...@tika.apache.org>
Subject: RE: Prometheus exporter for TikaServer
Raised https://issues.apache.org/jira/browse/TIKA-3327 for Simple server monitoring, for 1.x branch. This should have to no performance impact, as exposed over mbean.
From: Subhajit Das<ma...@live.com>
Sent: 17 March 2021 08:39 AM
To: user@tika.apache.org<ma...@tika.apache.org>; tallison@apache.org<ma...@apache.org>
Cc: user@tika.apache.org<ma...@tika.apache.org>
Subject: RE: Prometheus exporter for TikaServer
Hi Tim,
OpenAPI is for live API, usage + documentation. OpenAPI can replace the “/” welcome page (where all APIs are listed) and give a much more modern experience.
While metrics monitoring is a separate concern. This, exposes the metrics data, like how many requests, how much time, jvm metrics, other custom metrics, like how many files processed etc.
Here we would have to inject the metric gauges programmatically. And let micrometer expose the value.
Note: Monitoring always introduces a minor (some times negligible) performance overhead. So, an option has to be given to switch on/off monitoring. Just like Solr does. But, it can be simple as a command line flag.
Thanks and Regards,
Subhajit
From: Tim Allison<ma...@apache.org>
Sent: 16 March 2021 11:40 PM
To: Subhajit Das<ma...@live.com>
Cc: user@tika.apache.org<ma...@tika.apache.org>
Subject: Re: Prometheus exporter for TikaServer
Interesting...Does this play at all with OpenAPI?
https://issues.apache.org/jira/browse/TIKA-3082
On Tue, Mar 16, 2021 at 1:35 PM Subhajit Das <su...@live.com>> wrote:
>
> Hi Tim,
>
>
>
> Prometheus requires data in it’s format. It seems that Tika does not have extended metrics collection system (correct me, if I am wrong).
>
> The status is storing status in ServerStatus object, which dosen’t seems to be exposed as MBean.
>
> If this is exposed as MBean, basic requirement may be fulfilled with Prometheus JMX Exporter (https://github.com/prometheus/jmx_exporter) as a Javaagent.
>
>
>
> For more advanced,
>
> Solr uses Dropwizard (https://solr.apache.org/guide/8_8/metrics-reporting.html) for metrics, and a separate Prometheus exporter (https://solr.apache.org/guide/8_8/monitoring-solr-with-prometheus-and-grafana.html) to export metrics to Prometheus.
>
>
>
> But, a better solution might be to use Micrometer (https://micrometer.io/), as it supports inbuild exporting for multiple standards, and this is also standard for new Spring Boot 2.0 projects.
>
>
>
> Thanks,
>
> Subhajit
>
>
>
> From: Tim Allison
> Sent: 16 March 2021 09:32 PM
> To: user@tika.apache.org<ma...@tika.apache.org>
> Subject: Re: Prometheus exporter for TikaServer
>
>
>
> We have only a barebones /status endpoint at the moment. You have to
> configure it/turn it on. It is not available by default. If there
> are specific statistics or a specific format you need for integration
> with Prometheus, please let us know.
>
> On Mon, Mar 15, 2021 at 4:06 AM Subhajit Das <Su...@live.com>> wrote:
> >
> > Hi,
> >
> >
> >
> > Is there any Prometheus exporter functionally, on TikaServer?
> >
> > Need details on status endpoint over Prometheus as well.
> >
> >
> >
> > Then have to setup a Grafana dashboard for the same.
> >
> >
> >
> > Thanks in advance.
> >
> >
>
>
Re: Prometheus exporter for TikaServer
Posted by Tim Allison <ta...@apache.org>.
Thank you! I updated JIRA so that 1.26 is now "released". I'm also in the
process of merging 2.0 into 2.0.0.
On Mon, Apr 12, 2021 at 8:20 AM Subhajit Das <su...@live.com>
wrote:
> Hi Tim,
>
>
>
> Updated documentation. V14
>
>
>
> Thanks and Regards,
>
> Subhajit Das
>
>
>
> *From: *Tim Allison <ta...@apache.org>
> *Sent: *12 April 2021 03:44 AM
> *To: *Subhajit Das <su...@live.com>
> *Cc: *user@tika.apache.org
> *Subject: *Re: Prometheus exporter for TikaServer
>
>
>
> I may have just twiddled the permission bits correctly. Let me know if
> not. Thank you!!!
>
>
>
> On Sun, Apr 11, 2021 at 5:38 PM Subhajit Das <su...@live.com>
> wrote:
>
> Hi Tim,
>
>
>
> Thanks for merging the PR for this. But, CWIKI has to be updated for the
> same.
>
>
>
> It seems that, there is even no mention for /status endpoint in
> https://cwiki.apache.org/confluence/display/TIKA/TikaServer.
>
>
>
> I can make the necessary changes. Write access required for same.
>
>
>
> subhajitdas298 is my username for cwiki.
>
>
>
> Thanks and Regards,
>
> Subhajit Das
>
>
>
> *From: *Subhajit Das <su...@live.com>
> *Sent: *17 March 2021 09:39 AM
> *To: *user@tika.apache.org; tallison@apache.org
> *Cc: *user@tika.apache.org
> *Subject: *RE: Prometheus exporter for TikaServer
>
>
>
> Raised https://issues.apache.org/jira/browse/TIKA-3327 for Simple server
> monitoring, for 1.x branch. This should have to no performance impact, as
> exposed over mbean.
>
>
>
> *From: *Subhajit Das <su...@live.com>
> *Sent: *17 March 2021 08:39 AM
> *To: *user@tika.apache.org; tallison@apache.org
> *Cc: *user@tika.apache.org
> *Subject: *RE: Prometheus exporter for TikaServer
>
>
>
> Hi Tim,
>
>
>
> OpenAPI is for live API, usage + documentation. OpenAPI can replace the
> “/” welcome page (where all APIs are listed) and give a much more modern
> experience.
>
>
>
> While metrics monitoring is a separate concern. This, exposes the metrics
> data, like how many requests, how much time, jvm metrics, other custom
> metrics, like how many files processed etc.
>
>
>
> Here we would have to inject the metric gauges programmatically. And let
> micrometer expose the value.
>
>
>
> Note: Monitoring always introduces a minor (some times negligible)
> performance overhead. So, an option has to be given to switch on/off
> monitoring. Just like Solr does. But, it can be simple as a command line
> flag.
>
>
>
> Thanks and Regards,
>
> Subhajit
>
>
>
> *From: *Tim Allison <ta...@apache.org>
> *Sent: *16 March 2021 11:40 PM
> *To: *Subhajit Das <su...@live.com>
> *Cc: *user@tika.apache.org
> *Subject: *Re: Prometheus exporter for TikaServer
>
>
>
> Interesting...Does this play at all with OpenAPI?
> https://issues.apache.org/jira/browse/TIKA-3082
>
> On Tue, Mar 16, 2021 at 1:35 PM Subhajit Das <su...@live.com>
> wrote:
> >
> > Hi Tim,
> >
> >
> >
> > Prometheus requires data in it’s format. It seems that Tika does not
> have extended metrics collection system (correct me, if I am wrong).
> >
> > The status is storing status in ServerStatus object, which dosen’t
> seems to be exposed as MBean.
> >
> > If this is exposed as MBean, basic requirement may be fulfilled with
> Prometheus JMX Exporter (https://github.com/prometheus/jmx_exporter) as a
> Javaagent.
> >
> >
> >
> > For more advanced,
> >
> > Solr uses Dropwizard (
> https://solr.apache.org/guide/8_8/metrics-reporting.html) for metrics,
> and a separate Prometheus exporter (
> https://solr.apache.org/guide/8_8/monitoring-solr-with-prometheus-and-grafana.html)
> to export metrics to Prometheus.
> >
> >
> >
> > But, a better solution might be to use Micrometer (
> https://micrometer.io/), as it supports inbuild exporting for multiple
> standards, and this is also standard for new Spring Boot 2.0 projects.
> >
> >
> >
> > Thanks,
> >
> > Subhajit
> >
> >
> >
> > From: Tim Allison
> > Sent: 16 March 2021 09:32 PM
> > To: user@tika.apache.org
> > Subject: Re: Prometheus exporter for TikaServer
> >
> >
> >
> > We have only a barebones /status endpoint at the moment. You have to
> > configure it/turn it on. It is not available by default. If there
> > are specific statistics or a specific format you need for integration
> > with Prometheus, please let us know.
> >
> > On Mon, Mar 15, 2021 at 4:06 AM Subhajit Das <Su...@live.com>
> wrote:
> > >
> > > Hi,
> > >
> > >
> > >
> > > Is there any Prometheus exporter functionally, on TikaServer?
> > >
> > > Need details on status endpoint over Prometheus as well.
> > >
> > >
> > >
> > > Then have to setup a Grafana dashboard for the same.
> > >
> > >
> > >
> > > Thanks in advance.
> > >
> > >
> >
> >
>
>
>
>
>
>
>
>
>
RE: Prometheus exporter for TikaServer
Posted by Subhajit Das <su...@live.com>.
Hi Tim,
Updated documentation. V14
[cid:image003.png@01D72FC4.37AF9BB0]
Thanks and Regards,
Subhajit Das
From: Tim Allison<ma...@apache.org>
Sent: 12 April 2021 03:44 AM
To: Subhajit Das<ma...@live.com>
Cc: user@tika.apache.org<ma...@tika.apache.org>
Subject: Re: Prometheus exporter for TikaServer
I may have just twiddled the permission bits correctly. Let me know if not. Thank you!!!
On Sun, Apr 11, 2021 at 5:38 PM Subhajit Das <su...@live.com>> wrote:
Hi Tim,
Thanks for merging the PR for this. But, CWIKI has to be updated for the same.
It seems that, there is even no mention for /status endpoint in https://cwiki.apache.org/confluence/display/TIKA/TikaServer.
I can make the necessary changes. Write access required for same.
subhajitdas298 is my username for cwiki.
Thanks and Regards,
Subhajit Das
From: Subhajit Das<ma...@live.com>
Sent: 17 March 2021 09:39 AM
To: user@tika.apache.org<ma...@tika.apache.org>; tallison@apache.org<ma...@apache.org>
Cc: user@tika.apache.org<ma...@tika.apache.org>
Subject: RE: Prometheus exporter for TikaServer
Raised https://issues.apache.org/jira/browse/TIKA-3327 for Simple server monitoring, for 1.x branch. This should have to no performance impact, as exposed over mbean.
From: Subhajit Das<ma...@live.com>
Sent: 17 March 2021 08:39 AM
To: user@tika.apache.org<ma...@tika.apache.org>; tallison@apache.org<ma...@apache.org>
Cc: user@tika.apache.org<ma...@tika.apache.org>
Subject: RE: Prometheus exporter for TikaServer
Hi Tim,
OpenAPI is for live API, usage + documentation. OpenAPI can replace the “/” welcome page (where all APIs are listed) and give a much more modern experience.
While metrics monitoring is a separate concern. This, exposes the metrics data, like how many requests, how much time, jvm metrics, other custom metrics, like how many files processed etc.
Here we would have to inject the metric gauges programmatically. And let micrometer expose the value.
Note: Monitoring always introduces a minor (some times negligible) performance overhead. So, an option has to be given to switch on/off monitoring. Just like Solr does. But, it can be simple as a command line flag.
Thanks and Regards,
Subhajit
From: Tim Allison<ma...@apache.org>
Sent: 16 March 2021 11:40 PM
To: Subhajit Das<ma...@live.com>
Cc: user@tika.apache.org<ma...@tika.apache.org>
Subject: Re: Prometheus exporter for TikaServer
Interesting...Does this play at all with OpenAPI?
https://issues.apache.org/jira/browse/TIKA-3082
On Tue, Mar 16, 2021 at 1:35 PM Subhajit Das <su...@live.com>> wrote:
>
> Hi Tim,
>
>
>
> Prometheus requires data in it’s format. It seems that Tika does not have extended metrics collection system (correct me, if I am wrong).
>
> The status is storing status in ServerStatus object, which dosen’t seems to be exposed as MBean.
>
> If this is exposed as MBean, basic requirement may be fulfilled with Prometheus JMX Exporter (https://github.com/prometheus/jmx_exporter) as a Javaagent.
>
>
>
> For more advanced,
>
> Solr uses Dropwizard (https://solr.apache.org/guide/8_8/metrics-reporting.html) for metrics, and a separate Prometheus exporter (https://solr.apache.org/guide/8_8/monitoring-solr-with-prometheus-and-grafana.html) to export metrics to Prometheus.
>
>
>
> But, a better solution might be to use Micrometer (https://micrometer.io/), as it supports inbuild exporting for multiple standards, and this is also standard for new Spring Boot 2.0 projects.
>
>
>
> Thanks,
>
> Subhajit
>
>
>
> From: Tim Allison
> Sent: 16 March 2021 09:32 PM
> To: user@tika.apache.org<ma...@tika.apache.org>
> Subject: Re: Prometheus exporter for TikaServer
>
>
>
> We have only a barebones /status endpoint at the moment. You have to
> configure it/turn it on. It is not available by default. If there
> are specific statistics or a specific format you need for integration
> with Prometheus, please let us know.
>
> On Mon, Mar 15, 2021 at 4:06 AM Subhajit Das <Su...@live.com>> wrote:
> >
> > Hi,
> >
> >
> >
> > Is there any Prometheus exporter functionally, on TikaServer?
> >
> > Need details on status endpoint over Prometheus as well.
> >
> >
> >
> > Then have to setup a Grafana dashboard for the same.
> >
> >
> >
> > Thanks in advance.
> >
> >
>
>
Re: Prometheus exporter for TikaServer
Posted by Tim Allison <ta...@apache.org>.
I may have just twiddled the permission bits correctly. Let me know if not.
Thank you!!!
On Sun, Apr 11, 2021 at 5:38 PM Subhajit Das <su...@live.com>
wrote:
> Hi Tim,
>
>
>
> Thanks for merging the PR for this. But, CWIKI has to be updated for the
> same.
>
>
>
> It seems that, there is even no mention for /status endpoint in
> https://cwiki.apache.org/confluence/display/TIKA/TikaServer.
>
>
>
> I can make the necessary changes. Write access required for same.
>
>
>
> subhajitdas298 is my username for cwiki.
>
>
>
> Thanks and Regards,
>
> Subhajit Das
>
>
>
> *From: *Subhajit Das <su...@live.com>
> *Sent: *17 March 2021 09:39 AM
> *To: *user@tika.apache.org; tallison@apache.org
> *Cc: *user@tika.apache.org
> *Subject: *RE: Prometheus exporter for TikaServer
>
>
>
> Raised https://issues.apache.org/jira/browse/TIKA-3327 for Simple server
> monitoring, for 1.x branch. This should have to no performance impact, as
> exposed over mbean.
>
>
>
> *From: *Subhajit Das <su...@live.com>
> *Sent: *17 March 2021 08:39 AM
> *To: *user@tika.apache.org; tallison@apache.org
> *Cc: *user@tika.apache.org
> *Subject: *RE: Prometheus exporter for TikaServer
>
>
>
> Hi Tim,
>
>
>
> OpenAPI is for live API, usage + documentation. OpenAPI can replace the
> “/” welcome page (where all APIs are listed) and give a much more modern
> experience.
>
>
>
> While metrics monitoring is a separate concern. This, exposes the metrics
> data, like how many requests, how much time, jvm metrics, other custom
> metrics, like how many files processed etc.
>
>
>
> Here we would have to inject the metric gauges programmatically. And let
> micrometer expose the value.
>
>
>
> Note: Monitoring always introduces a minor (some times negligible)
> performance overhead. So, an option has to be given to switch on/off
> monitoring. Just like Solr does. But, it can be simple as a command line
> flag.
>
>
>
> Thanks and Regards,
>
> Subhajit
>
>
>
> *From: *Tim Allison <ta...@apache.org>
> *Sent: *16 March 2021 11:40 PM
> *To: *Subhajit Das <su...@live.com>
> *Cc: *user@tika.apache.org
> *Subject: *Re: Prometheus exporter for TikaServer
>
>
>
> Interesting...Does this play at all with OpenAPI?
> https://issues.apache.org/jira/browse/TIKA-3082
>
> On Tue, Mar 16, 2021 at 1:35 PM Subhajit Das <su...@live.com>
> wrote:
> >
> > Hi Tim,
> >
> >
> >
> > Prometheus requires data in it’s format. It seems that Tika does not
> have extended metrics collection system (correct me, if I am wrong).
> >
> > The status is storing status in ServerStatus object, which dosen’t
> seems to be exposed as MBean.
> >
> > If this is exposed as MBean, basic requirement may be fulfilled with
> Prometheus JMX Exporter (https://github.com/prometheus/jmx_exporter) as a
> Javaagent.
> >
> >
> >
> > For more advanced,
> >
> > Solr uses Dropwizard (
> https://solr.apache.org/guide/8_8/metrics-reporting.html) for metrics,
> and a separate Prometheus exporter (
> https://solr.apache.org/guide/8_8/monitoring-solr-with-prometheus-and-grafana.html)
> to export metrics to Prometheus.
> >
> >
> >
> > But, a better solution might be to use Micrometer (
> https://micrometer.io/), as it supports inbuild exporting for multiple
> standards, and this is also standard for new Spring Boot 2.0 projects.
> >
> >
> >
> > Thanks,
> >
> > Subhajit
> >
> >
> >
> > From: Tim Allison
> > Sent: 16 March 2021 09:32 PM
> > To: user@tika.apache.org
> > Subject: Re: Prometheus exporter for TikaServer
> >
> >
> >
> > We have only a barebones /status endpoint at the moment. You have to
> > configure it/turn it on. It is not available by default. If there
> > are specific statistics or a specific format you need for integration
> > with Prometheus, please let us know.
> >
> > On Mon, Mar 15, 2021 at 4:06 AM Subhajit Das <Su...@live.com>
> wrote:
> > >
> > > Hi,
> > >
> > >
> > >
> > > Is there any Prometheus exporter functionally, on TikaServer?
> > >
> > > Need details on status endpoint over Prometheus as well.
> > >
> > >
> > >
> > > Then have to setup a Grafana dashboard for the same.
> > >
> > >
> > >
> > > Thanks in advance.
> > >
> > >
> >
> >
>
>
>
>
>
>
>
RE: Prometheus exporter for TikaServer
Posted by Subhajit Das <su...@live.com>.
Hi Tim,
Thanks for merging the PR for this. But, CWIKI has to be updated for the same.
It seems that, there is even no mention for /status endpoint in https://cwiki.apache.org/confluence/display/TIKA/TikaServer.
I can make the necessary changes. Write access required for same.
subhajitdas298 is my username for cwiki.
Thanks and Regards,
Subhajit Das
From: Subhajit Das<ma...@live.com>
Sent: 17 March 2021 09:39 AM
To: user@tika.apache.org<ma...@tika.apache.org>; tallison@apache.org<ma...@apache.org>
Cc: user@tika.apache.org<ma...@tika.apache.org>
Subject: RE: Prometheus exporter for TikaServer
Raised https://issues.apache.org/jira/browse/TIKA-3327 for Simple server monitoring, for 1.x branch. This should have to no performance impact, as exposed over mbean.
From: Subhajit Das<ma...@live.com>
Sent: 17 March 2021 08:39 AM
To: user@tika.apache.org<ma...@tika.apache.org>; tallison@apache.org<ma...@apache.org>
Cc: user@tika.apache.org<ma...@tika.apache.org>
Subject: RE: Prometheus exporter for TikaServer
Hi Tim,
OpenAPI is for live API, usage + documentation. OpenAPI can replace the “/” welcome page (where all APIs are listed) and give a much more modern experience.
While metrics monitoring is a separate concern. This, exposes the metrics data, like how many requests, how much time, jvm metrics, other custom metrics, like how many files processed etc.
Here we would have to inject the metric gauges programmatically. And let micrometer expose the value.
Note: Monitoring always introduces a minor (some times negligible) performance overhead. So, an option has to be given to switch on/off monitoring. Just like Solr does. But, it can be simple as a command line flag.
Thanks and Regards,
Subhajit
From: Tim Allison<ma...@apache.org>
Sent: 16 March 2021 11:40 PM
To: Subhajit Das<ma...@live.com>
Cc: user@tika.apache.org<ma...@tika.apache.org>
Subject: Re: Prometheus exporter for TikaServer
Interesting...Does this play at all with OpenAPI?
https://issues.apache.org/jira/browse/TIKA-3082
On Tue, Mar 16, 2021 at 1:35 PM Subhajit Das <su...@live.com> wrote:
>
> Hi Tim,
>
>
>
> Prometheus requires data in it’s format. It seems that Tika does not have extended metrics collection system (correct me, if I am wrong).
>
> The status is storing status in ServerStatus object, which dosen’t seems to be exposed as MBean.
>
> If this is exposed as MBean, basic requirement may be fulfilled with Prometheus JMX Exporter (https://github.com/prometheus/jmx_exporter) as a Javaagent.
>
>
>
> For more advanced,
>
> Solr uses Dropwizard (https://solr.apache.org/guide/8_8/metrics-reporting.html) for metrics, and a separate Prometheus exporter (https://solr.apache.org/guide/8_8/monitoring-solr-with-prometheus-and-grafana.html) to export metrics to Prometheus.
>
>
>
> But, a better solution might be to use Micrometer (https://micrometer.io/), as it supports inbuild exporting for multiple standards, and this is also standard for new Spring Boot 2.0 projects.
>
>
>
> Thanks,
>
> Subhajit
>
>
>
> From: Tim Allison
> Sent: 16 March 2021 09:32 PM
> To: user@tika.apache.org
> Subject: Re: Prometheus exporter for TikaServer
>
>
>
> We have only a barebones /status endpoint at the moment. You have to
> configure it/turn it on. It is not available by default. If there
> are specific statistics or a specific format you need for integration
> with Prometheus, please let us know.
>
> On Mon, Mar 15, 2021 at 4:06 AM Subhajit Das <Su...@live.com> wrote:
> >
> > Hi,
> >
> >
> >
> > Is there any Prometheus exporter functionally, on TikaServer?
> >
> > Need details on status endpoint over Prometheus as well.
> >
> >
> >
> > Then have to setup a Grafana dashboard for the same.
> >
> >
> >
> > Thanks in advance.
> >
> >
>
>
RE: Prometheus exporter for TikaServer
Posted by Subhajit Das <su...@live.com>.
Raised https://issues.apache.org/jira/browse/TIKA-3327 for Simple server monitoring, for 1.x branch. This should have to no performance impact, as exposed over mbean.
From: Subhajit Das<ma...@live.com>
Sent: 17 March 2021 08:39 AM
To: user@tika.apache.org<ma...@tika.apache.org>; tallison@apache.org<ma...@apache.org>
Cc: user@tika.apache.org<ma...@tika.apache.org>
Subject: RE: Prometheus exporter for TikaServer
Hi Tim,
OpenAPI is for live API, usage + documentation. OpenAPI can replace the “/” welcome page (where all APIs are listed) and give a much more modern experience.
While metrics monitoring is a separate concern. This, exposes the metrics data, like how many requests, how much time, jvm metrics, other custom metrics, like how many files processed etc.
Here we would have to inject the metric gauges programmatically. And let micrometer expose the value.
Note: Monitoring always introduces a minor (some times negligible) performance overhead. So, an option has to be given to switch on/off monitoring. Just like Solr does. But, it can be simple as a command line flag.
Thanks and Regards,
Subhajit
From: Tim Allison<ma...@apache.org>
Sent: 16 March 2021 11:40 PM
To: Subhajit Das<ma...@live.com>
Cc: user@tika.apache.org<ma...@tika.apache.org>
Subject: Re: Prometheus exporter for TikaServer
Interesting...Does this play at all with OpenAPI?
https://issues.apache.org/jira/browse/TIKA-3082
On Tue, Mar 16, 2021 at 1:35 PM Subhajit Das <su...@live.com> wrote:
>
> Hi Tim,
>
>
>
> Prometheus requires data in it’s format. It seems that Tika does not have extended metrics collection system (correct me, if I am wrong).
>
> The status is storing status in ServerStatus object, which dosen’t seems to be exposed as MBean.
>
> If this is exposed as MBean, basic requirement may be fulfilled with Prometheus JMX Exporter (https://github.com/prometheus/jmx_exporter) as a Javaagent.
>
>
>
> For more advanced,
>
> Solr uses Dropwizard (https://solr.apache.org/guide/8_8/metrics-reporting.html) for metrics, and a separate Prometheus exporter (https://solr.apache.org/guide/8_8/monitoring-solr-with-prometheus-and-grafana.html) to export metrics to Prometheus.
>
>
>
> But, a better solution might be to use Micrometer (https://micrometer.io/), as it supports inbuild exporting for multiple standards, and this is also standard for new Spring Boot 2.0 projects.
>
>
>
> Thanks,
>
> Subhajit
>
>
>
> From: Tim Allison
> Sent: 16 March 2021 09:32 PM
> To: user@tika.apache.org
> Subject: Re: Prometheus exporter for TikaServer
>
>
>
> We have only a barebones /status endpoint at the moment. You have to
> configure it/turn it on. It is not available by default. If there
> are specific statistics or a specific format you need for integration
> with Prometheus, please let us know.
>
> On Mon, Mar 15, 2021 at 4:06 AM Subhajit Das <Su...@live.com> wrote:
> >
> > Hi,
> >
> >
> >
> > Is there any Prometheus exporter functionally, on TikaServer?
> >
> > Need details on status endpoint over Prometheus as well.
> >
> >
> >
> > Then have to setup a Grafana dashboard for the same.
> >
> >
> >
> > Thanks in advance.
> >
> >
>
>
RE: Prometheus exporter for TikaServer
Posted by Subhajit Das <su...@live.com>.
Hi Tim,
OpenAPI is for live API, usage + documentation. OpenAPI can replace the “/” welcome page (where all APIs are listed) and give a much more modern experience.
While metrics monitoring is a separate concern. This, exposes the metrics data, like how many requests, how much time, jvm metrics, other custom metrics, like how many files processed etc.
Here we would have to inject the metric gauges programmatically. And let micrometer expose the value.
Note: Monitoring always introduces a minor (some times negligible) performance overhead. So, an option has to be given to switch on/off monitoring. Just like Solr does. But, it can be simple as a command line flag.
Thanks and Regards,
Subhajit
From: Tim Allison<ma...@apache.org>
Sent: 16 March 2021 11:40 PM
To: Subhajit Das<ma...@live.com>
Cc: user@tika.apache.org<ma...@tika.apache.org>
Subject: Re: Prometheus exporter for TikaServer
Interesting...Does this play at all with OpenAPI?
https://issues.apache.org/jira/browse/TIKA-3082
On Tue, Mar 16, 2021 at 1:35 PM Subhajit Das <su...@live.com> wrote:
>
> Hi Tim,
>
>
>
> Prometheus requires data in it’s format. It seems that Tika does not have extended metrics collection system (correct me, if I am wrong).
>
> The status is storing status in ServerStatus object, which dosen’t seems to be exposed as MBean.
>
> If this is exposed as MBean, basic requirement may be fulfilled with Prometheus JMX Exporter (https://github.com/prometheus/jmx_exporter) as a Javaagent.
>
>
>
> For more advanced,
>
> Solr uses Dropwizard (https://solr.apache.org/guide/8_8/metrics-reporting.html) for metrics, and a separate Prometheus exporter (https://solr.apache.org/guide/8_8/monitoring-solr-with-prometheus-and-grafana.html) to export metrics to Prometheus.
>
>
>
> But, a better solution might be to use Micrometer (https://micrometer.io/), as it supports inbuild exporting for multiple standards, and this is also standard for new Spring Boot 2.0 projects.
>
>
>
> Thanks,
>
> Subhajit
>
>
>
> From: Tim Allison
> Sent: 16 March 2021 09:32 PM
> To: user@tika.apache.org
> Subject: Re: Prometheus exporter for TikaServer
>
>
>
> We have only a barebones /status endpoint at the moment. You have to
> configure it/turn it on. It is not available by default. If there
> are specific statistics or a specific format you need for integration
> with Prometheus, please let us know.
>
> On Mon, Mar 15, 2021 at 4:06 AM Subhajit Das <Su...@live.com> wrote:
> >
> > Hi,
> >
> >
> >
> > Is there any Prometheus exporter functionally, on TikaServer?
> >
> > Need details on status endpoint over Prometheus as well.
> >
> >
> >
> > Then have to setup a Grafana dashboard for the same.
> >
> >
> >
> > Thanks in advance.
> >
> >
>
>
Re: Prometheus exporter for TikaServer
Posted by Tim Allison <ta...@apache.org>.
Interesting...Does this play at all with OpenAPI?
https://issues.apache.org/jira/browse/TIKA-3082
On Tue, Mar 16, 2021 at 1:35 PM Subhajit Das <su...@live.com> wrote:
>
> Hi Tim,
>
>
>
> Prometheus requires data in it’s format. It seems that Tika does not have extended metrics collection system (correct me, if I am wrong).
>
> The status is storing status in ServerStatus object, which dosen’t seems to be exposed as MBean.
>
> If this is exposed as MBean, basic requirement may be fulfilled with Prometheus JMX Exporter (https://github.com/prometheus/jmx_exporter) as a Javaagent.
>
>
>
> For more advanced,
>
> Solr uses Dropwizard (https://solr.apache.org/guide/8_8/metrics-reporting.html) for metrics, and a separate Prometheus exporter (https://solr.apache.org/guide/8_8/monitoring-solr-with-prometheus-and-grafana.html) to export metrics to Prometheus.
>
>
>
> But, a better solution might be to use Micrometer (https://micrometer.io/), as it supports inbuild exporting for multiple standards, and this is also standard for new Spring Boot 2.0 projects.
>
>
>
> Thanks,
>
> Subhajit
>
>
>
> From: Tim Allison
> Sent: 16 March 2021 09:32 PM
> To: user@tika.apache.org
> Subject: Re: Prometheus exporter for TikaServer
>
>
>
> We have only a barebones /status endpoint at the moment. You have to
> configure it/turn it on. It is not available by default. If there
> are specific statistics or a specific format you need for integration
> with Prometheus, please let us know.
>
> On Mon, Mar 15, 2021 at 4:06 AM Subhajit Das <Su...@live.com> wrote:
> >
> > Hi,
> >
> >
> >
> > Is there any Prometheus exporter functionally, on TikaServer?
> >
> > Need details on status endpoint over Prometheus as well.
> >
> >
> >
> > Then have to setup a Grafana dashboard for the same.
> >
> >
> >
> > Thanks in advance.
> >
> >
>
>
RE: Prometheus exporter for TikaServer
Posted by Subhajit Das <su...@live.com>.
Hi Tim,
Prometheus requires data in it’s format. It seems that Tika does not have extended metrics collection system (correct me, if I am wrong).
The status is storing status in ServerStatus object, which dosen’t seems to be exposed as MBean.
If this is exposed as MBean, basic requirement may be fulfilled with Prometheus JMX Exporter (https://github.com/prometheus/jmx_exporter) as a Javaagent.
For more advanced,
Solr uses Dropwizard (https://solr.apache.org/guide/8_8/metrics-reporting.html) for metrics, and a separate Prometheus exporter (https://solr.apache.org/guide/8_8/monitoring-solr-with-prometheus-and-grafana.html) to export metrics to Prometheus.
But, a better solution might be to use Micrometer (https://micrometer.io/), as it supports inbuild exporting for multiple standards, and this is also standard for new Spring Boot 2.0 projects.
Thanks,
Subhajit
From: Tim Allison<ma...@apache.org>
Sent: 16 March 2021 09:32 PM
To: user@tika.apache.org<ma...@tika.apache.org>
Subject: Re: Prometheus exporter for TikaServer
We have only a barebones /status endpoint at the moment. You have to
configure it/turn it on. It is not available by default. If there
are specific statistics or a specific format you need for integration
with Prometheus, please let us know.
On Mon, Mar 15, 2021 at 4:06 AM Subhajit Das <Su...@live.com> wrote:
>
> Hi,
>
>
>
> Is there any Prometheus exporter functionally, on TikaServer?
>
> Need details on status endpoint over Prometheus as well.
>
>
>
> Then have to setup a Grafana dashboard for the same.
>
>
>
> Thanks in advance.
>
>
Re: Prometheus exporter for TikaServer
Posted by Tim Allison <ta...@apache.org>.
We have only a barebones /status endpoint at the moment. You have to
configure it/turn it on. It is not available by default. If there
are specific statistics or a specific format you need for integration
with Prometheus, please let us know.
On Mon, Mar 15, 2021 at 4:06 AM Subhajit Das <Su...@live.com> wrote:
>
> Hi,
>
>
>
> Is there any Prometheus exporter functionally, on TikaServer?
>
> Need details on status endpoint over Prometheus as well.
>
>
>
> Then have to setup a Grafana dashboard for the same.
>
>
>
> Thanks in advance.
>
>