You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Kevin Kilroy <ke...@gmail.com> on 2018/04/27 06:18:35 UTC

Logging to elasticsearch

Hi,

I'm trying to use an elasticsearch appender to log directly to elastic
search via the es /_bulk api.

It works fine when I run the Storm topology when I use the java command,
but nothing happens when I use the storm jar command (even in local
cluster).

Is it possible this way? or is there another way that you can recommend?

I'm currently using logback but will happily switch over to log4j2 if that
makes it easier.

I'm using storm 1.2.1 & have tried adding the appenders/loggers to both the
local cluster.xml & worker.xml but nothing.

Thanks in advance,

Kevin.

Re: Logging to elasticsearch

Posted by Yu Watanabe <yu...@gmail.com>.
Hello.

Try the EsBolt in elasticsearch hadoop connector.

https://www.elastic.co/guide/en/elasticsearch/hadoop/current/storm.html

storm-elasticsearch directly calls  the transport client  and sends PUT
request to the es node.

https://github.com/apache/storm/blob/master/external/storm-elasticsearch/src/main/java/org/apache/storm/elasticsearch/bolt/EsIndexBolt.java

client.performRequest("put", getEndpoint(index, type, id), params, new
StringEntity(source));

Hope this helps.

Thanks,
Yu

On Fri, Apr 27, 2018 at 3:18 PM, Kevin Kilroy <ke...@gmail.com> wrote:

> Hi,
>
> I'm trying to use an elasticsearch appender to log directly to elastic
> search via the es /_bulk api.
>
> It works fine when I run the Storm topology when I use the java command,
> but nothing happens when I use the storm jar command (even in local
> cluster).
>
> Is it possible this way? or is there another way that you can recommend?
>
> I'm currently using logback but will happily switch over to log4j2 if that
> makes it easier.
>
> I'm using storm 1.2.1 & have tried adding the appenders/loggers to both
> the local cluster.xml & worker.xml but nothing.
>
> Thanks in advance,
>
> Kevin.
>



-- 
Yu Watanabe
渡辺 裕

LinkedIn : jp.linkedin.com/in/yuwatanabe1