You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Pierre-Luc Dion <pd...@cloudops.com> on 2014/08/16 16:36:24 UTC
cloudstack event to logstash
Hi,
Does anyone tried to push CloudStack Event logs into logstash or graylog2
via rabbitmq ?
I've found nothing about this so far.
Thanks
*Pierre-Luc DION*
Architecte de Solution Cloud | Cloud Solutions Architect
t 855.652.5683
*CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
Re: cloudstack event to logstash
Posted by Pierre-Luc Dion <pd...@cloudops.com>.
Thanks Ian!
I'll give it a shot. This is quite straight forward!
*Pierre-Luc DION*
Architecte de Solution Cloud | Cloud Solutions Architect
t 855.652.5683
*CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Sat, Aug 16, 2014 at 7:28 PM, Ian Duffy <ia...@ianduffy.ie> wrote:
> Hi Pierre-Luc,
>
> I decided to have a look at this evening.
>
> I was successfully able to go cloudstack -> rabbitmq -> logstash ->
> elasticsearch.
>
> I created a new spring-context within:
>
> cloudstack/server/resources/META-INF/cloudstack/core
>
> called: spring-event-bus-context.xml
>
> contents: https://gist.github.com/imduffy15/234c6b5fdde57a8910b0
>
>
> I configured logstash to create a queue and bind it to my cloudstack-event
> exchange:
>
>
>
> input {
>
> rabbitmq {
>
> host => "localhost"
>
> queue => "cloudstack-queue"
>
> exchange => "cloudstack-events"
>
> key => "*.*.*.*.*"
>
> exclusive => true
>
> }
>
> }
>
>
> output {
>
> elasticsearch { host => localhost protocol => 'http' }
>
> }
>
>
> On 16 August 2014 18:19, Ian Duffy <ia...@ianduffy.ie> wrote:
>
> > Hi Pierre-Luc
> >
> > Have you seen the post by Chip? He details how to publish events to
> > RabbitMQ:
> >
> http://www.chipchilders.com/blog/2013/7/16/tapping-into-apache-cloudstack-events-via-amqp.html
> >
> > The logstash docs then discuss getting the data from rabbitmq:
> > http://logstash.net/docs/1.3.2/inputs/rabbitmq
> >
> > Never tried it but looks straight forward.
> >
> >
> > On 16 August 2014 15:36, Pierre-Luc Dion <pd...@cloudops.com> wrote:
> >
> >> Hi,
> >>
> >> Does anyone tried to push CloudStack Event logs into logstash or
> graylog2
> >> via rabbitmq ?
> >> I've found nothing about this so far.
> >>
> >> Thanks
> >>
> >> *Pierre-Luc DION*
> >> Architecte de Solution Cloud | Cloud Solutions Architect
> >> t 855.652.5683
> >>
> >> *CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
> >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> >> w cloudops.com *|* tw @CloudOps_
> >>
> >
> >
>
Re: cloudstack event to logstash
Posted by Erik Weber <te...@gmail.com>.
Found it here: https://issues.apache.org/jira/browse/CLOUDSTACK-5879
That's /etc/cloudstack/management/META-INF/cloudstack/core for me, seems to
work, still working on it :-)
Thanks though!
Erik
On Sun, Aug 17, 2014 at 9:18 PM, Ian Duffy <ia...@ianduffy.ie> wrote:
> Hi Erik,
>
> Not sure. I tried placing it into
> /usr/share/cloudstack-management/webapps/client/WEB-INF/classes which would
> be in the classpath but doesn't work.
>
> I'd imagine there's more knowledgable folk on dev@ that would be able to
> guide you.
>
>
> On 17 August 2014 19:19, Erik Weber <te...@gmail.com> wrote:
>
> > Hi Ian,
> >
> > Any idea where that context would be defined on a rpm installation?
> >
> > --
> > Erik
> >
> >
> > On Sun, Aug 17, 2014 at 1:28 AM, Ian Duffy <ia...@ianduffy.ie> wrote:
> >
> > > Hi Pierre-Luc,
> > >
> > > I decided to have a look at this evening.
> > >
> > > I was successfully able to go cloudstack -> rabbitmq -> logstash ->
> > > elasticsearch.
> > >
> > > I created a new spring-context within:
> > >
> > > cloudstack/server/resources/META-INF/cloudstack/core
> > >
> > > called: spring-event-bus-context.xml
> > >
> > > contents: https://gist.github.com/imduffy15/234c6b5fdde57a8910b0
> > >
> > >
> > > I configured logstash to create a queue and bind it to my
> > cloudstack-event
> > > exchange:
> > >
> > >
> > >
> > > input {
> > >
> > > rabbitmq {
> > >
> > > host => "localhost"
> > >
> > > queue => "cloudstack-queue"
> > >
> > > exchange => "cloudstack-events"
> > >
> > > key => "*.*.*.*.*"
> > >
> > > exclusive => true
> > >
> > > }
> > >
> > > }
> > >
> > >
> > > output {
> > >
> > > elasticsearch { host => localhost protocol => 'http' }
> > >
> > > }
> > >
> > >
> > > On 16 August 2014 18:19, Ian Duffy <ia...@ianduffy.ie> wrote:
> > >
> > > > Hi Pierre-Luc
> > > >
> > > > Have you seen the post by Chip? He details how to publish events to
> > > > RabbitMQ:
> > > >
> > >
> >
> http://www.chipchilders.com/blog/2013/7/16/tapping-into-apache-cloudstack-events-via-amqp.html
> > > >
> > > > The logstash docs then discuss getting the data from rabbitmq:
> > > > http://logstash.net/docs/1.3.2/inputs/rabbitmq
> > > >
> > > > Never tried it but looks straight forward.
> > > >
> > > >
> > > > On 16 August 2014 15:36, Pierre-Luc Dion <pd...@cloudops.com> wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> Does anyone tried to push CloudStack Event logs into logstash or
> > > graylog2
> > > >> via rabbitmq ?
> > > >> I've found nothing about this so far.
> > > >>
> > > >> Thanks
> > > >>
> > > >> *Pierre-Luc DION*
> > > >> Architecte de Solution Cloud | Cloud Solutions Architect
> > > >> t 855.652.5683
> > > >>
> > > >> *CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
> > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > > >> w cloudops.com *|* tw @CloudOps_
> > > >>
> > > >
> > > >
> > >
> >
>
Re: cloudstack event to logstash
Posted by Ian Duffy <ia...@ianduffy.ie>.
Hi Erik,
Not sure. I tried placing it into
/usr/share/cloudstack-management/webapps/client/WEB-INF/classes which would
be in the classpath but doesn't work.
I'd imagine there's more knowledgable folk on dev@ that would be able to
guide you.
On 17 August 2014 19:19, Erik Weber <te...@gmail.com> wrote:
> Hi Ian,
>
> Any idea where that context would be defined on a rpm installation?
>
> --
> Erik
>
>
> On Sun, Aug 17, 2014 at 1:28 AM, Ian Duffy <ia...@ianduffy.ie> wrote:
>
> > Hi Pierre-Luc,
> >
> > I decided to have a look at this evening.
> >
> > I was successfully able to go cloudstack -> rabbitmq -> logstash ->
> > elasticsearch.
> >
> > I created a new spring-context within:
> >
> > cloudstack/server/resources/META-INF/cloudstack/core
> >
> > called: spring-event-bus-context.xml
> >
> > contents: https://gist.github.com/imduffy15/234c6b5fdde57a8910b0
> >
> >
> > I configured logstash to create a queue and bind it to my
> cloudstack-event
> > exchange:
> >
> >
> >
> > input {
> >
> > rabbitmq {
> >
> > host => "localhost"
> >
> > queue => "cloudstack-queue"
> >
> > exchange => "cloudstack-events"
> >
> > key => "*.*.*.*.*"
> >
> > exclusive => true
> >
> > }
> >
> > }
> >
> >
> > output {
> >
> > elasticsearch { host => localhost protocol => 'http' }
> >
> > }
> >
> >
> > On 16 August 2014 18:19, Ian Duffy <ia...@ianduffy.ie> wrote:
> >
> > > Hi Pierre-Luc
> > >
> > > Have you seen the post by Chip? He details how to publish events to
> > > RabbitMQ:
> > >
> >
> http://www.chipchilders.com/blog/2013/7/16/tapping-into-apache-cloudstack-events-via-amqp.html
> > >
> > > The logstash docs then discuss getting the data from rabbitmq:
> > > http://logstash.net/docs/1.3.2/inputs/rabbitmq
> > >
> > > Never tried it but looks straight forward.
> > >
> > >
> > > On 16 August 2014 15:36, Pierre-Luc Dion <pd...@cloudops.com> wrote:
> > >
> > >> Hi,
> > >>
> > >> Does anyone tried to push CloudStack Event logs into logstash or
> > graylog2
> > >> via rabbitmq ?
> > >> I've found nothing about this so far.
> > >>
> > >> Thanks
> > >>
> > >> *Pierre-Luc DION*
> > >> Architecte de Solution Cloud | Cloud Solutions Architect
> > >> t 855.652.5683
> > >>
> > >> *CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
> > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > >> w cloudops.com *|* tw @CloudOps_
> > >>
> > >
> > >
> >
>
Re: cloudstack event to logstash
Posted by Erik Weber <te...@gmail.com>.
Hi Ian,
Any idea where that context would be defined on a rpm installation?
--
Erik
On Sun, Aug 17, 2014 at 1:28 AM, Ian Duffy <ia...@ianduffy.ie> wrote:
> Hi Pierre-Luc,
>
> I decided to have a look at this evening.
>
> I was successfully able to go cloudstack -> rabbitmq -> logstash ->
> elasticsearch.
>
> I created a new spring-context within:
>
> cloudstack/server/resources/META-INF/cloudstack/core
>
> called: spring-event-bus-context.xml
>
> contents: https://gist.github.com/imduffy15/234c6b5fdde57a8910b0
>
>
> I configured logstash to create a queue and bind it to my cloudstack-event
> exchange:
>
>
>
> input {
>
> rabbitmq {
>
> host => "localhost"
>
> queue => "cloudstack-queue"
>
> exchange => "cloudstack-events"
>
> key => "*.*.*.*.*"
>
> exclusive => true
>
> }
>
> }
>
>
> output {
>
> elasticsearch { host => localhost protocol => 'http' }
>
> }
>
>
> On 16 August 2014 18:19, Ian Duffy <ia...@ianduffy.ie> wrote:
>
> > Hi Pierre-Luc
> >
> > Have you seen the post by Chip? He details how to publish events to
> > RabbitMQ:
> >
> http://www.chipchilders.com/blog/2013/7/16/tapping-into-apache-cloudstack-events-via-amqp.html
> >
> > The logstash docs then discuss getting the data from rabbitmq:
> > http://logstash.net/docs/1.3.2/inputs/rabbitmq
> >
> > Never tried it but looks straight forward.
> >
> >
> > On 16 August 2014 15:36, Pierre-Luc Dion <pd...@cloudops.com> wrote:
> >
> >> Hi,
> >>
> >> Does anyone tried to push CloudStack Event logs into logstash or
> graylog2
> >> via rabbitmq ?
> >> I've found nothing about this so far.
> >>
> >> Thanks
> >>
> >> *Pierre-Luc DION*
> >> Architecte de Solution Cloud | Cloud Solutions Architect
> >> t 855.652.5683
> >>
> >> *CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
> >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> >> w cloudops.com *|* tw @CloudOps_
> >>
> >
> >
>
Re: cloudstack event to logstash
Posted by Ian Duffy <ia...@ianduffy.ie>.
Hi Pierre-Luc,
I decided to have a look at this evening.
I was successfully able to go cloudstack -> rabbitmq -> logstash ->
elasticsearch.
I created a new spring-context within:
cloudstack/server/resources/META-INF/cloudstack/core
called: spring-event-bus-context.xml
contents: https://gist.github.com/imduffy15/234c6b5fdde57a8910b0
I configured logstash to create a queue and bind it to my cloudstack-event
exchange:
input {
rabbitmq {
host => "localhost"
queue => "cloudstack-queue"
exchange => "cloudstack-events"
key => "*.*.*.*.*"
exclusive => true
}
}
output {
elasticsearch { host => localhost protocol => 'http' }
}
On 16 August 2014 18:19, Ian Duffy <ia...@ianduffy.ie> wrote:
> Hi Pierre-Luc
>
> Have you seen the post by Chip? He details how to publish events to
> RabbitMQ:
> http://www.chipchilders.com/blog/2013/7/16/tapping-into-apache-cloudstack-events-via-amqp.html
>
> The logstash docs then discuss getting the data from rabbitmq:
> http://logstash.net/docs/1.3.2/inputs/rabbitmq
>
> Never tried it but looks straight forward.
>
>
> On 16 August 2014 15:36, Pierre-Luc Dion <pd...@cloudops.com> wrote:
>
>> Hi,
>>
>> Does anyone tried to push CloudStack Event logs into logstash or graylog2
>> via rabbitmq ?
>> I've found nothing about this so far.
>>
>> Thanks
>>
>> *Pierre-Luc DION*
>> Architecte de Solution Cloud | Cloud Solutions Architect
>> t 855.652.5683
>>
>> *CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> w cloudops.com *|* tw @CloudOps_
>>
>
>
Re: cloudstack event to logstash
Posted by Ian Duffy <ia...@ianduffy.ie>.
Hi Pierre-Luc
Have you seen the post by Chip? He details how to publish events to
RabbitMQ:
http://www.chipchilders.com/blog/2013/7/16/tapping-into-apache-cloudstack-events-via-amqp.html
The logstash docs then discuss getting the data from rabbitmq:
http://logstash.net/docs/1.3.2/inputs/rabbitmq
Never tried it but looks straight forward.
On 16 August 2014 15:36, Pierre-Luc Dion <pd...@cloudops.com> wrote:
> Hi,
>
> Does anyone tried to push CloudStack Event logs into logstash or graylog2
> via rabbitmq ?
> I've found nothing about this so far.
>
> Thanks
>
> *Pierre-Luc DION*
> Architecte de Solution Cloud | Cloud Solutions Architect
> t 855.652.5683
>
> *CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>