You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Dani Abel Rayan (JIRA)" <ji...@apache.org> on 2011/08/05 19:31:27 UTC
[jira] [Created] (FLUME-726) OpenTSDB Sink
OpenTSDB Sink
-------------
Key: FLUME-726
URL: https://issues.apache.org/jira/browse/FLUME-726
Project: Flume
Issue Type: New Feature
Components: Sinks+Sources
Reporter: Dani Abel Rayan
OpenTSDB is free software and is available under the LGPLv3+ license. It allows you to collect many thousands of metrics from thousands of hosts and applications, at a high rate (every few seconds). and never deletes or downsample data and can easily store billions of data points. Under the hood it uses HBase (Asynchronous client) . Out of box, it gives you awesome graphs and hands-down options to tune your query. http://opentsdb.net/
OpenTSDB was collecting data using python scripts and Open-TSDB Flume plugin looked as natural alternative.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (FLUME-726) OpenTSDB Sink
Posted by Jonathan Hsieh <jo...@cloudera.com>.
Thai,
I'd love to see some of the plugins people are working one. If you make
them public feel free to add links to these here:
https://cwiki.apache.org/FLUME/flume-plugins.html
Thanks,
Jon.
On Thu, Sep 1, 2011 at 7:45 AM, Bao Thai Ngo <ba...@gmail.com> wrote:
> Hi,
>
> I have been working on this and can share if you are interested.
>
> ~Thai
>
> On Sat, Aug 6, 2011 at 5:51 AM, Huang, Zijian(Victor) <
> zijian.huang@etrade.com> wrote:
>
> > Hi,
> > Is there anybody working on a MySqlSink? We are thinking of implementing
> > one. If no one is doing that right now, we might want to be able to
> > contribute to the project.
> >
> > Thanks
> >
> > Victor Huang
> >
> >
> > -----Original Message-----
> > From: Dani Abel Rayan (JIRA) [mailto:jira@apache.org]
> > Sent: Friday, August 05, 2011 3:30 PM
> > To: flume-dev@incubator.apache.org
> > Subject: [jira] [Commented] (FLUME-726) OpenTSDB Sink
> >
> >
> > [
> >
> https://issues.apache.org/jira/browse/FLUME-726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080269#comment-13080269
> ]
> >
> > Dani Abel Rayan commented on FLUME-726:
> > ---------------------------------------
> >
> > You can also see the maven changes in above link.
> >
> > > OpenTSDB Sink
> > > -------------
> > >
> > > Key: FLUME-726
> > > URL: https://issues.apache.org/jira/browse/FLUME-726
> > > Project: Flume
> > > Issue Type: New Feature
> > > Components: Sinks+Sources
> > > Reporter: Dani Abel Rayan
> > > Assignee: Dani Abel Rayan
> > >
> > > OpenTSDB is free software and is available under the LGPLv3+ license.
> It
> > allows you to collect many thousands of metrics from thousands of hosts
> and
> > applications, at a high rate (every few seconds). and never deletes or
> > downsample data and can easily store billions of data points. Under the
> hood
> > it uses HBase (Asynchronous client) . Out of box, it gives you awesome
> > graphs and hands-down options to tune your query. http://opentsdb.net/
> > > OpenTSDB was collecting data using python scripts and Open-TSDB Flume
> > plugin looked as natural alternative.
> >
> > --
> > This message is automatically generated by JIRA.
> > For more information on JIRA, see:
> http://www.atlassian.com/software/jira
> >
> >
> >
>
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com
Re: [jira] [Commented] (FLUME-726) OpenTSDB Sink
Posted by Bao Thai Ngo <ba...@gmail.com>.
Hi,
I have been working on this and can share if you are interested.
~Thai
On Sat, Aug 6, 2011 at 5:51 AM, Huang, Zijian(Victor) <
zijian.huang@etrade.com> wrote:
> Hi,
> Is there anybody working on a MySqlSink? We are thinking of implementing
> one. If no one is doing that right now, we might want to be able to
> contribute to the project.
>
> Thanks
>
> Victor Huang
>
>
> -----Original Message-----
> From: Dani Abel Rayan (JIRA) [mailto:jira@apache.org]
> Sent: Friday, August 05, 2011 3:30 PM
> To: flume-dev@incubator.apache.org
> Subject: [jira] [Commented] (FLUME-726) OpenTSDB Sink
>
>
> [
> https://issues.apache.org/jira/browse/FLUME-726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080269#comment-13080269]
>
> Dani Abel Rayan commented on FLUME-726:
> ---------------------------------------
>
> You can also see the maven changes in above link.
>
> > OpenTSDB Sink
> > -------------
> >
> > Key: FLUME-726
> > URL: https://issues.apache.org/jira/browse/FLUME-726
> > Project: Flume
> > Issue Type: New Feature
> > Components: Sinks+Sources
> > Reporter: Dani Abel Rayan
> > Assignee: Dani Abel Rayan
> >
> > OpenTSDB is free software and is available under the LGPLv3+ license. It
> allows you to collect many thousands of metrics from thousands of hosts and
> applications, at a high rate (every few seconds). and never deletes or
> downsample data and can easily store billions of data points. Under the hood
> it uses HBase (Asynchronous client) . Out of box, it gives you awesome
> graphs and hands-down options to tune your query. http://opentsdb.net/
> > OpenTSDB was collecting data using python scripts and Open-TSDB Flume
> plugin looked as natural alternative.
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
RE: [jira] [Commented] (FLUME-726) OpenTSDB Sink
Posted by "Huang, Zijian(Victor)" <zi...@etrade.com>.
Hi,
Is there anybody working on a MySqlSink? We are thinking of implementing one. If no one is doing that right now, we might want to be able to contribute to the project.
Thanks
Victor Huang
-----Original Message-----
From: Dani Abel Rayan (JIRA) [mailto:jira@apache.org]
Sent: Friday, August 05, 2011 3:30 PM
To: flume-dev@incubator.apache.org
Subject: [jira] [Commented] (FLUME-726) OpenTSDB Sink
[ https://issues.apache.org/jira/browse/FLUME-726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080269#comment-13080269 ]
Dani Abel Rayan commented on FLUME-726:
---------------------------------------
You can also see the maven changes in above link.
> OpenTSDB Sink
> -------------
>
> Key: FLUME-726
> URL: https://issues.apache.org/jira/browse/FLUME-726
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Reporter: Dani Abel Rayan
> Assignee: Dani Abel Rayan
>
> OpenTSDB is free software and is available under the LGPLv3+ license. It allows you to collect many thousands of metrics from thousands of hosts and applications, at a high rate (every few seconds). and never deletes or downsample data and can easily store billions of data points. Under the hood it uses HBase (Asynchronous client) . Out of box, it gives you awesome graphs and hands-down options to tune your query. http://opentsdb.net/
> OpenTSDB was collecting data using python scripts and Open-TSDB Flume plugin looked as natural alternative.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-726) OpenTSDB Sink
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080293#comment-13080293 ]
Tom White commented on FLUME-726:
---------------------------------
LGPL-licensed dependencies are not allowed in Apache projects (see http://www.apache.org/legal/resolved.html#category-x) , so this can't be added to Flume, I'm afraid.
> OpenTSDB Sink
> -------------
>
> Key: FLUME-726
> URL: https://issues.apache.org/jira/browse/FLUME-726
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Reporter: Dani Abel Rayan
> Assignee: Dani Abel Rayan
>
> OpenTSDB is free software and is available under the LGPLv3+ license. It allows you to collect many thousands of metrics from thousands of hosts and applications, at a high rate (every few seconds). and never deletes or downsample data and can easily store billions of data points. Under the hood it uses HBase (Asynchronous client) . Out of box, it gives you awesome graphs and hands-down options to tune your query. http://opentsdb.net/
> OpenTSDB was collecting data using python scripts and Open-TSDB Flume plugin looked as natural alternative.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-726) OpenTSDB Sink
Posted by "Dani Abel Rayan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094230#comment-13094230 ]
Dani Abel Rayan commented on FLUME-726:
---------------------------------------
A related discussion: Async HBase Client https://issues.apache.org/jira/browse/HBASE-2170?focusedCommentId=13007027&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13007027
> OpenTSDB Sink
> -------------
>
> Key: FLUME-726
> URL: https://issues.apache.org/jira/browse/FLUME-726
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Reporter: Dani Abel Rayan
> Assignee: Dani Abel Rayan
>
> OpenTSDB is free software and is available under the LGPLv3+ license. It allows you to collect many thousands of metrics from thousands of hosts and applications, at a high rate (every few seconds). and never deletes or downsample data and can easily store billions of data points. Under the hood it uses HBase (Asynchronous client) . Out of box, it gives you awesome graphs and hands-down options to tune your query. http://opentsdb.net/
> OpenTSDB was collecting data using python scripts and Open-TSDB Flume plugin looked as natural alternative.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (FLUME-726) OpenTSDB Sink
Posted by "Dani Abel Rayan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dani Abel Rayan reassigned FLUME-726:
-------------------------------------
Assignee: Dani Abel Rayan
> OpenTSDB Sink
> -------------
>
> Key: FLUME-726
> URL: https://issues.apache.org/jira/browse/FLUME-726
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Reporter: Dani Abel Rayan
> Assignee: Dani Abel Rayan
>
> OpenTSDB is free software and is available under the LGPLv3+ license. It allows you to collect many thousands of metrics from thousands of hosts and applications, at a high rate (every few seconds). and never deletes or downsample data and can easily store billions of data points. Under the hood it uses HBase (Asynchronous client) . Out of box, it gives you awesome graphs and hands-down options to tune your query. http://opentsdb.net/
> OpenTSDB was collecting data using python scripts and Open-TSDB Flume plugin looked as natural alternative.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-726) OpenTSDB Sink
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080261#comment-13080261 ]
Jonathan Hsieh commented on FLUME-726:
--------------------------------------
@Dani,
This looks really cool!
A few questions:
Is the goal here to fold this into a contrib/plugin dir?
If so would committing this require the inclusion the LPGLv3 code/libs or can we use ivy/maven?
> OpenTSDB Sink
> -------------
>
> Key: FLUME-726
> URL: https://issues.apache.org/jira/browse/FLUME-726
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Reporter: Dani Abel Rayan
> Assignee: Dani Abel Rayan
>
> OpenTSDB is free software and is available under the LGPLv3+ license. It allows you to collect many thousands of metrics from thousands of hosts and applications, at a high rate (every few seconds). and never deletes or downsample data and can easily store billions of data points. Under the hood it uses HBase (Asynchronous client) . Out of box, it gives you awesome graphs and hands-down options to tune your query. http://opentsdb.net/
> OpenTSDB was collecting data using python scripts and Open-TSDB Flume plugin looked as natural alternative.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-726) OpenTSDB Sink
Posted by "Dani Abel Rayan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080268#comment-13080268 ]
Dani Abel Rayan commented on FLUME-726:
---------------------------------------
@Jon, Yes! Under the hood, it is just a HBase (Async) client.
I have put it under hbase plugn dir as separate plugin:
https://github.com/LeagueOfLegends/flume/commit/977c236effed7e092b52c9ec3fd815407f1b15dc
> OpenTSDB Sink
> -------------
>
> Key: FLUME-726
> URL: https://issues.apache.org/jira/browse/FLUME-726
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Reporter: Dani Abel Rayan
> Assignee: Dani Abel Rayan
>
> OpenTSDB is free software and is available under the LGPLv3+ license. It allows you to collect many thousands of metrics from thousands of hosts and applications, at a high rate (every few seconds). and never deletes or downsample data and can easily store billions of data points. Under the hood it uses HBase (Asynchronous client) . Out of box, it gives you awesome graphs and hands-down options to tune your query. http://opentsdb.net/
> OpenTSDB was collecting data using python scripts and Open-TSDB Flume plugin looked as natural alternative.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-726) OpenTSDB Sink
Posted by "Dani Abel Rayan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080269#comment-13080269 ]
Dani Abel Rayan commented on FLUME-726:
---------------------------------------
You can also see the maven changes in above link.
> OpenTSDB Sink
> -------------
>
> Key: FLUME-726
> URL: https://issues.apache.org/jira/browse/FLUME-726
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Reporter: Dani Abel Rayan
> Assignee: Dani Abel Rayan
>
> OpenTSDB is free software and is available under the LGPLv3+ license. It allows you to collect many thousands of metrics from thousands of hosts and applications, at a high rate (every few seconds). and never deletes or downsample data and can easily store billions of data points. Under the hood it uses HBase (Asynchronous client) . Out of box, it gives you awesome graphs and hands-down options to tune your query. http://opentsdb.net/
> OpenTSDB was collecting data using python scripts and Open-TSDB Flume plugin looked as natural alternative.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (FLUME-726) OpenTSDB Sink
Posted by "Dani Abel Rayan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094230#comment-13094230 ]
Dani Abel Rayan edited comment on FLUME-726 at 8/31/11 1:18 AM:
----------------------------------------------------------------
A related discussion: Async HBase Client https://issues.apache.org/jira/browse/HBASE-2170?focusedCommentId=13007026&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13007026
was (Author: danirayan):
A related discussion: Async HBase Client https://issues.apache.org/jira/browse/HBASE-2170?focusedCommentId=13007027&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13007027
> OpenTSDB Sink
> -------------
>
> Key: FLUME-726
> URL: https://issues.apache.org/jira/browse/FLUME-726
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Reporter: Dani Abel Rayan
> Assignee: Dani Abel Rayan
>
> OpenTSDB is free software and is available under the LGPLv3+ license. It allows you to collect many thousands of metrics from thousands of hosts and applications, at a high rate (every few seconds). and never deletes or downsample data and can easily store billions of data points. Under the hood it uses HBase (Asynchronous client) . Out of box, it gives you awesome graphs and hands-down options to tune your query. http://opentsdb.net/
> OpenTSDB was collecting data using python scripts and Open-TSDB Flume plugin looked as natural alternative.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira