You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Wido den Hollander <wi...@widodh.nl> on 2014/12/01 14:08:02 UTC
[DISCUSS] Automatic usage reporting / call-home for CloudStack
Hello,
As a project we currently don't have a lot of insight information on
about how CloudStack is being used. Surveys tell us a lot, but not
everybody fills in the survey, so we still miss a lot of information.
That's why I've written the Usage Reporting functionality for the
management server which automatically sends back anonymous information
about a CloudStack deployment.
It's currently in the 'reporter' branch. [0]
By default, every 7 days it generates a JSON document with:
- Hosts (Number, version, type, hypervisor)
- Clusters (Hypervisor en Management type)
- Primary storage (Type and provider)
- Zones (Network type and providers)
- Instances (Number and types)
This report is not complete yet, I'd like to add more information, but
that will be Management Server information.
The code on how this report is generated is obviously 100% Open Source,
so end-users can always exactly see how the information was compiled.
I want to discuss this new feature for CloudStack and the possible
implications it might have.
I'm opting for a opt-out. So every new or upgraded install to 4.6.0
(master) will have this enabled. Yes, we have to be very explicit in the
Release Notes that this has been added.
Why? It's the small price we as a project ask for using CloudStack. We
want a little bit of information on how CloudStack is being used so that
we can use this to make CloudStack even better.
Turning it off is also just one global setting and it will never turn on
again.
On the server-side there is a Python flask application [1] (found in the
reporter directory) which stores all the incoming information in a
ElasticSearch database. From there analytics can be gathered on
CloudStack deployments.
It currently points to http://cs-report.widodh.nl/report which will NOT
be the endpoint when this is merged into master.
For 'production' I want to have
https://report.cloudstack.apache.org/report where all reports are submitted.
For every setup a unique ID is determined by hashing the first row in
the 'version' table. This is the version + timestamp and that is hashed
using SHA256. Using this unique ID we can track changes in deployments
and see how they grow or shrink.
Technically this wasn't that hard to implement, but the politics
surrounding it might be the hardest part.
What do other have to say about this? Should there be a VOTE for this
feature to come into CloudStack? Opt-in, opt-out?
Wido
[0]:
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
[1]:
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter
RE: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Giles Sirett <gi...@shapeblue.com>.
Hi Wido
I recently had a thought on this.
Although I would be loathed that anybody has to "register" ACS when they use it, it would be good if we could differentiate between production deployments and lab/bench deployments.
From the marketing perspective, we've got this crazy situation where we KNOW that there are many 100's of people running ACS but still only 70 people told us about their use in the last survey.
The problem is, if we don’t differentiate between prod/lab - then the data you collect will be padded with people running it in their home labs.
I'm not sure how we can go about doing this. The simplest thing would be a global setting, but where would the incentive be for people to change that setting when they put something into production ?
Another approach, which some may think controversial, would be that we do an "about" screen/page that allows people to change this setting.
This screen could display a summary of version, #pods, clusters,etc (all the standard stuff people need to find out about their system) - so it would be useful for people
It could also contain info on how to join/access the community/ML's for support and (most controversially) it could give people the option to put in some basic company information if they wished.
I totally understand that the last bit may be a thought too far some some people
Kind Regards
Giles
D: +44 20 3603 0541 | M: +44 796 111 2055
Giles.Sirett@shapeblue.com
-----Original Message-----
From: Wido den Hollander [mailto:wido@widodh.nl]
Sent: 01 December 2014 18:08
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
On 12/01/2014 06:40 PM, Rohit Yadav wrote:
> +1
>
> Wido, will the statistics be accessible online for everyone (like
> real-time) or will the PMC release aggregated data periodically?
>
I was thinking to keep the data PMC-only to start with and indeed release aggregated data periodically.
> On Monday 01 December 2014 06:38 PM, Wido den Hollander wrote:
>> Hello,
>>
>> As a project we currently don't have a lot of insight information on
>> about how CloudStack is being used. Surveys tell us a lot, but not
>> everybody fills in the survey, so we still miss a lot of information.
>>
>> That's why I've written the Usage Reporting functionality for the
>> management server which automatically sends back anonymous
>> information about a CloudStack deployment.
>>
>> It's currently in the 'reporter' branch. [0]
>>
>> By default, every 7 days it generates a JSON document with:
>> - Hosts (Number, version, type, hypervisor)
>> - Clusters (Hypervisor en Management type)
>> - Primary storage (Type and provider)
>> - Zones (Network type and providers)
>> - Instances (Number and types)
>>
>> This report is not complete yet, I'd like to add more information,
>> but that will be Management Server information.
>>
>> The code on how this report is generated is obviously 100% Open
>> Source, so end-users can always exactly see how the information was compiled.
>>
>> I want to discuss this new feature for CloudStack and the possible
>> implications it might have.
>>
>> I'm opting for a opt-out. So every new or upgraded install to 4.6.0
>> (master) will have this enabled. Yes, we have to be very explicit in
>> the Release Notes that this has been added.
>>
>> Why? It's the small price we as a project ask for using CloudStack.
>> We want a little bit of information on how CloudStack is being used
>> so that we can use this to make CloudStack even better.
>>
>> Turning it off is also just one global setting and it will never turn
>> on again.
>>
>> On the server-side there is a Python flask application [1] (found in
>> the reporter directory) which stores all the incoming information in
>> a ElasticSearch database. From there analytics can be gathered on
>> CloudStack deployments.
>>
>> It currently points to http://cs-report.widodh.nl/report which will
>> NOT be the endpoint when this is merged into master.
>>
>> For 'production' I want to have
>> https://report.cloudstack.apache.org/report where all reports are
>> submitted.
>>
>> For every setup a unique ID is determined by hashing the first row in
>> the 'version' table. This is the version + timestamp and that is
>> hashed using SHA256. Using this unique ID we can track changes in
>> deployments and see how they grow or shrink.
>>
>> Technically this wasn't that hard to implement, but the politics
>> surrounding it might be the hardest part.
>>
>> What do other have to say about this? Should there be a VOTE for this
>> feature to come into CloudStack? Opt-in, opt-out?
>>
>> Wido
>>
>> [0]:
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h
>> =refs/heads/reporter
>>
>> [1]:
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=rep
>> orter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff55
>> 20855;hb=reporter
>>
>>
>
> --
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.yadav@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue
> and our range of CloudStack related services
>
> IaaS Cloud Design &
> Build<http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment
> framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Software
> Engineering<http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure
> Support<http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training
> Courses<http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are
> intended solely for the use of the individual to whom it is addressed.
> Any views or opinions expressed are solely those of the author and do
> not necessarily represent those of Shape Blue Ltd or related companies.
> If you are not the intended recipient of this email, you must neither
> take any action based upon its contents, nor copy or show it to anyone.
> Please contact the sender if you believe you have received this email
> in error. Shape Blue Ltd is a company incorporated in England & Wales.
> ShapeBlue Services India LLP is a company incorporated in India and is
> operated under license from Shape Blue Ltd. Shape Blue Brasil
> Consultoria Ltda is a company incorporated in Brasil and is operated
> under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company
> registered by The Republic of South Africa and is traded under license
> from Shape Blue Ltd. ShapeBlue is a registered trademark.
Find out more about ShapeBlue and our range of CloudStack related services
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Wido den Hollander <wi...@widodh.nl>.
On 12/01/2014 06:40 PM, Rohit Yadav wrote:
> +1
>
> Wido, will the statistics be accessible online for everyone (like
> real-time) or will the PMC release aggregated data periodically?
>
I was thinking to keep the data PMC-only to start with and indeed
release aggregated data periodically.
> On Monday 01 December 2014 06:38 PM, Wido den Hollander wrote:
>> Hello,
>>
>> As a project we currently don't have a lot of insight information on
>> about how CloudStack is being used. Surveys tell us a lot, but not
>> everybody fills in the survey, so we still miss a lot of information.
>>
>> That's why I've written the Usage Reporting functionality for the
>> management server which automatically sends back anonymous information
>> about a CloudStack deployment.
>>
>> It's currently in the 'reporter' branch. [0]
>>
>> By default, every 7 days it generates a JSON document with:
>> - Hosts (Number, version, type, hypervisor)
>> - Clusters (Hypervisor en Management type)
>> - Primary storage (Type and provider)
>> - Zones (Network type and providers)
>> - Instances (Number and types)
>>
>> This report is not complete yet, I'd like to add more information, but
>> that will be Management Server information.
>>
>> The code on how this report is generated is obviously 100% Open Source,
>> so end-users can always exactly see how the information was compiled.
>>
>> I want to discuss this new feature for CloudStack and the possible
>> implications it might have.
>>
>> I'm opting for a opt-out. So every new or upgraded install to 4.6.0
>> (master) will have this enabled. Yes, we have to be very explicit in the
>> Release Notes that this has been added.
>>
>> Why? It's the small price we as a project ask for using CloudStack. We
>> want a little bit of information on how CloudStack is being used so that
>> we can use this to make CloudStack even better.
>>
>> Turning it off is also just one global setting and it will never turn on
>> again.
>>
>> On the server-side there is a Python flask application [1] (found in the
>> reporter directory) which stores all the incoming information in a
>> ElasticSearch database. From there analytics can be gathered on
>> CloudStack deployments.
>>
>> It currently points to http://cs-report.widodh.nl/report which will NOT
>> be the endpoint when this is merged into master.
>>
>> For 'production' I want to have
>> https://report.cloudstack.apache.org/report where all reports are
>> submitted.
>>
>> For every setup a unique ID is determined by hashing the first row in
>> the 'version' table. This is the version + timestamp and that is hashed
>> using SHA256. Using this unique ID we can track changes in deployments
>> and see how they grow or shrink.
>>
>> Technically this wasn't that hard to implement, but the politics
>> surrounding it might be the hardest part.
>>
>> What do other have to say about this? Should there be a VOTE for this
>> feature to come into CloudStack? Opt-in, opt-out?
>>
>> Wido
>>
>> [0]:
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
>>
>> [1]:
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter
>>
>>
>
> --
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.yadav@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design &
> Build<http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Software
> Engineering<http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure
> Support<http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training
> Courses<http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are
> intended solely for the use of the individual to whom it is addressed.
> Any views or opinions expressed are solely those of the author and do
> not necessarily represent those of Shape Blue Ltd or related companies.
> If you are not the intended recipient of this email, you must neither
> take any action based upon its contents, nor copy or show it to anyone.
> Please contact the sender if you believe you have received this email in
> error. Shape Blue Ltd is a company incorporated in England & Wales.
> ShapeBlue Services India LLP is a company incorporated in India and is
> operated under license from Shape Blue Ltd. Shape Blue Brasil
> Consultoria Ltda is a company incorporated in Brasil and is operated
> under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company
> registered by The Republic of South Africa and is traded under license
> from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Rohit Yadav <ro...@shapeblue.com>.
+1
Wido, will the statistics be accessible online for everyone (like
real-time) or will the PMC release aggregated data periodically?
On Monday 01 December 2014 06:38 PM, Wido den Hollander wrote:
> Hello,
>
> As a project we currently don't have a lot of insight information on
> about how CloudStack is being used. Surveys tell us a lot, but not
> everybody fills in the survey, so we still miss a lot of information.
>
> That's why I've written the Usage Reporting functionality for the
> management server which automatically sends back anonymous information
> about a CloudStack deployment.
>
> It's currently in the 'reporter' branch. [0]
>
> By default, every 7 days it generates a JSON document with:
> - Hosts (Number, version, type, hypervisor)
> - Clusters (Hypervisor en Management type)
> - Primary storage (Type and provider)
> - Zones (Network type and providers)
> - Instances (Number and types)
>
> This report is not complete yet, I'd like to add more information, but
> that will be Management Server information.
>
> The code on how this report is generated is obviously 100% Open Source,
> so end-users can always exactly see how the information was compiled.
>
> I want to discuss this new feature for CloudStack and the possible
> implications it might have.
>
> I'm opting for a opt-out. So every new or upgraded install to 4.6.0
> (master) will have this enabled. Yes, we have to be very explicit in the
> Release Notes that this has been added.
>
> Why? It's the small price we as a project ask for using CloudStack. We
> want a little bit of information on how CloudStack is being used so that
> we can use this to make CloudStack even better.
>
> Turning it off is also just one global setting and it will never turn on
> again.
>
> On the server-side there is a Python flask application [1] (found in the
> reporter directory) which stores all the incoming information in a
> ElasticSearch database. From there analytics can be gathered on
> CloudStack deployments.
>
> It currently points to http://cs-report.widodh.nl/report which will NOT
> be the endpoint when this is merged into master.
>
> For 'production' I want to have
> https://report.cloudstack.apache.org/report where all reports are submitted.
>
> For every setup a unique ID is determined by hashing the first row in
> the 'version' table. This is the version + timestamp and that is hashed
> using SHA256. Using this unique ID we can track changes in deployments
> and see how they grow or shrink.
>
> Technically this wasn't that hard to implement, but the politics
> surrounding it might be the hardest part.
>
> What do other have to say about this? Should there be a VOTE for this
> feature to come into CloudStack? Opt-in, opt-out?
>
> Wido
>
> [0]:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
> [1]:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter
>
--
Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.yadav@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab
Find out more about ShapeBlue and our range of CloudStack related services
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Wido den Hollander <wi...@widodh.nl>.
On 12/01/2014 02:15 PM, Sebastien Goasguen wrote:
>
> On Dec 1, 2014, at 8:08 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>
>> Hello,
>>
>> As a project we currently don't have a lot of insight information on
>> about how CloudStack is being used. Surveys tell us a lot, but not
>> everybody fills in the survey, so we still miss a lot of information.
>>
>> That's why I've written the Usage Reporting functionality for the
>> management server which automatically sends back anonymous information
>> about a CloudStack deployment.
>>
>> It's currently in the 'reporter' branch. [0]
>>
>> By default, every 7 days it generates a JSON document with:
>> - Hosts (Number, version, type, hypervisor)
>> - Clusters (Hypervisor en Management type)
>> - Primary storage (Type and provider)
>> - Zones (Network type and providers)
>> - Instances (Number and types)
>>
>> This report is not complete yet, I'd like to add more information, but
>> that will be Management Server information.
>>
>> The code on how this report is generated is obviously 100% Open Source,
>> so end-users can always exactly see how the information was compiled.
>>
>> I want to discuss this new feature for CloudStack and the possible
>> implications it might have.
>>
>> I'm opting for a opt-out. So every new or upgraded install to 4.6.0
>> (master) will have this enabled. Yes, we have to be very explicit in the
>> Release Notes that this has been added.
>>
>> Why? It's the small price we as a project ask for using CloudStack. We
>> want a little bit of information on how CloudStack is being used so that
>> we can use this to make CloudStack even better.
>>
>> Turning it off is also just one global setting and it will never turn on
>> again.
>>
>> On the server-side there is a Python flask application [1] (found in the
>> reporter directory) which stores all the incoming information in a
>> ElasticSearch database. From there analytics can be gathered on
>> CloudStack deployments.
>>
>> It currently points to http://cs-report.widodh.nl/report which will NOT
>> be the endpoint when this is merged into master.
>>
>> For 'production' I want to have
>> https://report.cloudstack.apache.org/report where all reports are submitted.
>>
>> For every setup a unique ID is determined by hashing the first row in
>> the 'version' table. This is the version + timestamp and that is hashed
>> using SHA256. Using this unique ID we can track changes in deployments
>> and see how they grow or shrink.
>>
>> Technically this wasn't that hard to implement, but the politics
>> surrounding it might be the hardest part.
>>
>> What do other have to say about this? Should there be a VOTE for this
>> feature to come into CloudStack? Opt-in, opt-out?
>>
>
> We should ask users@ as well.
>
Yes, that is probably a good idea. First I'd like to have the politics
set before we take it to users.
In the end I'd like to see a VOTE coming up for this feature to go into
CloudStack.
Wido
> I am +1 on this,
>
>> Wido
>>
>> [0]:
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
>> [1]:
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter
>
Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Andrija Panic <an...@gmail.com>.
+1
On 1 December 2014 at 14:31, Erik Weber <te...@gmail.com> wrote:
> On Mon, Dec 1, 2014 at 2:15 PM, Sebastien Goasguen <ru...@gmail.com>
> wrote:
>
> >
> > On Dec 1, 2014, at 8:08 AM, Wido den Hollander <wi...@widodh.nl> wrote:
> >
> > > Hello,
> > >
> > > As a project we currently don't have a lot of insight information on
> > > about how CloudStack is being used. Surveys tell us a lot, but not
> > > everybody fills in the survey, so we still miss a lot of information.
> > >
> > > That's why I've written the Usage Reporting functionality for the
> > > management server which automatically sends back anonymous information
> > > about a CloudStack deployment.
> > >
> > > It's currently in the 'reporter' branch. [0]
> > >
> > > By default, every 7 days it generates a JSON document with:
> > > - Hosts (Number, version, type, hypervisor)
> > > - Clusters (Hypervisor en Management type)
> > > - Primary storage (Type and provider)
> > > - Zones (Network type and providers)
> > > - Instances (Number and types)
> > >
> > > This report is not complete yet, I'd like to add more information, but
> > > that will be Management Server information.
> > >
> > > The code on how this report is generated is obviously 100% Open Source,
> > > so end-users can always exactly see how the information was compiled.
> > >
> > > I want to discuss this new feature for CloudStack and the possible
> > > implications it might have.
> > >
> > > I'm opting for a opt-out. So every new or upgraded install to 4.6.0
> > > (master) will have this enabled. Yes, we have to be very explicit in
> the
> > > Release Notes that this has been added.
> > >
> > > Why? It's the small price we as a project ask for using CloudStack. We
> > > want a little bit of information on how CloudStack is being used so
> that
> > > we can use this to make CloudStack even better.
> > >
> > > Turning it off is also just one global setting and it will never turn
> on
> > > again.
> > >
> > > On the server-side there is a Python flask application [1] (found in
> the
> > > reporter directory) which stores all the incoming information in a
> > > ElasticSearch database. From there analytics can be gathered on
> > > CloudStack deployments.
> > >
> > > It currently points to http://cs-report.widodh.nl/report which will
> NOT
> > > be the endpoint when this is merged into master.
> > >
> > > For 'production' I want to have
> > > https://report.cloudstack.apache.org/report where all reports are
> > submitted.
> > >
> > > For every setup a unique ID is determined by hashing the first row in
> > > the 'version' table. This is the version + timestamp and that is hashed
> > > using SHA256. Using this unique ID we can track changes in deployments
> > > and see how they grow or shrink.
> > >
> > > Technically this wasn't that hard to implement, but the politics
> > > surrounding it might be the hardest part.
> > >
> > > What do other have to say about this? Should there be a VOTE for this
> > > feature to come into CloudStack? Opt-in, opt-out?
> > >
> >
> > We should ask users@ as well.
> >
> > I am +1 on this,
>
>
>
> As a cloudstack user I don't have any problem with this, +1.
>
> --
> Erik
>
--
Andrija Panić
Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Erik Weber <te...@gmail.com>.
On Mon, Dec 1, 2014 at 2:15 PM, Sebastien Goasguen <ru...@gmail.com> wrote:
>
> On Dec 1, 2014, at 8:08 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>
> > Hello,
> >
> > As a project we currently don't have a lot of insight information on
> > about how CloudStack is being used. Surveys tell us a lot, but not
> > everybody fills in the survey, so we still miss a lot of information.
> >
> > That's why I've written the Usage Reporting functionality for the
> > management server which automatically sends back anonymous information
> > about a CloudStack deployment.
> >
> > It's currently in the 'reporter' branch. [0]
> >
> > By default, every 7 days it generates a JSON document with:
> > - Hosts (Number, version, type, hypervisor)
> > - Clusters (Hypervisor en Management type)
> > - Primary storage (Type and provider)
> > - Zones (Network type and providers)
> > - Instances (Number and types)
> >
> > This report is not complete yet, I'd like to add more information, but
> > that will be Management Server information.
> >
> > The code on how this report is generated is obviously 100% Open Source,
> > so end-users can always exactly see how the information was compiled.
> >
> > I want to discuss this new feature for CloudStack and the possible
> > implications it might have.
> >
> > I'm opting for a opt-out. So every new or upgraded install to 4.6.0
> > (master) will have this enabled. Yes, we have to be very explicit in the
> > Release Notes that this has been added.
> >
> > Why? It's the small price we as a project ask for using CloudStack. We
> > want a little bit of information on how CloudStack is being used so that
> > we can use this to make CloudStack even better.
> >
> > Turning it off is also just one global setting and it will never turn on
> > again.
> >
> > On the server-side there is a Python flask application [1] (found in the
> > reporter directory) which stores all the incoming information in a
> > ElasticSearch database. From there analytics can be gathered on
> > CloudStack deployments.
> >
> > It currently points to http://cs-report.widodh.nl/report which will NOT
> > be the endpoint when this is merged into master.
> >
> > For 'production' I want to have
> > https://report.cloudstack.apache.org/report where all reports are
> submitted.
> >
> > For every setup a unique ID is determined by hashing the first row in
> > the 'version' table. This is the version + timestamp and that is hashed
> > using SHA256. Using this unique ID we can track changes in deployments
> > and see how they grow or shrink.
> >
> > Technically this wasn't that hard to implement, but the politics
> > surrounding it might be the hardest part.
> >
> > What do other have to say about this? Should there be a VOTE for this
> > feature to come into CloudStack? Opt-in, opt-out?
> >
>
> We should ask users@ as well.
>
> I am +1 on this,
As a cloudstack user I don't have any problem with this, +1.
--
Erik
Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Sebastien Goasguen <ru...@gmail.com>.
On Dec 1, 2014, at 8:08 AM, Wido den Hollander <wi...@widodh.nl> wrote:
> Hello,
>
> As a project we currently don't have a lot of insight information on
> about how CloudStack is being used. Surveys tell us a lot, but not
> everybody fills in the survey, so we still miss a lot of information.
>
> That's why I've written the Usage Reporting functionality for the
> management server which automatically sends back anonymous information
> about a CloudStack deployment.
>
> It's currently in the 'reporter' branch. [0]
>
> By default, every 7 days it generates a JSON document with:
> - Hosts (Number, version, type, hypervisor)
> - Clusters (Hypervisor en Management type)
> - Primary storage (Type and provider)
> - Zones (Network type and providers)
> - Instances (Number and types)
>
> This report is not complete yet, I'd like to add more information, but
> that will be Management Server information.
>
> The code on how this report is generated is obviously 100% Open Source,
> so end-users can always exactly see how the information was compiled.
>
> I want to discuss this new feature for CloudStack and the possible
> implications it might have.
>
> I'm opting for a opt-out. So every new or upgraded install to 4.6.0
> (master) will have this enabled. Yes, we have to be very explicit in the
> Release Notes that this has been added.
>
> Why? It's the small price we as a project ask for using CloudStack. We
> want a little bit of information on how CloudStack is being used so that
> we can use this to make CloudStack even better.
>
> Turning it off is also just one global setting and it will never turn on
> again.
>
> On the server-side there is a Python flask application [1] (found in the
> reporter directory) which stores all the incoming information in a
> ElasticSearch database. From there analytics can be gathered on
> CloudStack deployments.
>
> It currently points to http://cs-report.widodh.nl/report which will NOT
> be the endpoint when this is merged into master.
>
> For 'production' I want to have
> https://report.cloudstack.apache.org/report where all reports are submitted.
>
> For every setup a unique ID is determined by hashing the first row in
> the 'version' table. This is the version + timestamp and that is hashed
> using SHA256. Using this unique ID we can track changes in deployments
> and see how they grow or shrink.
>
> Technically this wasn't that hard to implement, but the politics
> surrounding it might be the hardest part.
>
> What do other have to say about this? Should there be a VOTE for this
> feature to come into CloudStack? Opt-in, opt-out?
>
We should ask users@ as well.
I am +1 on this,
> Wido
>
> [0]:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
> [1]:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter
RE: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Matthew Midgett <cl...@trick-solutions.com.INVALID>.
1+ from a user
-----Original Message-----
From: Wido den Hollander [mailto:wido@widodh.nl]
Sent: Monday, December 01, 2014 10:21 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
On 12/01/2014 04:12 PM, Nux! wrote:
> Hm, after an upgrade it might be tricky. Perhaps display it here
> somehow http://img.nux.ro/3hH-callhome.png
> Perhaps some UI/javascript specialist can help us here ...
>
Good idea. In that Menu we probably need a "Call-Home" link pointing to http://cloudstack.apache.org/callhome.html (doesn't exist yet!)
Wido
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
>> From: "Wido den Hollander" <wi...@widodh.nl>
>> To: dev@cloudstack.apache.org
>> Sent: Monday, 1 December, 2014 15:05:55
>> Subject: Re: [DISCUSS] Automatic usage reporting / call-home for
>> CloudStack
>
>> On 12/01/2014 04:00 PM, Nux! wrote:
>>> In addition: this should be opt-out, but clearly specified in the
>>> release notes, as well as visible in the UI.
>>>
>>
>> But where do we display this in the UI? I don't think we have a
>> mechanism to show a one-time message after a upgrade. Do we?
>>
>>> --
>>> Sent from the Delta quadrant using Borg technology!
>>>
>>> Nux!
>>> www.nux.ro
>>>
>>> ----- Original Message -----
>>>> From: "Nux!" <nu...@li.nux.ro>
>>>> To: dev@cloudstack.apache.org
>>>> Sent: Monday, 1 December, 2014 13:52:35
>>>> Subject: Re: [DISCUSS] Automatic usage reporting / call-home for
>>>> CloudStack
>>>
>>>> +1, good job!
>>>>
>>>> --
>>>> Sent from the Delta quadrant using Borg technology!
>>>>
>>>> Nux!
>>>> www.nux.ro
>>>>
>>>> ----- Original Message -----
>>>>> From: "Wido den Hollander" <wi...@widodh.nl>
>>>>> To: dev@cloudstack.apache.org
>>>>> Sent: Monday, 1 December, 2014 13:08:02
>>>>> Subject: [DISCUSS] Automatic usage reporting / call-home for
>>>>> CloudStack
>>>>
>>>>> Hello,
>>>>>
>>>>> As a project we currently don't have a lot of insight information
>>>>> on about how CloudStack is being used. Surveys tell us a lot, but
>>>>> not everybody fills in the survey, so we still miss a lot of information.
>>>>>
>>>>> That's why I've written the Usage Reporting functionality for the
>>>>> management server which automatically sends back anonymous
>>>>> information about a CloudStack deployment.
>>>>>
>>>>> It's currently in the 'reporter' branch. [0]
>>>>>
>>>>> By default, every 7 days it generates a JSON document with:
>>>>> - Hosts (Number, version, type, hypervisor)
>>>>> - Clusters (Hypervisor en Management type)
>>>>> - Primary storage (Type and provider)
>>>>> - Zones (Network type and providers)
>>>>> - Instances (Number and types)
>>>>>
>>>>> This report is not complete yet, I'd like to add more information,
>>>>> but that will be Management Server information.
>>>>>
>>>>> The code on how this report is generated is obviously 100% Open
>>>>> Source, so end-users can always exactly see how the information was compiled.
>>>>>
>>>>> I want to discuss this new feature for CloudStack and the possible
>>>>> implications it might have.
>>>>>
>>>>> I'm opting for a opt-out. So every new or upgraded install to
>>>>> 4.6.0
>>>>> (master) will have this enabled. Yes, we have to be very explicit
>>>>> in the Release Notes that this has been added.
>>>>>
>>>>> Why? It's the small price we as a project ask for using
>>>>> CloudStack. We want a little bit of information on how CloudStack
>>>>> is being used so that we can use this to make CloudStack even better.
>>>>>
>>>>> Turning it off is also just one global setting and it will never
>>>>> turn on again.
>>>>>
>>>>> On the server-side there is a Python flask application [1] (found
>>>>> in the reporter directory) which stores all the incoming
>>>>> information in a ElasticSearch database. From there analytics can
>>>>> be gathered on CloudStack deployments.
>>>>>
>>>>> It currently points to http://cs-report.widodh.nl/report which
>>>>> will NOT be the endpoint when this is merged into master.
>>>>>
>>>>> For 'production' I want to have
>>>>> https://report.cloudstack.apache.org/report where all reports are submitted.
>>>>>
>>>>> For every setup a unique ID is determined by hashing the first row
>>>>> in the 'version' table. This is the version + timestamp and that
>>>>> is hashed using SHA256. Using this unique ID we can track changes
>>>>> in deployments and see how they grow or shrink.
>>>>>
>>>>> Technically this wasn't that hard to implement, but the politics
>>>>> surrounding it might be the hardest part.
>>>>>
>>>>> What do other have to say about this? Should there be a VOTE for
>>>>> this feature to come into CloudStack? Opt-in, opt-out?
>>>>>
>>>>> Wido
>>>>>
>>>>> [0]:
>>>>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlo
>>>>> g;h=refs/heads/reporter
>>>>> [1]:
>>>>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=
>>>>> reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460
>>>>> cfff5520855;hb=reporter
Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Wido den Hollander <wi...@widodh.nl>.
On 12/01/2014 04:12 PM, Nux! wrote:
> Hm, after an upgrade it might be tricky. Perhaps display it here somehow http://img.nux.ro/3hH-callhome.png
> Perhaps some UI/javascript specialist can help us here ...
>
Good idea. In that Menu we probably need a "Call-Home" link pointing to
http://cloudstack.apache.org/callhome.html (doesn't exist yet!)
Wido
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
>> From: "Wido den Hollander" <wi...@widodh.nl>
>> To: dev@cloudstack.apache.org
>> Sent: Monday, 1 December, 2014 15:05:55
>> Subject: Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
>
>> On 12/01/2014 04:00 PM, Nux! wrote:
>>> In addition: this should be opt-out, but clearly specified in the release notes,
>>> as well as visible in the UI.
>>>
>>
>> But where do we display this in the UI? I don't think we have a
>> mechanism to show a one-time message after a upgrade. Do we?
>>
>>> --
>>> Sent from the Delta quadrant using Borg technology!
>>>
>>> Nux!
>>> www.nux.ro
>>>
>>> ----- Original Message -----
>>>> From: "Nux!" <nu...@li.nux.ro>
>>>> To: dev@cloudstack.apache.org
>>>> Sent: Monday, 1 December, 2014 13:52:35
>>>> Subject: Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
>>>
>>>> +1, good job!
>>>>
>>>> --
>>>> Sent from the Delta quadrant using Borg technology!
>>>>
>>>> Nux!
>>>> www.nux.ro
>>>>
>>>> ----- Original Message -----
>>>>> From: "Wido den Hollander" <wi...@widodh.nl>
>>>>> To: dev@cloudstack.apache.org
>>>>> Sent: Monday, 1 December, 2014 13:08:02
>>>>> Subject: [DISCUSS] Automatic usage reporting / call-home for CloudStack
>>>>
>>>>> Hello,
>>>>>
>>>>> As a project we currently don't have a lot of insight information on
>>>>> about how CloudStack is being used. Surveys tell us a lot, but not
>>>>> everybody fills in the survey, so we still miss a lot of information.
>>>>>
>>>>> That's why I've written the Usage Reporting functionality for the
>>>>> management server which automatically sends back anonymous information
>>>>> about a CloudStack deployment.
>>>>>
>>>>> It's currently in the 'reporter' branch. [0]
>>>>>
>>>>> By default, every 7 days it generates a JSON document with:
>>>>> - Hosts (Number, version, type, hypervisor)
>>>>> - Clusters (Hypervisor en Management type)
>>>>> - Primary storage (Type and provider)
>>>>> - Zones (Network type and providers)
>>>>> - Instances (Number and types)
>>>>>
>>>>> This report is not complete yet, I'd like to add more information, but
>>>>> that will be Management Server information.
>>>>>
>>>>> The code on how this report is generated is obviously 100% Open Source,
>>>>> so end-users can always exactly see how the information was compiled.
>>>>>
>>>>> I want to discuss this new feature for CloudStack and the possible
>>>>> implications it might have.
>>>>>
>>>>> I'm opting for a opt-out. So every new or upgraded install to 4.6.0
>>>>> (master) will have this enabled. Yes, we have to be very explicit in the
>>>>> Release Notes that this has been added.
>>>>>
>>>>> Why? It's the small price we as a project ask for using CloudStack. We
>>>>> want a little bit of information on how CloudStack is being used so that
>>>>> we can use this to make CloudStack even better.
>>>>>
>>>>> Turning it off is also just one global setting and it will never turn on
>>>>> again.
>>>>>
>>>>> On the server-side there is a Python flask application [1] (found in the
>>>>> reporter directory) which stores all the incoming information in a
>>>>> ElasticSearch database. From there analytics can be gathered on
>>>>> CloudStack deployments.
>>>>>
>>>>> It currently points to http://cs-report.widodh.nl/report which will NOT
>>>>> be the endpoint when this is merged into master.
>>>>>
>>>>> For 'production' I want to have
>>>>> https://report.cloudstack.apache.org/report where all reports are submitted.
>>>>>
>>>>> For every setup a unique ID is determined by hashing the first row in
>>>>> the 'version' table. This is the version + timestamp and that is hashed
>>>>> using SHA256. Using this unique ID we can track changes in deployments
>>>>> and see how they grow or shrink.
>>>>>
>>>>> Technically this wasn't that hard to implement, but the politics
>>>>> surrounding it might be the hardest part.
>>>>>
>>>>> What do other have to say about this? Should there be a VOTE for this
>>>>> feature to come into CloudStack? Opt-in, opt-out?
>>>>>
>>>>> Wido
>>>>>
>>>>> [0]:
>>>>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
>>>>> [1]:
>>>>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter
Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Nux! <nu...@li.nux.ro>.
Hm, after an upgrade it might be tricky. Perhaps display it here somehow http://img.nux.ro/3hH-callhome.png
Perhaps some UI/javascript specialist can help us here ...
Lucian
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Wido den Hollander" <wi...@widodh.nl>
> To: dev@cloudstack.apache.org
> Sent: Monday, 1 December, 2014 15:05:55
> Subject: Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
> On 12/01/2014 04:00 PM, Nux! wrote:
>> In addition: this should be opt-out, but clearly specified in the release notes,
>> as well as visible in the UI.
>>
>
> But where do we display this in the UI? I don't think we have a
> mechanism to show a one-time message after a upgrade. Do we?
>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>> ----- Original Message -----
>>> From: "Nux!" <nu...@li.nux.ro>
>>> To: dev@cloudstack.apache.org
>>> Sent: Monday, 1 December, 2014 13:52:35
>>> Subject: Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
>>
>>> +1, good job!
>>>
>>> --
>>> Sent from the Delta quadrant using Borg technology!
>>>
>>> Nux!
>>> www.nux.ro
>>>
>>> ----- Original Message -----
>>>> From: "Wido den Hollander" <wi...@widodh.nl>
>>>> To: dev@cloudstack.apache.org
>>>> Sent: Monday, 1 December, 2014 13:08:02
>>>> Subject: [DISCUSS] Automatic usage reporting / call-home for CloudStack
>>>
>>>> Hello,
>>>>
>>>> As a project we currently don't have a lot of insight information on
>>>> about how CloudStack is being used. Surveys tell us a lot, but not
>>>> everybody fills in the survey, so we still miss a lot of information.
>>>>
>>>> That's why I've written the Usage Reporting functionality for the
>>>> management server which automatically sends back anonymous information
>>>> about a CloudStack deployment.
>>>>
>>>> It's currently in the 'reporter' branch. [0]
>>>>
>>>> By default, every 7 days it generates a JSON document with:
>>>> - Hosts (Number, version, type, hypervisor)
>>>> - Clusters (Hypervisor en Management type)
>>>> - Primary storage (Type and provider)
>>>> - Zones (Network type and providers)
>>>> - Instances (Number and types)
>>>>
>>>> This report is not complete yet, I'd like to add more information, but
>>>> that will be Management Server information.
>>>>
>>>> The code on how this report is generated is obviously 100% Open Source,
>>>> so end-users can always exactly see how the information was compiled.
>>>>
>>>> I want to discuss this new feature for CloudStack and the possible
>>>> implications it might have.
>>>>
>>>> I'm opting for a opt-out. So every new or upgraded install to 4.6.0
>>>> (master) will have this enabled. Yes, we have to be very explicit in the
>>>> Release Notes that this has been added.
>>>>
>>>> Why? It's the small price we as a project ask for using CloudStack. We
>>>> want a little bit of information on how CloudStack is being used so that
>>>> we can use this to make CloudStack even better.
>>>>
>>>> Turning it off is also just one global setting and it will never turn on
>>>> again.
>>>>
>>>> On the server-side there is a Python flask application [1] (found in the
>>>> reporter directory) which stores all the incoming information in a
>>>> ElasticSearch database. From there analytics can be gathered on
>>>> CloudStack deployments.
>>>>
>>>> It currently points to http://cs-report.widodh.nl/report which will NOT
>>>> be the endpoint when this is merged into master.
>>>>
>>>> For 'production' I want to have
>>>> https://report.cloudstack.apache.org/report where all reports are submitted.
>>>>
>>>> For every setup a unique ID is determined by hashing the first row in
>>>> the 'version' table. This is the version + timestamp and that is hashed
>>>> using SHA256. Using this unique ID we can track changes in deployments
>>>> and see how they grow or shrink.
>>>>
>>>> Technically this wasn't that hard to implement, but the politics
>>>> surrounding it might be the hardest part.
>>>>
>>>> What do other have to say about this? Should there be a VOTE for this
>>>> feature to come into CloudStack? Opt-in, opt-out?
>>>>
>>>> Wido
>>>>
>>>> [0]:
>>>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
>>>> [1]:
> >>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter
Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Wido den Hollander <wi...@widodh.nl>.
On 12/01/2014 04:00 PM, Nux! wrote:
> In addition: this should be opt-out, but clearly specified in the release notes, as well as visible in the UI.
>
But where do we display this in the UI? I don't think we have a
mechanism to show a one-time message after a upgrade. Do we?
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
>> From: "Nux!" <nu...@li.nux.ro>
>> To: dev@cloudstack.apache.org
>> Sent: Monday, 1 December, 2014 13:52:35
>> Subject: Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
>
>> +1, good job!
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>> ----- Original Message -----
>>> From: "Wido den Hollander" <wi...@widodh.nl>
>>> To: dev@cloudstack.apache.org
>>> Sent: Monday, 1 December, 2014 13:08:02
>>> Subject: [DISCUSS] Automatic usage reporting / call-home for CloudStack
>>
>>> Hello,
>>>
>>> As a project we currently don't have a lot of insight information on
>>> about how CloudStack is being used. Surveys tell us a lot, but not
>>> everybody fills in the survey, so we still miss a lot of information.
>>>
>>> That's why I've written the Usage Reporting functionality for the
>>> management server which automatically sends back anonymous information
>>> about a CloudStack deployment.
>>>
>>> It's currently in the 'reporter' branch. [0]
>>>
>>> By default, every 7 days it generates a JSON document with:
>>> - Hosts (Number, version, type, hypervisor)
>>> - Clusters (Hypervisor en Management type)
>>> - Primary storage (Type and provider)
>>> - Zones (Network type and providers)
>>> - Instances (Number and types)
>>>
>>> This report is not complete yet, I'd like to add more information, but
>>> that will be Management Server information.
>>>
>>> The code on how this report is generated is obviously 100% Open Source,
>>> so end-users can always exactly see how the information was compiled.
>>>
>>> I want to discuss this new feature for CloudStack and the possible
>>> implications it might have.
>>>
>>> I'm opting for a opt-out. So every new or upgraded install to 4.6.0
>>> (master) will have this enabled. Yes, we have to be very explicit in the
>>> Release Notes that this has been added.
>>>
>>> Why? It's the small price we as a project ask for using CloudStack. We
>>> want a little bit of information on how CloudStack is being used so that
>>> we can use this to make CloudStack even better.
>>>
>>> Turning it off is also just one global setting and it will never turn on
>>> again.
>>>
>>> On the server-side there is a Python flask application [1] (found in the
>>> reporter directory) which stores all the incoming information in a
>>> ElasticSearch database. From there analytics can be gathered on
>>> CloudStack deployments.
>>>
>>> It currently points to http://cs-report.widodh.nl/report which will NOT
>>> be the endpoint when this is merged into master.
>>>
>>> For 'production' I want to have
>>> https://report.cloudstack.apache.org/report where all reports are submitted.
>>>
>>> For every setup a unique ID is determined by hashing the first row in
>>> the 'version' table. This is the version + timestamp and that is hashed
>>> using SHA256. Using this unique ID we can track changes in deployments
>>> and see how they grow or shrink.
>>>
>>> Technically this wasn't that hard to implement, but the politics
>>> surrounding it might be the hardest part.
>>>
>>> What do other have to say about this? Should there be a VOTE for this
>>> feature to come into CloudStack? Opt-in, opt-out?
>>>
>>> Wido
>>>
>>> [0]:
>>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
>>> [1]:
>>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter
Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Nux! <nu...@li.nux.ro>.
In addition: this should be opt-out, but clearly specified in the release notes, as well as visible in the UI.
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Nux!" <nu...@li.nux.ro>
> To: dev@cloudstack.apache.org
> Sent: Monday, 1 December, 2014 13:52:35
> Subject: Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
> +1, good job!
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
>> From: "Wido den Hollander" <wi...@widodh.nl>
>> To: dev@cloudstack.apache.org
>> Sent: Monday, 1 December, 2014 13:08:02
>> Subject: [DISCUSS] Automatic usage reporting / call-home for CloudStack
>
>> Hello,
>>
>> As a project we currently don't have a lot of insight information on
>> about how CloudStack is being used. Surveys tell us a lot, but not
>> everybody fills in the survey, so we still miss a lot of information.
>>
>> That's why I've written the Usage Reporting functionality for the
>> management server which automatically sends back anonymous information
>> about a CloudStack deployment.
>>
>> It's currently in the 'reporter' branch. [0]
>>
>> By default, every 7 days it generates a JSON document with:
>> - Hosts (Number, version, type, hypervisor)
>> - Clusters (Hypervisor en Management type)
>> - Primary storage (Type and provider)
>> - Zones (Network type and providers)
>> - Instances (Number and types)
>>
>> This report is not complete yet, I'd like to add more information, but
>> that will be Management Server information.
>>
>> The code on how this report is generated is obviously 100% Open Source,
>> so end-users can always exactly see how the information was compiled.
>>
>> I want to discuss this new feature for CloudStack and the possible
>> implications it might have.
>>
>> I'm opting for a opt-out. So every new or upgraded install to 4.6.0
>> (master) will have this enabled. Yes, we have to be very explicit in the
>> Release Notes that this has been added.
>>
>> Why? It's the small price we as a project ask for using CloudStack. We
>> want a little bit of information on how CloudStack is being used so that
>> we can use this to make CloudStack even better.
>>
>> Turning it off is also just one global setting and it will never turn on
>> again.
>>
>> On the server-side there is a Python flask application [1] (found in the
>> reporter directory) which stores all the incoming information in a
>> ElasticSearch database. From there analytics can be gathered on
>> CloudStack deployments.
>>
>> It currently points to http://cs-report.widodh.nl/report which will NOT
>> be the endpoint when this is merged into master.
>>
>> For 'production' I want to have
>> https://report.cloudstack.apache.org/report where all reports are submitted.
>>
>> For every setup a unique ID is determined by hashing the first row in
>> the 'version' table. This is the version + timestamp and that is hashed
>> using SHA256. Using this unique ID we can track changes in deployments
>> and see how they grow or shrink.
>>
>> Technically this wasn't that hard to implement, but the politics
>> surrounding it might be the hardest part.
>>
>> What do other have to say about this? Should there be a VOTE for this
>> feature to come into CloudStack? Opt-in, opt-out?
>>
>> Wido
>>
>> [0]:
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
>> [1]:
> > https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter
Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Nux! <nu...@li.nux.ro>.
+1, good job!
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Wido den Hollander" <wi...@widodh.nl>
> To: dev@cloudstack.apache.org
> Sent: Monday, 1 December, 2014 13:08:02
> Subject: [DISCUSS] Automatic usage reporting / call-home for CloudStack
> Hello,
>
> As a project we currently don't have a lot of insight information on
> about how CloudStack is being used. Surveys tell us a lot, but not
> everybody fills in the survey, so we still miss a lot of information.
>
> That's why I've written the Usage Reporting functionality for the
> management server which automatically sends back anonymous information
> about a CloudStack deployment.
>
> It's currently in the 'reporter' branch. [0]
>
> By default, every 7 days it generates a JSON document with:
> - Hosts (Number, version, type, hypervisor)
> - Clusters (Hypervisor en Management type)
> - Primary storage (Type and provider)
> - Zones (Network type and providers)
> - Instances (Number and types)
>
> This report is not complete yet, I'd like to add more information, but
> that will be Management Server information.
>
> The code on how this report is generated is obviously 100% Open Source,
> so end-users can always exactly see how the information was compiled.
>
> I want to discuss this new feature for CloudStack and the possible
> implications it might have.
>
> I'm opting for a opt-out. So every new or upgraded install to 4.6.0
> (master) will have this enabled. Yes, we have to be very explicit in the
> Release Notes that this has been added.
>
> Why? It's the small price we as a project ask for using CloudStack. We
> want a little bit of information on how CloudStack is being used so that
> we can use this to make CloudStack even better.
>
> Turning it off is also just one global setting and it will never turn on
> again.
>
> On the server-side there is a Python flask application [1] (found in the
> reporter directory) which stores all the incoming information in a
> ElasticSearch database. From there analytics can be gathered on
> CloudStack deployments.
>
> It currently points to http://cs-report.widodh.nl/report which will NOT
> be the endpoint when this is merged into master.
>
> For 'production' I want to have
> https://report.cloudstack.apache.org/report where all reports are submitted.
>
> For every setup a unique ID is determined by hashing the first row in
> the 'version' table. This is the version + timestamp and that is hashed
> using SHA256. Using this unique ID we can track changes in deployments
> and see how they grow or shrink.
>
> Technically this wasn't that hard to implement, but the politics
> surrounding it might be the hardest part.
>
> What do other have to say about this? Should there be a VOTE for this
> feature to come into CloudStack? Opt-in, opt-out?
>
> Wido
>
> [0]:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
> [1]:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter
Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Wido den Hollander <wi...@widodh.nl>.
On 27-07-15 11:52, Daan Hoogland wrote:
> gogogo
>
The PR is out there: https://github.com/apache/cloudstack/pull/625
> On Mon, Jul 27, 2015 at 10:28 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>> Hi,
>>
>> It has been quite some time and this feature hasn't merged yet.
>>
>> I created a issue for this:
>> https://issues.apache.org/jira/browse/CLOUDSTACK-8677
>>
>> The code is ready to merge and is in the "reporter" [0] branch on the
>> Git repo.
>>
>> Even if the VM isn't ready, I would like to merge this feature into 4.6
>> and get a VERY CLEAR doc into the Release Notes.
>>
>> Without Infra to send stats to, it seems like this feature is worthless,
>> but it isn't. Since we got the code running, so as soon as we get
>> call-home.cloudstack.org online we are good.
>>
>> Merging would just be sending a PR, before I do so, any objections?
>>
>> Wido
>>
>> [0]:
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
>>
>> On 01-12-14 14:08, Wido den Hollander wrote:
>>> Hello,
>>>
>>> As a project we currently don't have a lot of insight information on
>>> about how CloudStack is being used. Surveys tell us a lot, but not
>>> everybody fills in the survey, so we still miss a lot of information.
>>>
>>> That's why I've written the Usage Reporting functionality for the
>>> management server which automatically sends back anonymous information
>>> about a CloudStack deployment.
>>>
>>> It's currently in the 'reporter' branch. [0]
>>>
>>> By default, every 7 days it generates a JSON document with:
>>> - Hosts (Number, version, type, hypervisor)
>>> - Clusters (Hypervisor en Management type)
>>> - Primary storage (Type and provider)
>>> - Zones (Network type and providers)
>>> - Instances (Number and types)
>>>
>>> This report is not complete yet, I'd like to add more information, but
>>> that will be Management Server information.
>>>
>>> The code on how this report is generated is obviously 100% Open Source,
>>> so end-users can always exactly see how the information was compiled.
>>>
>>> I want to discuss this new feature for CloudStack and the possible
>>> implications it might have.
>>>
>>> I'm opting for a opt-out. So every new or upgraded install to 4.6.0
>>> (master) will have this enabled. Yes, we have to be very explicit in the
>>> Release Notes that this has been added.
>>>
>>> Why? It's the small price we as a project ask for using CloudStack. We
>>> want a little bit of information on how CloudStack is being used so that
>>> we can use this to make CloudStack even better.
>>>
>>> Turning it off is also just one global setting and it will never turn on
>>> again.
>>>
>>> On the server-side there is a Python flask application [1] (found in the
>>> reporter directory) which stores all the incoming information in a
>>> ElasticSearch database. From there analytics can be gathered on
>>> CloudStack deployments.
>>>
>>> It currently points to http://cs-report.widodh.nl/report which will NOT
>>> be the endpoint when this is merged into master.
>>>
>>> For 'production' I want to have
>>> https://report.cloudstack.apache.org/report where all reports are submitted.
>>>
>>> For every setup a unique ID is determined by hashing the first row in
>>> the 'version' table. This is the version + timestamp and that is hashed
>>> using SHA256. Using this unique ID we can track changes in deployments
>>> and see how they grow or shrink.
>>>
>>> Technically this wasn't that hard to implement, but the politics
>>> surrounding it might be the hardest part.
>>>
>>> What do other have to say about this? Should there be a VOTE for this
>>> feature to come into CloudStack? Opt-in, opt-out?
>>>
>>> Wido
>>>
>>> [0]:
>>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
>>> [1]:
>>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter
>>>
>
>
>
Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Daan Hoogland <da...@gmail.com>.
gogogo
On Mon, Jul 27, 2015 at 10:28 AM, Wido den Hollander <wi...@widodh.nl> wrote:
> Hi,
>
> It has been quite some time and this feature hasn't merged yet.
>
> I created a issue for this:
> https://issues.apache.org/jira/browse/CLOUDSTACK-8677
>
> The code is ready to merge and is in the "reporter" [0] branch on the
> Git repo.
>
> Even if the VM isn't ready, I would like to merge this feature into 4.6
> and get a VERY CLEAR doc into the Release Notes.
>
> Without Infra to send stats to, it seems like this feature is worthless,
> but it isn't. Since we got the code running, so as soon as we get
> call-home.cloudstack.org online we are good.
>
> Merging would just be sending a PR, before I do so, any objections?
>
> Wido
>
> [0]:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
>
> On 01-12-14 14:08, Wido den Hollander wrote:
>> Hello,
>>
>> As a project we currently don't have a lot of insight information on
>> about how CloudStack is being used. Surveys tell us a lot, but not
>> everybody fills in the survey, so we still miss a lot of information.
>>
>> That's why I've written the Usage Reporting functionality for the
>> management server which automatically sends back anonymous information
>> about a CloudStack deployment.
>>
>> It's currently in the 'reporter' branch. [0]
>>
>> By default, every 7 days it generates a JSON document with:
>> - Hosts (Number, version, type, hypervisor)
>> - Clusters (Hypervisor en Management type)
>> - Primary storage (Type and provider)
>> - Zones (Network type and providers)
>> - Instances (Number and types)
>>
>> This report is not complete yet, I'd like to add more information, but
>> that will be Management Server information.
>>
>> The code on how this report is generated is obviously 100% Open Source,
>> so end-users can always exactly see how the information was compiled.
>>
>> I want to discuss this new feature for CloudStack and the possible
>> implications it might have.
>>
>> I'm opting for a opt-out. So every new or upgraded install to 4.6.0
>> (master) will have this enabled. Yes, we have to be very explicit in the
>> Release Notes that this has been added.
>>
>> Why? It's the small price we as a project ask for using CloudStack. We
>> want a little bit of information on how CloudStack is being used so that
>> we can use this to make CloudStack even better.
>>
>> Turning it off is also just one global setting and it will never turn on
>> again.
>>
>> On the server-side there is a Python flask application [1] (found in the
>> reporter directory) which stores all the incoming information in a
>> ElasticSearch database. From there analytics can be gathered on
>> CloudStack deployments.
>>
>> It currently points to http://cs-report.widodh.nl/report which will NOT
>> be the endpoint when this is merged into master.
>>
>> For 'production' I want to have
>> https://report.cloudstack.apache.org/report where all reports are submitted.
>>
>> For every setup a unique ID is determined by hashing the first row in
>> the 'version' table. This is the version + timestamp and that is hashed
>> using SHA256. Using this unique ID we can track changes in deployments
>> and see how they grow or shrink.
>>
>> Technically this wasn't that hard to implement, but the politics
>> surrounding it might be the hardest part.
>>
>> What do other have to say about this? Should there be a VOTE for this
>> feature to come into CloudStack? Opt-in, opt-out?
>>
>> Wido
>>
>> [0]:
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
>> [1]:
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter
>>
--
Daan
Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack
Posted by Wido den Hollander <wi...@widodh.nl>.
Hi,
It has been quite some time and this feature hasn't merged yet.
I created a issue for this:
https://issues.apache.org/jira/browse/CLOUDSTACK-8677
The code is ready to merge and is in the "reporter" [0] branch on the
Git repo.
Even if the VM isn't ready, I would like to merge this feature into 4.6
and get a VERY CLEAR doc into the Release Notes.
Without Infra to send stats to, it seems like this feature is worthless,
but it isn't. Since we got the code running, so as soon as we get
call-home.cloudstack.org online we are good.
Merging would just be sending a PR, before I do so, any objections?
Wido
[0]:
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
On 01-12-14 14:08, Wido den Hollander wrote:
> Hello,
>
> As a project we currently don't have a lot of insight information on
> about how CloudStack is being used. Surveys tell us a lot, but not
> everybody fills in the survey, so we still miss a lot of information.
>
> That's why I've written the Usage Reporting functionality for the
> management server which automatically sends back anonymous information
> about a CloudStack deployment.
>
> It's currently in the 'reporter' branch. [0]
>
> By default, every 7 days it generates a JSON document with:
> - Hosts (Number, version, type, hypervisor)
> - Clusters (Hypervisor en Management type)
> - Primary storage (Type and provider)
> - Zones (Network type and providers)
> - Instances (Number and types)
>
> This report is not complete yet, I'd like to add more information, but
> that will be Management Server information.
>
> The code on how this report is generated is obviously 100% Open Source,
> so end-users can always exactly see how the information was compiled.
>
> I want to discuss this new feature for CloudStack and the possible
> implications it might have.
>
> I'm opting for a opt-out. So every new or upgraded install to 4.6.0
> (master) will have this enabled. Yes, we have to be very explicit in the
> Release Notes that this has been added.
>
> Why? It's the small price we as a project ask for using CloudStack. We
> want a little bit of information on how CloudStack is being used so that
> we can use this to make CloudStack even better.
>
> Turning it off is also just one global setting and it will never turn on
> again.
>
> On the server-side there is a Python flask application [1] (found in the
> reporter directory) which stores all the incoming information in a
> ElasticSearch database. From there analytics can be gathered on
> CloudStack deployments.
>
> It currently points to http://cs-report.widodh.nl/report which will NOT
> be the endpoint when this is merged into master.
>
> For 'production' I want to have
> https://report.cloudstack.apache.org/report where all reports are submitted.
>
> For every setup a unique ID is determined by hashing the first row in
> the 'version' table. This is the version + timestamp and that is hashed
> using SHA256. Using this unique ID we can track changes in deployments
> and see how they grow or shrink.
>
> Technically this wasn't that hard to implement, but the politics
> surrounding it might be the hardest part.
>
> What do other have to say about this? Should there be a VOTE for this
> feature to come into CloudStack? Opt-in, opt-out?
>
> Wido
>
> [0]:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
> [1]:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter
>