You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Lasindu Charith <la...@wso2.com> on 2014/08/03 03:49:48 UTC
Re: Convention for including Puppet forge's modules in Stratos
Hi devs,
I was able to create the nodejs cartridge using the puppet labs module [1
<https://forge.puppetlabs.com/puppetlabs/nodejs>] and write an extension to
manage the nodejs applications. It has following features
- Supports installing Node and NPM (Package Manager) in many Debian and
RedHat distributions. Currently Stratos mainly supports Debian/Ubuntu and
if we are to move into other OS's, this will be a useful feature in the
future.
- Downloads and installs all the required dependencies specified in the
package.json file in any nodejs application via npm.
- Downloads and installs Forever [2
<https://github.com/nodejitsu/forever>] which will manage keeping the
application alive and make sure applications run continuously with updates
take in place. There are several other process managers available like pm2 [
3 <https://github.com/Unitech/pm2>] with more features which can be used
as well.
- Reads the main js file from the package.json and starts the
application using forever. This way, there's no need of modifying the
nodejs application, if it has followed the standard nodejs application
structure.
Yet I have following concerns when it comes to Nodejs specific applications.
1. The nodejs application itself defines the ports in use. This may
create problems in specifying the cartridge definition prior to the
deployment. The port to be listened has to be specified correctly in the
cartridge definition in order for a particular instance to be activated
properly.
2. Since we only can specify a single cartridge definition per service
type, all the ports which will be used by the nodejs application has to be
pre-defined/pre-known and at the time of defining the cartridge definition.
Also instance might hang up waiting, listing to the ports which are
currently not used.
3. Monitoring the health would also be difficult.
4. Deploying multiple applications using the same git repository could
be done. But will have to use a separate configuration file to specify
which/how to start those applications specifically. Also the necessary
ports have to be specified in the definition prior to the deployment.
Please let me know any specifics/suggestions which I can use to overcome
the above issues.
[1] https://forge.puppetlabs.com/puppetlabs/nodejs
[2] https://github.com/nodejitsu/forever
[3] https://github.com/Unitech/pm2
Thanks,
On Sun, Jul 27, 2014 at 5:54 PM, Lakmal Warusawithana <la...@wso2.com>
wrote:
>
>
>
> On Sun, Jul 27, 2014 at 5:49 PM, Lasindu Charith <la...@wso2.com> wrote:
>
>> Hi,
>>
>> I'm trying to re-create(modify) Nodejs cartridge for Stratos. When going
>> through the PuppetLabs's forge, I found there are several nodejs puppet
>> modules in forge, even a module from puppet enterprise itself with Apache
>> 2.0 licence as well. There are many puppetlabs/puppet enterprise supported
>> modules [1
>> <https://forge.puppetlabs.com/modules?utf-8=%E2%9C%93&sort=&supported=yes>]
>> which are compatible with several OS's and are quite popular among puppet
>> community.
>>
>> So I was wondering what if we can include at-least most common puppet
>> enterprise supported modules inside stratos without re-inventing the wheel
>> out of the box.
>>
>
> +1 Lasindu. If you have time, please see what we can bring other generic
> software as cartridges.
>
>
>>
>> Is there a convention to be used ? or anyone tried using them?
>>
>> I tried replacing the existing nodejs module inside Stratos and it worked
>> fine as expected.
>>
>> [1]
>> https://forge.puppetlabs.com/modules?utf-8=%E2%9C%93&sort=&supported=yes
>>
>> Thanks,
>> --
>> *Lasindu Charith*
>> Software Engineer, WSO2 Inc.
>> Mobile: +94714427192
>> Web: blog.lasindu.com
>>
>
>
>
> --
> Lakmal Warusawithana
> Vice President, Apache Stratos
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
--
*Lasindu Charith*
Software Engineer, WSO2 Inc.
Mobile: +94714427192
Web: blog.lasindu.com
Re: Convention for including Puppet forge's modules in Stratos
Posted by Imesh Gunaratne <im...@apache.org>.
Yes may be we do not have that option currently to send payload parameters
at subscription time. May be we could add it in a future release.
Thanks
On Sun, Aug 3, 2014 at 11:54 PM, Nirmal Fernando <ni...@gmail.com>
wrote:
> I also don't think that we have an option to provide payload params on a
> subscription and we probably have this somewhere in Jira as a future
> improvement. (this can't be supported for Multi-tenant cartridges)
>
> Ports check should definitely be an extension (this was raised before too)
> and should remove the code portion which does that by default IMO.
>
>
> On Mon, Aug 4, 2014 at 1:48 AM, Sajith Kariyawasam <sa...@wso2.com>
> wrote:
>
>>
>> On Sun, Aug 3, 2014 at 6:51 PM, Imesh Gunaratne <im...@apache.org> wrote:
>>
>>>
>>> Great work Lasindu! Few comments inline:
>>>
>>> On Sat, Aug 2, 2014 at 9:49 PM, Lasindu Charith <la...@wso2.com>
>>> wrote:
>>>
>>>
>>>> Yet I have following concerns when it comes to Nodejs specific
>>>> applications.
>>>>
>>>> 1. The nodejs application itself defines the ports in use.
>>>>
>>>>
>> But still, as per my understanding that can be controlled using
>> environment variables [1] , and we should ask the application developers
>> (tenants) to use that way.
>> Because, even if it specified ports as they wish, those are anyway
>> controlled by the IaaS specific security-groups, which is pre-defined by
>> the Stratos super admins
>>
>> If we are to provide ports per subscription (As Akila also have mentioned
>> I'm not sure how / whether its supported), we need to support
>> security-groups per subscription as well
>>
>> [1]
>> http://stackoverflow.com/questions/18864677/what-is-process-env-port-in-node-js
>>
>>>
>>>> 1. This may create problems in specifying the cartridge definition
>>>> prior to the deployment. The port to be listened has to be specified
>>>> correctly in the cartridge definition in order for a particular instance to
>>>> be activated properly.
>>>>
>>>> IMO this is not a problem. Cartridge definition anyway needs to define
>>> the ports.
>>>
>>>>
>>>> 1. Since we only can specify a single cartridge definition per
>>>> service type, all the ports which will be used by the nodejs application
>>>> has to be pre-defined/pre-known and at the time of defining the cartridge
>>>> definition. Also instance might hang up waiting, listing to the ports which
>>>> are currently not used.
>>>>
>>>> Not exactly. If required we can send the ports at the subscription time
>>> using payload parameters (if the instance can configure them accordingly).
>>>
>>>>
>>>> 1. Monitoring the health would also be difficult.
>>>>
>>>> What's problem we have here? We should be able to monitor the sockets I
>>> guess.
>>>
>>>>
>>>> 1. Deploying multiple applications using the same git repository
>>>> could be done. But will have to use a separate configuration file to
>>>> specify which/how to start those applications specifically. Also the
>>>> necessary ports have to be specified in the definition prior to the
>>>> deployment.
>>>>
>>>> Our current design is to use a Git repository per
>>> application/subscription.
>>>
>>>>
>>>>
>>>> Please let me know any specifics/suggestions which I can use to
>>>> overcome the above issues.
>>>>
>>>> [1] https://forge.puppetlabs.com/puppetlabs/nodejs
>>>> [2] https://github.com/nodejitsu/forever
>>>> [3] https://github.com/Unitech/pm2
>>>>
>>>> Thanks,
>>>>
>>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PPMC Member, Apache Stratos
>>>
>>
>>
>>
>> --
>> *--*
>> *Sajith Kariyawasam*
>> *Mobile: +94772269575 <%2B94772269575>*
>>
>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PPMC Member, Apache Stratos
Re: Convention for including Puppet forge's modules in Stratos
Posted by Nirmal Fernando <ni...@gmail.com>.
I also don't think that we have an option to provide payload params on a
subscription and we probably have this somewhere in Jira as a future
improvement. (this can't be supported for Multi-tenant cartridges)
Ports check should definitely be an extension (this was raised before too)
and should remove the code portion which does that by default IMO.
On Mon, Aug 4, 2014 at 1:48 AM, Sajith Kariyawasam <sa...@wso2.com> wrote:
>
> On Sun, Aug 3, 2014 at 6:51 PM, Imesh Gunaratne <im...@apache.org> wrote:
>
>>
>> Great work Lasindu! Few comments inline:
>>
>> On Sat, Aug 2, 2014 at 9:49 PM, Lasindu Charith <la...@wso2.com> wrote:
>>
>>
>>> Yet I have following concerns when it comes to Nodejs specific
>>> applications.
>>>
>>> 1. The nodejs application itself defines the ports in use.
>>>
>>>
> But still, as per my understanding that can be controlled using
> environment variables [1] , and we should ask the application developers
> (tenants) to use that way.
> Because, even if it specified ports as they wish, those are anyway
> controlled by the IaaS specific security-groups, which is pre-defined by
> the Stratos super admins
>
> If we are to provide ports per subscription (As Akila also have mentioned
> I'm not sure how / whether its supported), we need to support
> security-groups per subscription as well
>
> [1]
> http://stackoverflow.com/questions/18864677/what-is-process-env-port-in-node-js
>
>>
>>> 1. This may create problems in specifying the cartridge definition
>>> prior to the deployment. The port to be listened has to be specified
>>> correctly in the cartridge definition in order for a particular instance to
>>> be activated properly.
>>>
>>> IMO this is not a problem. Cartridge definition anyway needs to define
>> the ports.
>>
>>>
>>> 1. Since we only can specify a single cartridge definition per
>>> service type, all the ports which will be used by the nodejs application
>>> has to be pre-defined/pre-known and at the time of defining the cartridge
>>> definition. Also instance might hang up waiting, listing to the ports which
>>> are currently not used.
>>>
>>> Not exactly. If required we can send the ports at the subscription time
>> using payload parameters (if the instance can configure them accordingly).
>>
>>>
>>> 1. Monitoring the health would also be difficult.
>>>
>>> What's problem we have here? We should be able to monitor the sockets I
>> guess.
>>
>>>
>>> 1. Deploying multiple applications using the same git repository
>>> could be done. But will have to use a separate configuration file to
>>> specify which/how to start those applications specifically. Also the
>>> necessary ports have to be specified in the definition prior to the
>>> deployment.
>>>
>>> Our current design is to use a Git repository per
>> application/subscription.
>>
>>>
>>>
>>> Please let me know any specifics/suggestions which I can use to overcome
>>> the above issues.
>>>
>>> [1] https://forge.puppetlabs.com/puppetlabs/nodejs
>>> [2] https://github.com/nodejitsu/forever
>>> [3] https://github.com/Unitech/pm2
>>>
>>> Thanks,
>>>
>>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PPMC Member, Apache Stratos
>>
>
>
>
> --
> *--*
> *Sajith Kariyawasam*
> *Mobile: +94772269575 <%2B94772269575>*
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: Convention for including Puppet forge's modules in Stratos
Posted by Sajith Kariyawasam <sa...@wso2.com>.
On Sun, Aug 3, 2014 at 6:51 PM, Imesh Gunaratne <im...@apache.org> wrote:
>
> Great work Lasindu! Few comments inline:
>
> On Sat, Aug 2, 2014 at 9:49 PM, Lasindu Charith <la...@wso2.com> wrote:
>
>
>> Yet I have following concerns when it comes to Nodejs specific
>> applications.
>>
>> 1. The nodejs application itself defines the ports in use.
>>
>>
But still, as per my understanding that can be controlled using environment
variables [1] , and we should ask the application developers (tenants) to
use that way.
Because, even if it specified ports as they wish, those are anyway
controlled by the IaaS specific security-groups, which is pre-defined by
the Stratos super admins
If we are to provide ports per subscription (As Akila also have mentioned
I'm not sure how / whether its supported), we need to support
security-groups per subscription as well
[1]
http://stackoverflow.com/questions/18864677/what-is-process-env-port-in-node-js
>
>> 1. This may create problems in specifying the cartridge definition
>> prior to the deployment. The port to be listened has to be specified
>> correctly in the cartridge definition in order for a particular instance to
>> be activated properly.
>>
>> IMO this is not a problem. Cartridge definition anyway needs to define
> the ports.
>
>>
>> 1. Since we only can specify a single cartridge definition per
>> service type, all the ports which will be used by the nodejs application
>> has to be pre-defined/pre-known and at the time of defining the cartridge
>> definition. Also instance might hang up waiting, listing to the ports which
>> are currently not used.
>>
>> Not exactly. If required we can send the ports at the subscription time
> using payload parameters (if the instance can configure them accordingly).
>
>>
>> 1. Monitoring the health would also be difficult.
>>
>> What's problem we have here? We should be able to monitor the sockets I
> guess.
>
>>
>> 1. Deploying multiple applications using the same git repository
>> could be done. But will have to use a separate configuration file to
>> specify which/how to start those applications specifically. Also the
>> necessary ports have to be specified in the definition prior to the
>> deployment.
>>
>> Our current design is to use a Git repository per
> application/subscription.
>
>>
>>
>> Please let me know any specifics/suggestions which I can use to overcome
>> the above issues.
>>
>> [1] https://forge.puppetlabs.com/puppetlabs/nodejs
>> [2] https://github.com/nodejitsu/forever
>> [3] https://github.com/Unitech/pm2
>>
>> Thanks,
>>
>>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PPMC Member, Apache Stratos
>
--
*--*
*Sajith Kariyawasam*
*Mobile: +94772269575*
Re: Convention for including Puppet forge's modules in Stratos
Posted by Isuru Haththotuwa <is...@apache.org>.
On Sun, Aug 3, 2014 at 9:17 PM, Akila Ravihansa Perera <ra...@wso2.com>
wrote:
> Hi Imesh,
>
> > Not exactly. If required we can send the ports at the subscription time
> > using payload parameters (if the instance can configure them
> accordingly).
>
> Could you please explain how this can be done? I don't see any payload
> parameter field in current cartridge subscription request json
> structure in [1].
>
> >>
> >> Monitoring the health would also be difficult.
> >
> > What's problem we have here? We should be able to monitor the sockets I
> > guess.
>
> Currently we are not periodically monitoring the sockets in cartridge
> instances that are being opened by the application running on it.
> Cartridge agent will do a one-time check on specified ports in
> cartridge deployment json. This does not guarantee the health of the
> application running on cartridge instance.
>
> Also I don't think checking whether ports are active is a good method
> of measuring application health. IMHO, we should provide a agent
> extension for this since health checking logic depends on the nature
> of the application running on the instance.
>
Checking the whether the relevant ports are active is a very simple but a
useful way of health checking. IMHO this can be the default way, while it
should be extendable to support any other logic depending on the
application.
>
>
> [1]
> https://cwiki.apache.org/confluence/display/STRATOS/4.0.0+Subscribing+to+a+Cartridge
>
> Thanks.
>
> --
> Thanks and Regards,
>
> Isuru H.
> +94 716 358 048* <http://wso2.com/>*
>
>
> * <http://wso2.com/>*
>
>
>
Re: Convention for including Puppet forge's modules in Stratos
Posted by Akila Ravihansa Perera <ra...@wso2.com>.
Hi Imesh,
> Not exactly. If required we can send the ports at the subscription time
> using payload parameters (if the instance can configure them accordingly).
Could you please explain how this can be done? I don't see any payload
parameter field in current cartridge subscription request json
structure in [1].
>>
>> Monitoring the health would also be difficult.
>
> What's problem we have here? We should be able to monitor the sockets I
> guess.
Currently we are not periodically monitoring the sockets in cartridge
instances that are being opened by the application running on it.
Cartridge agent will do a one-time check on specified ports in
cartridge deployment json. This does not guarantee the health of the
application running on cartridge instance.
Also I don't think checking whether ports are active is a good method
of measuring application health. IMHO, we should provide a agent
extension for this since health checking logic depends on the nature
of the application running on the instance.
[1] https://cwiki.apache.org/confluence/display/STRATOS/4.0.0+Subscribing+to+a+Cartridge
Thanks.
Re: Convention for including Puppet forge's modules in Stratos
Posted by Imesh Gunaratne <im...@apache.org>.
Great work Lasindu! Few comments inline:
On Sat, Aug 2, 2014 at 9:49 PM, Lasindu Charith <la...@wso2.com> wrote:
> Yet I have following concerns when it comes to Nodejs specific
> applications.
>
> 1. The nodejs application itself defines the ports in use. This may
> create problems in specifying the cartridge definition prior to the
> deployment. The port to be listened has to be specified correctly in the
> cartridge definition in order for a particular instance to be activated
> properly.
>
> IMO this is not a problem. Cartridge definition anyway needs to define the
ports.
>
> 1. Since we only can specify a single cartridge definition per service
> type, all the ports which will be used by the nodejs application has to be
> pre-defined/pre-known and at the time of defining the cartridge definition.
> Also instance might hang up waiting, listing to the ports which are
> currently not used.
>
> Not exactly. If required we can send the ports at the subscription time
using payload parameters (if the instance can configure them accordingly).
>
> 1. Monitoring the health would also be difficult.
>
> What's problem we have here? We should be able to monitor the sockets I
guess.
>
> 1. Deploying multiple applications using the same git repository could
> be done. But will have to use a separate configuration file to specify
> which/how to start those applications specifically. Also the necessary
> ports have to be specified in the definition prior to the deployment.
>
> Our current design is to use a Git repository per
application/subscription.
>
>
> Please let me know any specifics/suggestions which I can use to overcome
> the above issues.
>
> [1] https://forge.puppetlabs.com/puppetlabs/nodejs
> [2] https://github.com/nodejitsu/forever
> [3] https://github.com/Unitech/pm2
>
> Thanks,
>
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PPMC Member, Apache Stratos