You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by Otto Fowler <ot...@gmail.com> on 2017/05/08 12:06:17 UTC
[DISCUSS] REST + ambari
I was thinking about have an ambari ‘service’ in the rest api.
The initial purpose would be to be able to retrieve ambari configuration
variables for the metron service and components.
The api would be
PUT: login to ambari -> ambari credentials for use with ambari, session
variable
GET: /component | service if login provided, returns the configuration
GET:/component | service | property name if login provided get the
property value for property name
So, the caller would put the credentials, and after that point the caller
or other services can use ambari service configurations,
for example to get the configured metron hdfs root directory.
This is opposed to configuring credentials in the rest configuration.
Thoughts?
Re: [DISCUSS] REST + ambari
Posted by Otto Fowler <ot...@gmail.com>.
Thank you everyone. The property I needed was actually already in the
configuration, but not referenced in the code, so I missed it.
I have refactored to use the configuration properly.
On May 8, 2017 at 10:21:14, Simon Elliston Ball (simon@simonellistonball.com)
wrote:
Not from an ENV variable per-se, but from the spring config. Hence anything
in the application.yml can be pushed into a field in a REST service
endpoint with the @Value annotation.
Simon
On 8 May 2017, at 14:59, Otto Fowler <ot...@gmail.com> wrote:
So that stuff is read from the environment variable?
On May 8, 2017 at 09:27:48, Otto Fowler (ottobackwards@gmail.com) wrote:
OK, that is great. My works has bridged the move of rest into ambari.
Mentally, I obviously haven’t caught up :)
I’ll look for an example where we are reading application configuration
variables out in the rest service.
Thanks!
On May 8, 2017 at 09:08:46, Simon Elliston Ball (simon@simonellistonball.com)
wrote:
My proposal would be that the REST api use it’s application.yml for all the
parameters and have settings it needs included in that. E.g. the metron
directory you need is set as a property of that yml, which is then
accessible through the spring config auto-wiring
- @Value(“${metron.directory}”) private String path; in your service
endpoint.
The application.yml file is managed (generated) by Ambari on start of the
rest api service (also managed through ambari now).
All the parameters are built from a combination of user specified settings
in Ambari, or service locations. "Where is storm?” is not something a user
should have to specify; Ambari already knows. However "where would you like
the metron directory?" is an input box somewhere in Ambari. Both are
exposed to the rest api through the application.yml file.
It’s more Ambari pushes the config to the REST layer than pull from Ambari.
This way we’re using Ambari the way it is intended rather than trying to
make it be something it is not (e.g. etcd)
Simon
On 8 May 2017, at 14:00, Otto Fowler <ot...@gmail.com> wrote:
I’m not sure I understand. Could you go on a bit?
Right now, we have env parameters in ambari, that rest should honor. I
don’t understand how moving rest config into ambari get’s my rest service
access to those parameters
On May 8, 2017 at 08:56:21, Simon Elliston Ball (simon@simonellistonball.com)
wrote:
Perhaps a better way of doing this would be to push the configuration of
the REST api (the application.yml) into ambari control and expose
parameters like that. That means the REST api doesn’t have to couple
directly to Ambari, and doesn’t have to reinvent the Ambari paradigm
(Ambari is not a configuration server, it’s a configuration file manager at
heart). If the relevant parameters changed in Ambari, a restart of the REST
service (and consequently re-write of it’s application.yml) would be
forced. The REST API itself would then just use a simple spring config
expression to pick up the value.
Simon
> On 8 May 2017, at 13:52, Otto Fowler <ot...@gmail.com> wrote:
>
> The issue is that we have services that need to access the ambari rest
api from
> inside the rest server.
>
> Technically, if someone changes the defaults for the metron directory
etc, the
> rest won’t pick it up.
>
> I have a PR coming as a follow on to METRON-777 for installing extensions,
> and I write to hdfs, and hard coded to directory, but we allow the
configuration.
>
> Other rest services write to hdfs as well.
>
>
> On May 8, 2017 at 08:45:32, Nick Allen (nick@nickallen.org) wrote:
>
> As opposed to using the Ambari REST API to get this information?
>
> On Mon, May 8, 2017 at 8:06 AM, Otto Fowler <ot...@gmail.com>
wrote:
>
>> I was thinking about have an ambari ‘service’ in the rest api.
>> The initial purpose would be to be able to retrieve ambari configuration
>> variables for the metron service and components.
>>
>> The api would be
>>
>> PUT: login to ambari -> ambari credentials for use with ambari, session
>> variable
>> GET: /component | service if login provided, returns the configuration
>> GET:/component | service | property name if login provided get the
>> property value for property name
>>
>> So, the caller would put the credentials, and after that point the caller
>> or other services can use ambari service configurations,
>> for example to get the configured metron hdfs root directory.
>>
>> This is opposed to configuring credentials in the rest configuration.
>>
>> Thoughts?
>>
Re: [DISCUSS] REST + ambari
Posted by Simon Elliston Ball <si...@simonellistonball.com>.
Not from an ENV variable per-se, but from the spring config. Hence anything in the application.yml can be pushed into a field in a REST service endpoint with the @Value annotation.
Simon
> On 8 May 2017, at 14:59, Otto Fowler <ot...@gmail.com> wrote:
>
> So that stuff is read from the environment variable?
>
>
> On May 8, 2017 at 09:27:48, Otto Fowler (ottobackwards@gmail.com <ma...@gmail.com>) wrote:
>
>> OK, that is great. My works has bridged the move of rest into ambari. Mentally, I obviously haven’t caught up :)
>> I’ll look for an example where we are reading application configuration variables out in the rest service.
>>
>> Thanks!
>>
>>
>>
>> On May 8, 2017 at 09:08:46, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>>
>>> My proposal would be that the REST api use it’s application.yml for all the parameters and have settings it needs included in that. E.g. the metron directory you need is set as a property of that yml, which is then accessible through the spring config auto-wiring - @Value(“${metron.directory}”) private String path; in your service endpoint.
>>>
>>> The application.yml file is managed (generated) by Ambari on start of the rest api service (also managed through ambari now).
>>>
>>> All the parameters are built from a combination of user specified settings in Ambari, or service locations. "Where is storm?” is not something a user should have to specify; Ambari already knows. However "where would you like the metron directory?" is an input box somewhere in Ambari. Both are exposed to the rest api through the application.yml file.
>>>
>>> It’s more Ambari pushes the config to the REST layer than pull from Ambari. This way we’re using Ambari the way it is intended rather than trying to make it be something it is not (e.g. etcd)
>>>
>>> Simon
>>>
>>>
>>>
>>>> On 8 May 2017, at 14:00, Otto Fowler <ottobackwards@gmail.com <ma...@gmail.com>> wrote:
>>>>
>>>> I’m not sure I understand. Could you go on a bit?
>>>>
>>>> Right now, we have env parameters in ambari, that rest should honor. I don’t understand how moving rest config into ambari get’s my rest service access to those parameters
>>>>
>>>>
>>>>
>>>> On May 8, 2017 at 08:56:21, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>>>>
>>>>> Perhaps a better way of doing this would be to push the configuration of the REST api (the application.yml) into ambari control and expose parameters like that. That means the REST api doesn’t have to couple directly to Ambari, and doesn’t have to reinvent the Ambari paradigm (Ambari is not a configuration server, it’s a configuration file manager at heart). If the relevant parameters changed in Ambari, a restart of the REST service (and consequently re-write of it’s application.yml) would be forced. The REST API itself would then just use a simple spring config expression to pick up the value.
>>>>>
>>>>> Simon
>>>>>
>>>>> > On 8 May 2017, at 13:52, Otto Fowler <ottobackwards@gmail.com <ma...@gmail.com>> wrote:
>>>>> >
>>>>> > The issue is that we have services that need to access the ambari rest api from
>>>>> > inside the rest server.
>>>>> >
>>>>> > Technically, if someone changes the defaults for the metron directory etc, the
>>>>> > rest won’t pick it up.
>>>>> >
>>>>> > I have a PR coming as a follow on to METRON-777 for installing extensions,
>>>>> > and I write to hdfs, and hard coded to directory, but we allow the configuration.
>>>>> >
>>>>> > Other rest services write to hdfs as well.
>>>>> >
>>>>> >
>>>>> > On May 8, 2017 at 08:45:32, Nick Allen (nick@nickallen.org <ma...@nickallen.org>) wrote:
>>>>> >
>>>>> > As opposed to using the Ambari REST API to get this information?
>>>>> >
>>>>> > On Mon, May 8, 2017 at 8:06 AM, Otto Fowler <ottobackwards@gmail.com <ma...@gmail.com>> wrote:
>>>>> >
>>>>> >> I was thinking about have an ambari ‘service’ in the rest api.
>>>>> >> The initial purpose would be to be able to retrieve ambari configuration
>>>>> >> variables for the metron service and components.
>>>>> >>
>>>>> >> The api would be
>>>>> >>
>>>>> >> PUT: login to ambari -> ambari credentials for use with ambari, session
>>>>> >> variable
>>>>> >> GET: /component | service if login provided, returns the configuration
>>>>> >> GET:/component | service | property name if login provided get the
>>>>> >> property value for property name
>>>>> >>
>>>>> >> So, the caller would put the credentials, and after that point the caller
>>>>> >> or other services can use ambari service configurations,
>>>>> >> for example to get the configured metron hdfs root directory.
>>>>> >>
>>>>> >> This is opposed to configuring credentials in the rest configuration.
>>>>> >>
>>>>> >> Thoughts?
>>>>> >>
Re: [DISCUSS] REST + ambari
Posted by Otto Fowler <ot...@gmail.com>.
Can anyone describe the configuration flow from packaging/ambari to the
rest application FooConfig? If I was going to add a new ambari variable
from metron_env.xml and expose it to rest?
On May 8, 2017 at 09:59:10, Otto Fowler (ottobackwards@gmail.com) wrote:
So that stuff is read from the environment variable?
On May 8, 2017 at 09:27:48, Otto Fowler (ottobackwards@gmail.com) wrote:
OK, that is great. My works has bridged the move of rest into ambari.
Mentally, I obviously haven’t caught up :)
I’ll look for an example where we are reading application configuration
variables out in the rest service.
Thanks!
On May 8, 2017 at 09:08:46, Simon Elliston Ball (simon@simonellistonball.com)
wrote:
My proposal would be that the REST api use it’s application.yml for all the
parameters and have settings it needs included in that. E.g. the metron
directory you need is set as a property of that yml, which is then
accessible through the spring config auto-wiring
- @Value(“${metron.directory}”) private String path; in your service
endpoint.
The application.yml file is managed (generated) by Ambari on start of the
rest api service (also managed through ambari now).
All the parameters are built from a combination of user specified settings
in Ambari, or service locations. "Where is storm?” is not something a user
should have to specify; Ambari already knows. However "where would you like
the metron directory?" is an input box somewhere in Ambari. Both are
exposed to the rest api through the application.yml file.
It’s more Ambari pushes the config to the REST layer than pull from Ambari.
This way we’re using Ambari the way it is intended rather than trying to
make it be something it is not (e.g. etcd)
Simon
On 8 May 2017, at 14:00, Otto Fowler <ot...@gmail.com> wrote:
I’m not sure I understand. Could you go on a bit?
Right now, we have env parameters in ambari, that rest should honor. I
don’t understand how moving rest config into ambari get’s my rest service
access to those parameters
On May 8, 2017 at 08:56:21, Simon Elliston Ball (simon@simonellistonball.com)
wrote:
Perhaps a better way of doing this would be to push the configuration of
the REST api (the application.yml) into ambari control and expose
parameters like that. That means the REST api doesn’t have to couple
directly to Ambari, and doesn’t have to reinvent the Ambari paradigm
(Ambari is not a configuration server, it’s a configuration file manager at
heart). If the relevant parameters changed in Ambari, a restart of the REST
service (and consequently re-write of it’s application.yml) would be
forced. The REST API itself would then just use a simple spring config
expression to pick up the value.
Simon
> On 8 May 2017, at 13:52, Otto Fowler <ot...@gmail.com> wrote:
>
> The issue is that we have services that need to access the ambari rest
api from
> inside the rest server.
>
> Technically, if someone changes the defaults for the metron directory
etc, the
> rest won’t pick it up.
>
> I have a PR coming as a follow on to METRON-777 for installing extensions,
> and I write to hdfs, and hard coded to directory, but we allow the
configuration.
>
> Other rest services write to hdfs as well.
>
>
> On May 8, 2017 at 08:45:32, Nick Allen (nick@nickallen.org) wrote:
>
> As opposed to using the Ambari REST API to get this information?
>
> On Mon, May 8, 2017 at 8:06 AM, Otto Fowler <ot...@gmail.com>
wrote:
>
>> I was thinking about have an ambari ‘service’ in the rest api.
>> The initial purpose would be to be able to retrieve ambari configuration
>> variables for the metron service and components.
>>
>> The api would be
>>
>> PUT: login to ambari -> ambari credentials for use with ambari, session
>> variable
>> GET: /component | service if login provided, returns the configuration
>> GET:/component | service | property name if login provided get the
>> property value for property name
>>
>> So, the caller would put the credentials, and after that point the caller
>> or other services can use ambari service configurations,
>> for example to get the configured metron hdfs root directory.
>>
>> This is opposed to configuring credentials in the rest configuration.
>>
>> Thoughts?
>>
Re: [DISCUSS] REST + ambari
Posted by Otto Fowler <ot...@gmail.com>.
So that stuff is read from the environment variable?
On May 8, 2017 at 09:27:48, Otto Fowler (ottobackwards@gmail.com) wrote:
OK, that is great. My works has bridged the move of rest into ambari.
Mentally, I obviously haven’t caught up :)
I’ll look for an example where we are reading application configuration
variables out in the rest service.
Thanks!
On May 8, 2017 at 09:08:46, Simon Elliston Ball (simon@simonellistonball.com)
wrote:
My proposal would be that the REST api use it’s application.yml for all the
parameters and have settings it needs included in that. E.g. the metron
directory you need is set as a property of that yml, which is then
accessible through the spring config auto-wiring
- @Value(“${metron.directory}”) private String path; in your service
endpoint.
The application.yml file is managed (generated) by Ambari on start of the
rest api service (also managed through ambari now).
All the parameters are built from a combination of user specified settings
in Ambari, or service locations. "Where is storm?” is not something a user
should have to specify; Ambari already knows. However "where would you like
the metron directory?" is an input box somewhere in Ambari. Both are
exposed to the rest api through the application.yml file.
It’s more Ambari pushes the config to the REST layer than pull from Ambari.
This way we’re using Ambari the way it is intended rather than trying to
make it be something it is not (e.g. etcd)
Simon
On 8 May 2017, at 14:00, Otto Fowler <ot...@gmail.com> wrote:
I’m not sure I understand. Could you go on a bit?
Right now, we have env parameters in ambari, that rest should honor. I
don’t understand how moving rest config into ambari get’s my rest service
access to those parameters
On May 8, 2017 at 08:56:21, Simon Elliston Ball (simon@simonellistonball.com)
wrote:
Perhaps a better way of doing this would be to push the configuration of
the REST api (the application.yml) into ambari control and expose
parameters like that. That means the REST api doesn’t have to couple
directly to Ambari, and doesn’t have to reinvent the Ambari paradigm
(Ambari is not a configuration server, it’s a configuration file manager at
heart). If the relevant parameters changed in Ambari, a restart of the REST
service (and consequently re-write of it’s application.yml) would be
forced. The REST API itself would then just use a simple spring config
expression to pick up the value.
Simon
> On 8 May 2017, at 13:52, Otto Fowler <ot...@gmail.com> wrote:
>
> The issue is that we have services that need to access the ambari rest
api from
> inside the rest server.
>
> Technically, if someone changes the defaults for the metron directory
etc, the
> rest won’t pick it up.
>
> I have a PR coming as a follow on to METRON-777 for installing extensions,
> and I write to hdfs, and hard coded to directory, but we allow the
configuration.
>
> Other rest services write to hdfs as well.
>
>
> On May 8, 2017 at 08:45:32, Nick Allen (nick@nickallen.org) wrote:
>
> As opposed to using the Ambari REST API to get this information?
>
> On Mon, May 8, 2017 at 8:06 AM, Otto Fowler <ot...@gmail.com>
wrote:
>
>> I was thinking about have an ambari ‘service’ in the rest api.
>> The initial purpose would be to be able to retrieve ambari configuration
>> variables for the metron service and components.
>>
>> The api would be
>>
>> PUT: login to ambari -> ambari credentials for use with ambari, session
>> variable
>> GET: /component | service if login provided, returns the configuration
>> GET:/component | service | property name if login provided get the
>> property value for property name
>>
>> So, the caller would put the credentials, and after that point the caller
>> or other services can use ambari service configurations,
>> for example to get the configured metron hdfs root directory.
>>
>> This is opposed to configuring credentials in the rest configuration.
>>
>> Thoughts?
>>
Re: [DISCUSS] REST + ambari
Posted by Otto Fowler <ot...@gmail.com>.
OK, that is great. My works has bridged the move of rest into ambari.
Mentally, I obviously haven’t caught up :)
I’ll look for an example where we are reading application configuration
variables out in the rest service.
Thanks!
On May 8, 2017 at 09:08:46, Simon Elliston Ball (simon@simonellistonball.com)
wrote:
My proposal would be that the REST api use it’s application.yml for all the
parameters and have settings it needs included in that. E.g. the metron
directory you need is set as a property of that yml, which is then
accessible through the spring config auto-wiring
- @Value(“${metron.directory}”) private String path; in your service
endpoint.
The application.yml file is managed (generated) by Ambari on start of the
rest api service (also managed through ambari now).
All the parameters are built from a combination of user specified settings
in Ambari, or service locations. "Where is storm?” is not something a user
should have to specify; Ambari already knows. However "where would you like
the metron directory?" is an input box somewhere in Ambari. Both are
exposed to the rest api through the application.yml file.
It’s more Ambari pushes the config to the REST layer than pull from Ambari.
This way we’re using Ambari the way it is intended rather than trying to
make it be something it is not (e.g. etcd)
Simon
On 8 May 2017, at 14:00, Otto Fowler <ot...@gmail.com> wrote:
I’m not sure I understand. Could you go on a bit?
Right now, we have env parameters in ambari, that rest should honor. I
don’t understand how moving rest config into ambari get’s my rest service
access to those parameters
On May 8, 2017 at 08:56:21, Simon Elliston Ball (simon@simonellistonball.com)
wrote:
Perhaps a better way of doing this would be to push the configuration of
the REST api (the application.yml) into ambari control and expose
parameters like that. That means the REST api doesn’t have to couple
directly to Ambari, and doesn’t have to reinvent the Ambari paradigm
(Ambari is not a configuration server, it’s a configuration file manager at
heart). If the relevant parameters changed in Ambari, a restart of the REST
service (and consequently re-write of it’s application.yml) would be
forced. The REST API itself would then just use a simple spring config
expression to pick up the value.
Simon
> On 8 May 2017, at 13:52, Otto Fowler <ot...@gmail.com> wrote:
>
> The issue is that we have services that need to access the ambari rest
api from
> inside the rest server.
>
> Technically, if someone changes the defaults for the metron directory
etc, the
> rest won’t pick it up.
>
> I have a PR coming as a follow on to METRON-777 for installing extensions,
> and I write to hdfs, and hard coded to directory, but we allow the
configuration.
>
> Other rest services write to hdfs as well.
>
>
> On May 8, 2017 at 08:45:32, Nick Allen (nick@nickallen.org) wrote:
>
> As opposed to using the Ambari REST API to get this information?
>
> On Mon, May 8, 2017 at 8:06 AM, Otto Fowler <ot...@gmail.com>
wrote:
>
>> I was thinking about have an ambari ‘service’ in the rest api.
>> The initial purpose would be to be able to retrieve ambari configuration
>> variables for the metron service and components.
>>
>> The api would be
>>
>> PUT: login to ambari -> ambari credentials for use with ambari, session
>> variable
>> GET: /component | service if login provided, returns the configuration
>> GET:/component | service | property name if login provided get the
>> property value for property name
>>
>> So, the caller would put the credentials, and after that point the caller
>> or other services can use ambari service configurations,
>> for example to get the configured metron hdfs root directory.
>>
>> This is opposed to configuring credentials in the rest configuration.
>>
>> Thoughts?
>>
Re: [DISCUSS] REST + ambari
Posted by Ryan Merriman <me...@gmail.com>.
It already works the way Simon describes.
On Mon, May 8, 2017 at 8:08 AM, Simon Elliston Ball <
simon@simonellistonball.com> wrote:
> My proposal would be that the REST api use it’s application.yml for all
> the parameters and have settings it needs included in that. E.g. the metron
> directory you need is set as a property of that yml, which is then
> accessible through the spring config auto-wiring -
> @Value(“${metron.directory}”) private String path; in your service endpoint.
>
> The application.yml file is managed (generated) by Ambari on start of the
> rest api service (also managed through ambari now).
>
> All the parameters are built from a combination of user specified settings
> in Ambari, or service locations. "Where is storm?” is not something a user
> should have to specify; Ambari already knows. However "where would you like
> the metron directory?" is an input box somewhere in Ambari. Both are
> exposed to the rest api through the application.yml file.
>
> It’s more Ambari pushes the config to the REST layer than pull from
> Ambari. This way we’re using Ambari the way it is intended rather than
> trying to make it be something it is not (e.g. etcd)
>
> Simon
>
>
>
> > On 8 May 2017, at 14:00, Otto Fowler <ot...@gmail.com> wrote:
> >
> > I’m not sure I understand. Could you go on a bit?
> >
> > Right now, we have env parameters in ambari, that rest should honor. I
> don’t understand how moving rest config into ambari get’s my rest service
> access to those parameters
> >
> >
> >
> > On May 8, 2017 at 08:56:21, Simon Elliston Ball (
> simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
> >
> >> Perhaps a better way of doing this would be to push the configuration
> of the REST api (the application.yml) into ambari control and expose
> parameters like that. That means the REST api doesn’t have to couple
> directly to Ambari, and doesn’t have to reinvent the Ambari paradigm
> (Ambari is not a configuration server, it’s a configuration file manager at
> heart). If the relevant parameters changed in Ambari, a restart of the REST
> service (and consequently re-write of it’s application.yml) would be
> forced. The REST API itself would then just use a simple spring config
> expression to pick up the value.
> >>
> >> Simon
> >>
> >> > On 8 May 2017, at 13:52, Otto Fowler <ot...@gmail.com> wrote:
> >> >
> >> > The issue is that we have services that need to access the ambari
> rest api from
> >> > inside the rest server.
> >> >
> >> > Technically, if someone changes the defaults for the metron directory
> etc, the
> >> > rest won’t pick it up.
> >> >
> >> > I have a PR coming as a follow on to METRON-777 for installing
> extensions,
> >> > and I write to hdfs, and hard coded to directory, but we allow the
> configuration.
> >> >
> >> > Other rest services write to hdfs as well.
> >> >
> >> >
> >> > On May 8, 2017 at 08:45:32, Nick Allen (nick@nickallen.org) wrote:
> >> >
> >> > As opposed to using the Ambari REST API to get this information?
> >> >
> >> > On Mon, May 8, 2017 at 8:06 AM, Otto Fowler <ot...@gmail.com>
> wrote:
> >> >
> >> >> I was thinking about have an ambari ‘service’ in the rest api.
> >> >> The initial purpose would be to be able to retrieve ambari
> configuration
> >> >> variables for the metron service and components.
> >> >>
> >> >> The api would be
> >> >>
> >> >> PUT: login to ambari -> ambari credentials for use with ambari,
> session
> >> >> variable
> >> >> GET: /component | service if login provided, returns the
> configuration
> >> >> GET:/component | service | property name if login provided get the
> >> >> property value for property name
> >> >>
> >> >> So, the caller would put the credentials, and after that point the
> caller
> >> >> or other services can use ambari service configurations,
> >> >> for example to get the configured metron hdfs root directory.
> >> >>
> >> >> This is opposed to configuring credentials in the rest configuration.
> >> >>
> >> >> Thoughts?
> >> >>
>
>
Re: [DISCUSS] REST + ambari
Posted by Simon Elliston Ball <si...@simonellistonball.com>.
My proposal would be that the REST api use it’s application.yml for all the parameters and have settings it needs included in that. E.g. the metron directory you need is set as a property of that yml, which is then accessible through the spring config auto-wiring - @Value(“${metron.directory}”) private String path; in your service endpoint.
The application.yml file is managed (generated) by Ambari on start of the rest api service (also managed through ambari now).
All the parameters are built from a combination of user specified settings in Ambari, or service locations. "Where is storm?” is not something a user should have to specify; Ambari already knows. However "where would you like the metron directory?" is an input box somewhere in Ambari. Both are exposed to the rest api through the application.yml file.
It’s more Ambari pushes the config to the REST layer than pull from Ambari. This way we’re using Ambari the way it is intended rather than trying to make it be something it is not (e.g. etcd)
Simon
> On 8 May 2017, at 14:00, Otto Fowler <ot...@gmail.com> wrote:
>
> I’m not sure I understand. Could you go on a bit?
>
> Right now, we have env parameters in ambari, that rest should honor. I don’t understand how moving rest config into ambari get’s my rest service access to those parameters
>
>
>
> On May 8, 2017 at 08:56:21, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>
>> Perhaps a better way of doing this would be to push the configuration of the REST api (the application.yml) into ambari control and expose parameters like that. That means the REST api doesn’t have to couple directly to Ambari, and doesn’t have to reinvent the Ambari paradigm (Ambari is not a configuration server, it’s a configuration file manager at heart). If the relevant parameters changed in Ambari, a restart of the REST service (and consequently re-write of it’s application.yml) would be forced. The REST API itself would then just use a simple spring config expression to pick up the value.
>>
>> Simon
>>
>> > On 8 May 2017, at 13:52, Otto Fowler <ot...@gmail.com> wrote:
>> >
>> > The issue is that we have services that need to access the ambari rest api from
>> > inside the rest server.
>> >
>> > Technically, if someone changes the defaults for the metron directory etc, the
>> > rest won’t pick it up.
>> >
>> > I have a PR coming as a follow on to METRON-777 for installing extensions,
>> > and I write to hdfs, and hard coded to directory, but we allow the configuration.
>> >
>> > Other rest services write to hdfs as well.
>> >
>> >
>> > On May 8, 2017 at 08:45:32, Nick Allen (nick@nickallen.org) wrote:
>> >
>> > As opposed to using the Ambari REST API to get this information?
>> >
>> > On Mon, May 8, 2017 at 8:06 AM, Otto Fowler <ot...@gmail.com> wrote:
>> >
>> >> I was thinking about have an ambari ‘service’ in the rest api.
>> >> The initial purpose would be to be able to retrieve ambari configuration
>> >> variables for the metron service and components.
>> >>
>> >> The api would be
>> >>
>> >> PUT: login to ambari -> ambari credentials for use with ambari, session
>> >> variable
>> >> GET: /component | service if login provided, returns the configuration
>> >> GET:/component | service | property name if login provided get the
>> >> property value for property name
>> >>
>> >> So, the caller would put the credentials, and after that point the caller
>> >> or other services can use ambari service configurations,
>> >> for example to get the configured metron hdfs root directory.
>> >>
>> >> This is opposed to configuring credentials in the rest configuration.
>> >>
>> >> Thoughts?
>> >>
Re: [DISCUSS] REST + ambari
Posted by Otto Fowler <ot...@gmail.com>.
I’m not sure I understand. Could you go on a bit?
Right now, we have env parameters in ambari, that rest should honor. I don’t understand how moving rest config into ambari get’s my rest service access to those parameters
On May 8, 2017 at 08:56:21, Simon Elliston Ball (simon@simonellistonball.com) wrote:
Perhaps a better way of doing this would be to push the configuration of the REST api (the application.yml) into ambari control and expose parameters like that. That means the REST api doesn’t have to couple directly to Ambari, and doesn’t have to reinvent the Ambari paradigm (Ambari is not a configuration server, it’s a configuration file manager at heart). If the relevant parameters changed in Ambari, a restart of the REST service (and consequently re-write of it’s application.yml) would be forced. The REST API itself would then just use a simple spring config expression to pick up the value.
Simon
> On 8 May 2017, at 13:52, Otto Fowler <ot...@gmail.com> wrote:
>
> The issue is that we have services that need to access the ambari rest api from
> inside the rest server.
>
> Technically, if someone changes the defaults for the metron directory etc, the
> rest won’t pick it up.
>
> I have a PR coming as a follow on to METRON-777 for installing extensions,
> and I write to hdfs, and hard coded to directory, but we allow the configuration.
>
> Other rest services write to hdfs as well.
>
>
> On May 8, 2017 at 08:45:32, Nick Allen (nick@nickallen.org) wrote:
>
> As opposed to using the Ambari REST API to get this information?
>
> On Mon, May 8, 2017 at 8:06 AM, Otto Fowler <ot...@gmail.com> wrote:
>
>> I was thinking about have an ambari ‘service’ in the rest api.
>> The initial purpose would be to be able to retrieve ambari configuration
>> variables for the metron service and components.
>>
>> The api would be
>>
>> PUT: login to ambari -> ambari credentials for use with ambari, session
>> variable
>> GET: /component | service if login provided, returns the configuration
>> GET:/component | service | property name if login provided get the
>> property value for property name
>>
>> So, the caller would put the credentials, and after that point the caller
>> or other services can use ambari service configurations,
>> for example to get the configured metron hdfs root directory.
>>
>> This is opposed to configuring credentials in the rest configuration.
>>
>> Thoughts?
>>
Re: [DISCUSS] REST + ambari
Posted by Simon Elliston Ball <si...@simonellistonball.com>.
Perhaps a better way of doing this would be to push the configuration of the REST api (the application.yml) into ambari control and expose parameters like that. That means the REST api doesn’t have to couple directly to Ambari, and doesn’t have to reinvent the Ambari paradigm (Ambari is not a configuration server, it’s a configuration file manager at heart). If the relevant parameters changed in Ambari, a restart of the REST service (and consequently re-write of it’s application.yml) would be forced. The REST API itself would then just use a simple spring config expression to pick up the value.
Simon
> On 8 May 2017, at 13:52, Otto Fowler <ot...@gmail.com> wrote:
>
> The issue is that we have services that need to access the ambari rest api from
> inside the rest server.
>
> Technically, if someone changes the defaults for the metron directory etc, the
> rest won’t pick it up.
>
> I have a PR coming as a follow on to METRON-777 for installing extensions,
> and I write to hdfs, and hard coded to directory, but we allow the configuration.
>
> Other rest services write to hdfs as well.
>
>
> On May 8, 2017 at 08:45:32, Nick Allen (nick@nickallen.org) wrote:
>
> As opposed to using the Ambari REST API to get this information?
>
> On Mon, May 8, 2017 at 8:06 AM, Otto Fowler <ot...@gmail.com> wrote:
>
>> I was thinking about have an ambari ‘service’ in the rest api.
>> The initial purpose would be to be able to retrieve ambari configuration
>> variables for the metron service and components.
>>
>> The api would be
>>
>> PUT: login to ambari -> ambari credentials for use with ambari, session
>> variable
>> GET: /component | service if login provided, returns the configuration
>> GET:/component | service | property name if login provided get the
>> property value for property name
>>
>> So, the caller would put the credentials, and after that point the caller
>> or other services can use ambari service configurations,
>> for example to get the configured metron hdfs root directory.
>>
>> This is opposed to configuring credentials in the rest configuration.
>>
>> Thoughts?
>>
Re: [DISCUSS] REST + ambari
Posted by Otto Fowler <ot...@gmail.com>.
The issue is that we have services that need to access the ambari rest api from
inside the rest server.
Technically, if someone changes the defaults for the metron directory etc, the
rest won’t pick it up.
I have a PR coming as a follow on to METRON-777 for installing extensions,
and I write to hdfs, and hard coded to directory, but we allow the configuration.
Other rest services write to hdfs as well.
On May 8, 2017 at 08:45:32, Nick Allen (nick@nickallen.org) wrote:
As opposed to using the Ambari REST API to get this information?
On Mon, May 8, 2017 at 8:06 AM, Otto Fowler <ot...@gmail.com> wrote:
> I was thinking about have an ambari ‘service’ in the rest api.
> The initial purpose would be to be able to retrieve ambari configuration
> variables for the metron service and components.
>
> The api would be
>
> PUT: login to ambari -> ambari credentials for use with ambari, session
> variable
> GET: /component | service if login provided, returns the configuration
> GET:/component | service | property name if login provided get the
> property value for property name
>
> So, the caller would put the credentials, and after that point the caller
> or other services can use ambari service configurations,
> for example to get the configured metron hdfs root directory.
>
> This is opposed to configuring credentials in the rest configuration.
>
> Thoughts?
>
Re: [DISCUSS] REST + ambari
Posted by Nick Allen <ni...@nickallen.org>.
As opposed to using the Ambari REST API to get this information?
On Mon, May 8, 2017 at 8:06 AM, Otto Fowler <ot...@gmail.com> wrote:
> I was thinking about have an ambari ‘service’ in the rest api.
> The initial purpose would be to be able to retrieve ambari configuration
> variables for the metron service and components.
>
> The api would be
>
> PUT: login to ambari -> ambari credentials for use with ambari, session
> variable
> GET: /component | service if login provided, returns the configuration
> GET:/component | service | property name if login provided get the
> property value for property name
>
> So, the caller would put the credentials, and after that point the caller
> or other services can use ambari service configurations,
> for example to get the configured metron hdfs root directory.
>
> This is opposed to configuring credentials in the rest configuration.
>
> Thoughts?
>