You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Roman Shaposhnik <rv...@apache.org> on 2012/09/06 01:51:12 UTC
running init.d scripts by hand vs. service(8)
Hi!
Here's a thorny issue that I'm looking at right now which I'd
appreciate a broader perspective on: it seems that currently
there's a subtle (but a very dangerous) difference between
running any of our init.d scripts by hand -- e.g.
# /etc/init.d/hadoop-hdfs-datanode start
vs. running them via a service(8) command -- e.g.
# service hadoop-hdfs-datanode start
It just so happens that service is guaranteed to clean
the environment up, but running the scripts directly
would run them in whatever environment root might
have currently.
IOW, if for any reason root would set a variable up
that corresponds to a var from a particular /etc/default/<name>
sourced by the init.d scripts -- that would affect the script
execution.
What do you all think -- is this a bug or a feature? Should
we, somehow, make sure that a direct execution would
scrub the environment the same way that a service(8) one
does?
Thanks,
Roman.
P.S. Now, it might seem to be a trivial matter of cleanliness but
I've got bitten by it today and spent quite a bit of time debugging
the "incorrect" behavior only to realize that it was my environment
to blame.
Re: running init.d scripts by hand vs. service(8)
Posted by Roman Shaposhnik <rv...@apache.org>.
On Tue, Sep 11, 2012 at 11:35 PM, Bruno Mahé <bm...@apache.org> wrote:
> su?
> Although I haven't had time to play with this issue.
It is not a straight su, it is a little bit more complicated since you
actually have to setup those env. vars that are part of /etc/default/foo
Thus it ends being something really ugly like:
env -i LANG="$LANG" PATH="$PATH" TERM="$TERM" bash -c \
"[ -e /etc/default/foo ] && . /etc/default/foo
;\
exec $AGENT_SCRIPT $AGENT_ARGS > $AGENT_OUT 2>&1" &
and even then it is not a mechanical change. Since all of the init.d
scripts need to be audited for export FOO statements.
Actually, the more I think about it the more it feels like we might
benefit from templatizing all of our init.d related code and then
fixing this. I would like templatizing to be one of the features
for the Bigtop 0.5.0 releases, but perhaps for now we can just
keep in mind to always invoke service.
Thanks,
Roman.
Re: running init.d scripts by hand vs. service(8)
Posted by Bruno Mahé <bm...@apache.org>.
On 09/07/2012 05:51 PM, Roman Shaposhnik wrote:
> On Fri, Sep 7, 2012 at 11:43 AM, Konstantin Boudnik <co...@apache.org> wrote:
>>> These scripts could be called in various ways and I am a little bit
>>> worried about closing some doors.
>>> What about having a consistent behavior of always scrubbing the
>>> environment (so the script will always work by default), except when
>>> some debugging variable/parameter is passed to it.
>>> Hopefully people spend more time using these scripts than debugging them :)
>
> Part of the problem is that if we decide to scrub by default then I can't
> come up with any code that isn't extremely ugly in order to do that
> consistently. Ideas are appreciated.
>
> Personally, though, I feel like the lowest hanging fruit is to NOT scrub,
> but warn based on presence of HOME variable. Will this be acceptable?
>
> Thanks,
> Roman.
>
su?
Although I haven't had time to play with this issue.
Thanks,
Bruno
Re: running init.d scripts by hand vs. service(8)
Posted by Roman Shaposhnik <rv...@apache.org>.
On Fri, Sep 7, 2012 at 11:43 AM, Konstantin Boudnik <co...@apache.org> wrote:
>> These scripts could be called in various ways and I am a little bit
>> worried about closing some doors.
>> What about having a consistent behavior of always scrubbing the
>> environment (so the script will always work by default), except when
>> some debugging variable/parameter is passed to it.
>> Hopefully people spend more time using these scripts than debugging them :)
Part of the problem is that if we decide to scrub by default then I can't
come up with any code that isn't extremely ugly in order to do that
consistently. Ideas are appreciated.
Personally, though, I feel like the lowest hanging fruit is to NOT scrub,
but warn based on presence of HOME variable. Will this be acceptable?
Thanks,
Roman.
Re: running init.d scripts by hand vs. service(8)
Posted by Konstantin Boudnik <co...@apache.org>.
On Fri, Sep 07, 2012 at 01:49AM, Bruno MahИ wrote:
> On 09/06/2012 10:31 AM, Sean Mackrory wrote:
> >+1 to Alejandro's idea. I like the having the ability to manually override
> >environment variables and invoke scripts as a developer. I'd be annoyed if
> >I used a service and it refused to start up without service or if it
> >ignored my environment in an unnecessarily subtle way. That being said,
> >since one of the benefits of using Bigtop is more out-of-the-box
> >reliability and simplicity, I also like the idea of strongly encouraging
> >users to use 1 supported method of service invocation.
> >
> >On Thu, Sep 6, 2012 at 9:15 AM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
> >
> >>The verbosity would be there only when running the scripts outside of
> >>service, to avoid YOU tripping again :)
> >>
> >>On Thu, Sep 6, 2012 at 9:11 AM, Roman Shaposhnik <rv...@apache.org> wrote:
> >>>On Thu, Sep 6, 2012 at 8:14 AM, Alejandro Abdelnur <tu...@cloudera.com>
> >>wrote:
> >>>>I think is a desirable feature to be able to modify ENV then run start
> >>>>the daemon directly. It may be handy on troubleshooting.
> >>>
> >>>As I said -- it is entirely possible we can all agree that this is a
> >>feature.
> >>>
> >>>>I'd propose a modification to Cos' proposal, print a NICE BIG WARN
> >>>>when starting the daemons without service(8) stating 'current ENV
> >>>>settings may affect the daemon (Roman remember that)'
> >>>
> >>>I don't think I have a guaranteed way of detecting one vs. the other.
> >>>/usr/bin/service happens to be a shell script that at the end of the
> >>>day simply scrubs the environment and execs the actual init.d script:
> >>> exec env -i LANG="$LANG" PATH="$PATH" TERM="$TERM"
> >>>"$SERVICEDIR/$SERVICE" ${ACTION} ${OPTIONS}
> >>>
> >>>Perhaps we can try to detect that we're NOT exec'ed by service via
> >>>absence of things like HOME, etc.
> >>>
> >>>The question then becomes whether this is too much trouble and whether
> >>>it'll make our scripts too verbose.
> >>>
> >>>Thanks,
> >>>Roman.
> >>
> >>
> >>
> >>--
> >>Alejandro
> >>
> >
>
> These scripts could be called in various ways and I am a little bit
> worried about closing some doors.
> What about having a consistent behavior of always scrubbing the
> environment (so the script will always work by default), except when
> some debugging variable/parameter is passed to it.
> Hopefully people spend more time using these scripts than debugging them :)
Sounds good.
Cos
Re: running init.d scripts by hand vs. service(8)
Posted by Bruno Mahé <bm...@apache.org>.
On 09/06/2012 10:31 AM, Sean Mackrory wrote:
> +1 to Alejandro's idea. I like the having the ability to manually override
> environment variables and invoke scripts as a developer. I'd be annoyed if
> I used a service and it refused to start up without service or if it
> ignored my environment in an unnecessarily subtle way. That being said,
> since one of the benefits of using Bigtop is more out-of-the-box
> reliability and simplicity, I also like the idea of strongly encouraging
> users to use 1 supported method of service invocation.
>
> On Thu, Sep 6, 2012 at 9:15 AM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
>
>> The verbosity would be there only when running the scripts outside of
>> service, to avoid YOU tripping again :)
>>
>> On Thu, Sep 6, 2012 at 9:11 AM, Roman Shaposhnik <rv...@apache.org> wrote:
>>> On Thu, Sep 6, 2012 at 8:14 AM, Alejandro Abdelnur <tu...@cloudera.com>
>> wrote:
>>>> I think is a desirable feature to be able to modify ENV then run start
>>>> the daemon directly. It may be handy on troubleshooting.
>>>
>>> As I said -- it is entirely possible we can all agree that this is a
>> feature.
>>>
>>>> I'd propose a modification to Cos' proposal, print a NICE BIG WARN
>>>> when starting the daemons without service(8) stating 'current ENV
>>>> settings may affect the daemon (Roman remember that)'
>>>
>>> I don't think I have a guaranteed way of detecting one vs. the other.
>>> /usr/bin/service happens to be a shell script that at the end of the
>>> day simply scrubs the environment and execs the actual init.d script:
>>> exec env -i LANG="$LANG" PATH="$PATH" TERM="$TERM"
>>> "$SERVICEDIR/$SERVICE" ${ACTION} ${OPTIONS}
>>>
>>> Perhaps we can try to detect that we're NOT exec'ed by service via
>>> absence of things like HOME, etc.
>>>
>>> The question then becomes whether this is too much trouble and whether
>>> it'll make our scripts too verbose.
>>>
>>> Thanks,
>>> Roman.
>>
>>
>>
>> --
>> Alejandro
>>
>
These scripts could be called in various ways and I am a little bit
worried about closing some doors.
What about having a consistent behavior of always scrubbing the
environment (so the script will always work by default), except when
some debugging variable/parameter is passed to it.
Hopefully people spend more time using these scripts than debugging them :)
Thanks,
Bruno
Re: running init.d scripts by hand vs. service(8)
Posted by Sean Mackrory <ma...@gmail.com>.
+1 to Alejandro's idea. I like the having the ability to manually override
environment variables and invoke scripts as a developer. I'd be annoyed if
I used a service and it refused to start up without service or if it
ignored my environment in an unnecessarily subtle way. That being said,
since one of the benefits of using Bigtop is more out-of-the-box
reliability and simplicity, I also like the idea of strongly encouraging
users to use 1 supported method of service invocation.
On Thu, Sep 6, 2012 at 9:15 AM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
> The verbosity would be there only when running the scripts outside of
> service, to avoid YOU tripping again :)
>
> On Thu, Sep 6, 2012 at 9:11 AM, Roman Shaposhnik <rv...@apache.org> wrote:
> > On Thu, Sep 6, 2012 at 8:14 AM, Alejandro Abdelnur <tu...@cloudera.com>
> wrote:
> >> I think is a desirable feature to be able to modify ENV then run start
> >> the daemon directly. It may be handy on troubleshooting.
> >
> > As I said -- it is entirely possible we can all agree that this is a
> feature.
> >
> >> I'd propose a modification to Cos' proposal, print a NICE BIG WARN
> >> when starting the daemons without service(8) stating 'current ENV
> >> settings may affect the daemon (Roman remember that)'
> >
> > I don't think I have a guaranteed way of detecting one vs. the other.
> > /usr/bin/service happens to be a shell script that at the end of the
> > day simply scrubs the environment and execs the actual init.d script:
> > exec env -i LANG="$LANG" PATH="$PATH" TERM="$TERM"
> > "$SERVICEDIR/$SERVICE" ${ACTION} ${OPTIONS}
> >
> > Perhaps we can try to detect that we're NOT exec'ed by service via
> > absence of things like HOME, etc.
> >
> > The question then becomes whether this is too much trouble and whether
> > it'll make our scripts too verbose.
> >
> > Thanks,
> > Roman.
>
>
>
> --
> Alejandro
>
Re: running init.d scripts by hand vs. service(8)
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
The verbosity would be there only when running the scripts outside of
service, to avoid YOU tripping again :)
On Thu, Sep 6, 2012 at 9:11 AM, Roman Shaposhnik <rv...@apache.org> wrote:
> On Thu, Sep 6, 2012 at 8:14 AM, Alejandro Abdelnur <tu...@cloudera.com> wrote:
>> I think is a desirable feature to be able to modify ENV then run start
>> the daemon directly. It may be handy on troubleshooting.
>
> As I said -- it is entirely possible we can all agree that this is a feature.
>
>> I'd propose a modification to Cos' proposal, print a NICE BIG WARN
>> when starting the daemons without service(8) stating 'current ENV
>> settings may affect the daemon (Roman remember that)'
>
> I don't think I have a guaranteed way of detecting one vs. the other.
> /usr/bin/service happens to be a shell script that at the end of the
> day simply scrubs the environment and execs the actual init.d script:
> exec env -i LANG="$LANG" PATH="$PATH" TERM="$TERM"
> "$SERVICEDIR/$SERVICE" ${ACTION} ${OPTIONS}
>
> Perhaps we can try to detect that we're NOT exec'ed by service via
> absence of things like HOME, etc.
>
> The question then becomes whether this is too much trouble and whether
> it'll make our scripts too verbose.
>
> Thanks,
> Roman.
--
Alejandro
Re: running init.d scripts by hand vs. service(8)
Posted by Roman Shaposhnik <rv...@apache.org>.
On Thu, Sep 6, 2012 at 8:14 AM, Alejandro Abdelnur <tu...@cloudera.com> wrote:
> I think is a desirable feature to be able to modify ENV then run start
> the daemon directly. It may be handy on troubleshooting.
As I said -- it is entirely possible we can all agree that this is a feature.
> I'd propose a modification to Cos' proposal, print a NICE BIG WARN
> when starting the daemons without service(8) stating 'current ENV
> settings may affect the daemon (Roman remember that)'
I don't think I have a guaranteed way of detecting one vs. the other.
/usr/bin/service happens to be a shell script that at the end of the
day simply scrubs the environment and execs the actual init.d script:
exec env -i LANG="$LANG" PATH="$PATH" TERM="$TERM"
"$SERVICEDIR/$SERVICE" ${ACTION} ${OPTIONS}
Perhaps we can try to detect that we're NOT exec'ed by service via
absence of things like HOME, etc.
The question then becomes whether this is too much trouble and whether
it'll make our scripts too verbose.
Thanks,
Roman.
Re: running init.d scripts by hand vs. service(8)
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
I think is a desirable feature to be able to modify ENV then run start
the daemon directly. It may be handy on troubleshooting.
I'd propose a modification to Cos' proposal, print a NICE BIG WARN
when starting the daemons without service(8) stating 'current ENV
settings may affect the daemon (Roman remember that)'
Thx
On Wed, Sep 5, 2012 at 5:41 PM, Konstantin Boudnik <co...@apache.org> wrote:
> I'd say consistency is the way to go.
>
> The easiest perhaps is to check if the invocation doesn't caused by the
> service(8) command then simply reject to proceed ;)
>
> Cos
>
> On Wed, Sep 05, 2012 at 04:51PM, Roman Shaposhnik wrote:
>> Hi!
>>
>> Here's a thorny issue that I'm looking at right now which I'd
>> appreciate a broader perspective on: it seems that currently
>> there's a subtle (but a very dangerous) difference between
>> running any of our init.d scripts by hand -- e.g.
>> # /etc/init.d/hadoop-hdfs-datanode start
>> vs. running them via a service(8) command -- e.g.
>> # service hadoop-hdfs-datanode start
>>
>> It just so happens that service is guaranteed to clean
>> the environment up, but running the scripts directly
>> would run them in whatever environment root might
>> have currently.
>>
>> IOW, if for any reason root would set a variable up
>> that corresponds to a var from a particular /etc/default/<name>
>> sourced by the init.d scripts -- that would affect the script
>> execution.
>>
>> What do you all think -- is this a bug or a feature? Should
>> we, somehow, make sure that a direct execution would
>> scrub the environment the same way that a service(8) one
>> does?
>>
>> Thanks,
>> Roman.
>>
>> P.S. Now, it might seem to be a trivial matter of cleanliness but
>> I've got bitten by it today and spent quite a bit of time debugging
>> the "incorrect" behavior only to realize that it was my environment
>> to blame.
--
Alejandro
Re: running init.d scripts by hand vs. service(8)
Posted by Konstantin Boudnik <co...@apache.org>.
I'd say consistency is the way to go.
The easiest perhaps is to check if the invocation doesn't caused by the
service(8) command then simply reject to proceed ;)
Cos
On Wed, Sep 05, 2012 at 04:51PM, Roman Shaposhnik wrote:
> Hi!
>
> Here's a thorny issue that I'm looking at right now which I'd
> appreciate a broader perspective on: it seems that currently
> there's a subtle (but a very dangerous) difference between
> running any of our init.d scripts by hand -- e.g.
> # /etc/init.d/hadoop-hdfs-datanode start
> vs. running them via a service(8) command -- e.g.
> # service hadoop-hdfs-datanode start
>
> It just so happens that service is guaranteed to clean
> the environment up, but running the scripts directly
> would run them in whatever environment root might
> have currently.
>
> IOW, if for any reason root would set a variable up
> that corresponds to a var from a particular /etc/default/<name>
> sourced by the init.d scripts -- that would affect the script
> execution.
>
> What do you all think -- is this a bug or a feature? Should
> we, somehow, make sure that a direct execution would
> scrub the environment the same way that a service(8) one
> does?
>
> Thanks,
> Roman.
>
> P.S. Now, it might seem to be a trivial matter of cleanliness but
> I've got bitten by it today and spent quite a bit of time debugging
> the "incorrect" behavior only to realize that it was my environment
> to blame.
Re: running init.d scripts by hand vs. service(8)
Posted by Jos Backus <jo...@catnook.com>.
On Wed, Sep 5, 2012 at 4:51 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> Hi!
>
> Here's a thorny issue that I'm looking at right now which I'd
> appreciate a broader perspective on: it seems that currently
> there's a subtle (but a very dangerous) difference between
> running any of our init.d scripts by hand -- e.g.
> # /etc/init.d/hadoop-hdfs-datanode start
> vs. running them via a service(8) command -- e.g.
> # service hadoop-hdfs-datanode start
>
> It just so happens that service is guaranteed to clean
> the environment up, but running the scripts directly
> would run them in whatever environment root might
> have currently.
>
> IOW, if for any reason root would set a variable up
> that corresponds to a var from a particular /etc/default/<name>
> sourced by the init.d scripts -- that would affect the script
> execution.
>
> What do you all think -- is this a bug or a feature? Should
> we, somehow, make sure that a direct execution would
> scrub the environment the same way that a service(8) one
> does?
>
> Thanks,
> Roman.
>
> P.S. Now, it might seem to be a trivial matter of cleanliness but
> I've got bitten by it today and spent quite a bit of time debugging
> the "incorrect" behavior only to realize that it was my environment
> to blame.
>
Thanks for making the case for process supervision, Roman! It mystifies me
why people are still using init scripts. Must be tradition/inertia or
resistance to change or something.
Fwiw, I would vote for consistency in behavior between the two.
I will upload my daemontools-encore service scripts to Jira soon.
Cheers,
Jos
--
Jos Backus
jos at catnook.com