You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by John Omernik <jo...@omernik.com> on 2016/11/01 14:45:51 UTC
DRILL_LOG_PREFIX in drill-config
Hey all,
I am getting my head around the new Drill config files in 1.8, and I have
been setting DRILL_LOG_PREFIX in previous versions. Based on what I am
seeing in drill-config.sh though, it looks like I no longer can set this as
instead of using a "If set use what's set otherwise default to" methodology
like other variables, drill-config.sh just exports a static value... is
this correct?
My goal is to have logs all in a single directory but have prefixes based
on the host names... "centralized" logging if you will :) Is this possible
in 1.8 or do I need a JIRA?
export DRILL_LOG_PREFIX="$DRILL_LOG_DIR/drillbit"
John
Re: DRILL_LOG_PREFIX in drill-config
Posted by Paul Rogers <pr...@maprtech.com>.
Hi John,
Yes, that’s what the line in drill-config.sh does:
export DRILL_LOG_NAME=${DRILL_LOG_NAME:-"drillbit”}
If a log name has been provided, it is used. If not, then “drillbit” is the default.
I guessed you were a bit busy, so I went ahead and filed a JIRA for this: DRILL-4986.
I also submitted a pull request to make this change. The unit tests show that you can set the DRILL_LOG_NAME variable in either the environment or drill-env.sh and have it “do the right thing.”
You mentioned you are on 1.8. If you are not yet ready to upgrade 1.9, you can grab the new scripts from 1.9 to replace those in your 1.8 build — things should work fine.
- Paul
> On Nov 2, 2016, at 5:19 AM, John Omernik <jo...@omernik.com> wrote:
>
> That works, sorry I was not able to suggest changes yesterday, I was heads
> down on a bunch of stuff, this would work for me. However, won't we have to
> change drill-config.sh so it doesn't hard code DRILL_LOG_PREFIX?
>
> John
>
> On Tue, Nov 1, 2016 at 1:14 PM, Paul Rogers <pr...@maprtech.com> wrote:
>
>> Hi All,
>>
>> Since we are so close to release, I’ll see if I can get a fix PR submitted
>> today based on John’s suggestions.
>>
>> Just to be super careful, I suggest we introduce a new variable that holds
>> the log name base file. In drill-env.sh:
>>
>> # Define a custom base name for Drill log files.
>>
>> # DRILL_LOG_NAME=“drillbit”
>>
>> In drill-config.sh:
>>
>> export DRILL_LOG_NAME=${DRILL_LOG_NAME:-"drillbit”}
>> export DRILL_LOG_PREFIX="$DRILL_LOG_DIR/$DRILL_LOG_NAME”
>>
>> All other code can stay the same:
>>
>> export DRILLBIT_LOG_PATH="${DRILL_LOG_PREFIX}.log"
>> logout="${DRILL_LOG_PREFIX}.out"
>> logqueries="${DRILL_LOG_PREFIX}_queries.json"
>> loggc="${DRILL_LOG_PREFIX}.gc”
>> etc.
>>
>> If folks have made customizations that count on this pattern, those
>> changes should work unchanged after the revision.
>>
>> Thoughts?
>>
>> - Paul
>>
>>
>>> On Nov 1, 2016, at 7:58 AM, John Omernik <jo...@omernik.com> wrote:
>>>
>>> export DRILL_LOG_PREFIX="$DRILL_LOG_DIR/drillbit"
>>>
>>> So looking at this, I think that DRILL_LOG_PREFIX is a misnomer, it
>> should
>>> be DRILL_LOG_PATH = Path to a directory, DRILL_LOG_PREFIX = A prefix to
>>> prepend to drill log files, thus, in reality, DRILL_LOG_PREFIX should not
>>> be set to $DRILL_LOG_DIR/drillbit, instead the default should be "
>>>
>>>
>>> export DRILL_LOG_PREFIX=${DRILL_LOG_PREFIX:-"drillbit"}
>>>
>>> and then the next line that makes the drill log path should be:
>>>
>>> export DRILLBIT_LOG_PATH="${DRILL_LOG_DIR}/${DRILL_LOG_PREFIX}.log"
>>>
>>> That way the Prefix is just a file prefix, and then in the drill-env.sh I
>>> could set
>>>
>>>
>>> export DRILL_LOG_PREFIX="drillbit-$HOSTNAME"
>>>
>>>
>>> and have all my logs in one folder, but not be overwritten by easy
>> other....
>>>
>>>
>>>
>>>
>>> On Tue, Nov 1, 2016 at 9:45 AM, John Omernik <jo...@omernik.com> wrote:
>>>
>>>> Hey all,
>>>>
>>>>
>>>> I am getting my head around the new Drill config files in 1.8, and I
>> have
>>>> been setting DRILL_LOG_PREFIX in previous versions. Based on what I am
>>>> seeing in drill-config.sh though, it looks like I no longer can set
>> this as
>>>> instead of using a "If set use what's set otherwise default to"
>> methodology
>>>> like other variables, drill-config.sh just exports a static value... is
>>>> this correct?
>>>>
>>>>
>>>> My goal is to have logs all in a single directory but have prefixes
>> based
>>>> on the host names... "centralized" logging if you will :) Is this
>> possible
>>>> in 1.8 or do I need a JIRA?
>>>>
>>>>
>>>>
>>>>
>>>> export DRILL_LOG_PREFIX="$DRILL_LOG_DIR/drillbit"
>>>>
>>>>
>>>>
>>>> John
>>>>
>>
>>
Re: DRILL_LOG_PREFIX in drill-config
Posted by John Omernik <jo...@omernik.com>.
That works, sorry I was not able to suggest changes yesterday, I was heads
down on a bunch of stuff, this would work for me. However, won't we have to
change drill-config.sh so it doesn't hard code DRILL_LOG_PREFIX?
John
On Tue, Nov 1, 2016 at 1:14 PM, Paul Rogers <pr...@maprtech.com> wrote:
> Hi All,
>
> Since we are so close to release, I’ll see if I can get a fix PR submitted
> today based on John’s suggestions.
>
> Just to be super careful, I suggest we introduce a new variable that holds
> the log name base file. In drill-env.sh:
>
> # Define a custom base name for Drill log files.
>
> # DRILL_LOG_NAME=“drillbit”
>
> In drill-config.sh:
>
> export DRILL_LOG_NAME=${DRILL_LOG_NAME:-"drillbit”}
> export DRILL_LOG_PREFIX="$DRILL_LOG_DIR/$DRILL_LOG_NAME”
>
> All other code can stay the same:
>
> export DRILLBIT_LOG_PATH="${DRILL_LOG_PREFIX}.log"
> logout="${DRILL_LOG_PREFIX}.out"
> logqueries="${DRILL_LOG_PREFIX}_queries.json"
> loggc="${DRILL_LOG_PREFIX}.gc”
> etc.
>
> If folks have made customizations that count on this pattern, those
> changes should work unchanged after the revision.
>
> Thoughts?
>
> - Paul
>
>
> > On Nov 1, 2016, at 7:58 AM, John Omernik <jo...@omernik.com> wrote:
> >
> > export DRILL_LOG_PREFIX="$DRILL_LOG_DIR/drillbit"
> >
> > So looking at this, I think that DRILL_LOG_PREFIX is a misnomer, it
> should
> > be DRILL_LOG_PATH = Path to a directory, DRILL_LOG_PREFIX = A prefix to
> > prepend to drill log files, thus, in reality, DRILL_LOG_PREFIX should not
> > be set to $DRILL_LOG_DIR/drillbit, instead the default should be "
> >
> >
> > export DRILL_LOG_PREFIX=${DRILL_LOG_PREFIX:-"drillbit"}
> >
> > and then the next line that makes the drill log path should be:
> >
> > export DRILLBIT_LOG_PATH="${DRILL_LOG_DIR}/${DRILL_LOG_PREFIX}.log"
> >
> > That way the Prefix is just a file prefix, and then in the drill-env.sh I
> > could set
> >
> >
> > export DRILL_LOG_PREFIX="drillbit-$HOSTNAME"
> >
> >
> > and have all my logs in one folder, but not be overwritten by easy
> other....
> >
> >
> >
> >
> > On Tue, Nov 1, 2016 at 9:45 AM, John Omernik <jo...@omernik.com> wrote:
> >
> >> Hey all,
> >>
> >>
> >> I am getting my head around the new Drill config files in 1.8, and I
> have
> >> been setting DRILL_LOG_PREFIX in previous versions. Based on what I am
> >> seeing in drill-config.sh though, it looks like I no longer can set
> this as
> >> instead of using a "If set use what's set otherwise default to"
> methodology
> >> like other variables, drill-config.sh just exports a static value... is
> >> this correct?
> >>
> >>
> >> My goal is to have logs all in a single directory but have prefixes
> based
> >> on the host names... "centralized" logging if you will :) Is this
> possible
> >> in 1.8 or do I need a JIRA?
> >>
> >>
> >>
> >>
> >> export DRILL_LOG_PREFIX="$DRILL_LOG_DIR/drillbit"
> >>
> >>
> >>
> >> John
> >>
>
>
Re: DRILL_LOG_PREFIX in drill-config
Posted by Paul Rogers <pr...@maprtech.com>.
Hi All,
Since we are so close to release, I’ll see if I can get a fix PR submitted today based on John’s suggestions.
Just to be super careful, I suggest we introduce a new variable that holds the log name base file. In drill-env.sh:
# Define a custom base name for Drill log files.
# DRILL_LOG_NAME=“drillbit”
In drill-config.sh:
export DRILL_LOG_NAME=${DRILL_LOG_NAME:-"drillbit”}
export DRILL_LOG_PREFIX="$DRILL_LOG_DIR/$DRILL_LOG_NAME”
All other code can stay the same:
export DRILLBIT_LOG_PATH="${DRILL_LOG_PREFIX}.log"
logout="${DRILL_LOG_PREFIX}.out"
logqueries="${DRILL_LOG_PREFIX}_queries.json"
loggc="${DRILL_LOG_PREFIX}.gc”
etc.
If folks have made customizations that count on this pattern, those changes should work unchanged after the revision.
Thoughts?
- Paul
> On Nov 1, 2016, at 7:58 AM, John Omernik <jo...@omernik.com> wrote:
>
> export DRILL_LOG_PREFIX="$DRILL_LOG_DIR/drillbit"
>
> So looking at this, I think that DRILL_LOG_PREFIX is a misnomer, it should
> be DRILL_LOG_PATH = Path to a directory, DRILL_LOG_PREFIX = A prefix to
> prepend to drill log files, thus, in reality, DRILL_LOG_PREFIX should not
> be set to $DRILL_LOG_DIR/drillbit, instead the default should be "
>
>
> export DRILL_LOG_PREFIX=${DRILL_LOG_PREFIX:-"drillbit"}
>
> and then the next line that makes the drill log path should be:
>
> export DRILLBIT_LOG_PATH="${DRILL_LOG_DIR}/${DRILL_LOG_PREFIX}.log"
>
> That way the Prefix is just a file prefix, and then in the drill-env.sh I
> could set
>
>
> export DRILL_LOG_PREFIX="drillbit-$HOSTNAME"
>
>
> and have all my logs in one folder, but not be overwritten by easy other....
>
>
>
>
> On Tue, Nov 1, 2016 at 9:45 AM, John Omernik <jo...@omernik.com> wrote:
>
>> Hey all,
>>
>>
>> I am getting my head around the new Drill config files in 1.8, and I have
>> been setting DRILL_LOG_PREFIX in previous versions. Based on what I am
>> seeing in drill-config.sh though, it looks like I no longer can set this as
>> instead of using a "If set use what's set otherwise default to" methodology
>> like other variables, drill-config.sh just exports a static value... is
>> this correct?
>>
>>
>> My goal is to have logs all in a single directory but have prefixes based
>> on the host names... "centralized" logging if you will :) Is this possible
>> in 1.8 or do I need a JIRA?
>>
>>
>>
>>
>> export DRILL_LOG_PREFIX="$DRILL_LOG_DIR/drillbit"
>>
>>
>>
>> John
>>
Re: DRILL_LOG_PREFIX in drill-config
Posted by Paul Rogers <pr...@maprtech.com>.
Hi John,
Please file a JIRA for this. In fact, if you have a fix, please go ahead and issue a pull request; I’ll review it immediately.
Thanks,
- Paul
> On Nov 1, 2016, at 7:58 AM, John Omernik <jo...@omernik.com> wrote:
>
> export DRILL_LOG_PREFIX="$DRILL_LOG_DIR/drillbit"
>
> So looking at this, I think that DRILL_LOG_PREFIX is a misnomer, it should
> be DRILL_LOG_PATH = Path to a directory, DRILL_LOG_PREFIX = A prefix to
> prepend to drill log files, thus, in reality, DRILL_LOG_PREFIX should not
> be set to $DRILL_LOG_DIR/drillbit, instead the default should be "
>
>
> export DRILL_LOG_PREFIX=${DRILL_LOG_PREFIX:-"drillbit"}
>
> and then the next line that makes the drill log path should be:
>
> export DRILLBIT_LOG_PATH="${DRILL_LOG_DIR}/${DRILL_LOG_PREFIX}.log"
>
> That way the Prefix is just a file prefix, and then in the drill-env.sh I
> could set
>
>
> export DRILL_LOG_PREFIX="drillbit-$HOSTNAME"
>
>
> and have all my logs in one folder, but not be overwritten by easy other....
>
>
>
>
> On Tue, Nov 1, 2016 at 9:45 AM, John Omernik <jo...@omernik.com> wrote:
>
>> Hey all,
>>
>>
>> I am getting my head around the new Drill config files in 1.8, and I have
>> been setting DRILL_LOG_PREFIX in previous versions. Based on what I am
>> seeing in drill-config.sh though, it looks like I no longer can set this as
>> instead of using a "If set use what's set otherwise default to" methodology
>> like other variables, drill-config.sh just exports a static value... is
>> this correct?
>>
>>
>> My goal is to have logs all in a single directory but have prefixes based
>> on the host names... "centralized" logging if you will :) Is this possible
>> in 1.8 or do I need a JIRA?
>>
>>
>>
>>
>> export DRILL_LOG_PREFIX="$DRILL_LOG_DIR/drillbit"
>>
>>
>>
>> John
>>
Re: DRILL_LOG_PREFIX in drill-config
Posted by John Omernik <jo...@omernik.com>.
export DRILL_LOG_PREFIX="$DRILL_LOG_DIR/drillbit"
So looking at this, I think that DRILL_LOG_PREFIX is a misnomer, it should
be DRILL_LOG_PATH = Path to a directory, DRILL_LOG_PREFIX = A prefix to
prepend to drill log files, thus, in reality, DRILL_LOG_PREFIX should not
be set to $DRILL_LOG_DIR/drillbit, instead the default should be "
export DRILL_LOG_PREFIX=${DRILL_LOG_PREFIX:-"drillbit"}
and then the next line that makes the drill log path should be:
export DRILLBIT_LOG_PATH="${DRILL_LOG_DIR}/${DRILL_LOG_PREFIX}.log"
That way the Prefix is just a file prefix, and then in the drill-env.sh I
could set
export DRILL_LOG_PREFIX="drillbit-$HOSTNAME"
and have all my logs in one folder, but not be overwritten by easy other....
On Tue, Nov 1, 2016 at 9:45 AM, John Omernik <jo...@omernik.com> wrote:
> Hey all,
>
>
> I am getting my head around the new Drill config files in 1.8, and I have
> been setting DRILL_LOG_PREFIX in previous versions. Based on what I am
> seeing in drill-config.sh though, it looks like I no longer can set this as
> instead of using a "If set use what's set otherwise default to" methodology
> like other variables, drill-config.sh just exports a static value... is
> this correct?
>
>
> My goal is to have logs all in a single directory but have prefixes based
> on the host names... "centralized" logging if you will :) Is this possible
> in 1.8 or do I need a JIRA?
>
>
>
>
> export DRILL_LOG_PREFIX="$DRILL_LOG_DIR/drillbit"
>
>
>
> John
>