You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jesse Yates (JIRA)" <ji...@apache.org> on 2012/11/03 01:02:12 UTC
[jira] [Created] (HBASE-7091) support custom GC options in
hbase-env.sh
Jesse Yates created HBASE-7091:
----------------------------------
Summary: support custom GC options in hbase-env.sh
Key: HBASE-7091
URL: https://issues.apache.org/jira/browse/HBASE-7091
Project: HBase
Issue Type: Bug
Components: scripts
Reporter: Jesse Yates
When running things like bin/start-hbase and bin/hbase-daemon.sh start [master|regionserver|etc] we end up setting HBASE_OPTS property a couple times via calling hbase-env.sh. This is generally not a problem for most cases, but when you want to set your own GC log properties, one would think you should set HBASE_GC_OPTS, which get added to HBASE_OPTS.
NOPE! That would make too much sense.
Running bin/hbase-daemons.sh will run bin/hbase-daemon.sh with the daemons it needs to start. Each time through hbase-daemon.sh we also call bin/hbase. This isn't a big deal except for each call to hbase-daemon.sh, we also source hbase-env.sh twice (once in the script and once in bin/hbase). This is important for my next point.
Note that to turn on GC logging, you uncomment:
{code}
# export HBASE_OPTS="$HBASE_OPTS -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps $HBASE_GC_OPTS"
{code}
and then to log to a gc file for each server, you then uncomment:
{code}
# export HBASE_USE_GC_LOGFILE=true
{code}
in hbase-env.sh
On the first pass through hbase-daemon.sh, HBASE_GC_OPTS isn't set, so HBASE_OPTS doesn't get anything funky, but we set HBASE_USE_GC_LOGFILE, which then sets HBASE_GC_OPTS to the log file (-Xloggc:...). Then in bin/hbase we again run hbase-env.sh, which now hs HBASE_GC_OPTS set, adding the GC file.
This isn't a general problem because HBASE_OPTS is set without prefixing the existing HBASE_OPTS (eg. HBASE_OPTS="$HBASE_OPTS ..."), allowing easy updating. However, GC OPTS don't work the same and this is really odd behavior when you want to set your own GC opts, which can include turning on GC log rolling (yes, yes, they really are jvm opts, but they ought to support their own param, to help minimize clutter).
The simple version of this patch will just add an idempotent GC option to hbase-env.sh and some comments that uncommenting
{code}
# export HBASE_USE_GC_LOGFILE=true
{code}
will lead to a custom gc log file per server (along with an example name), so you don't need to set "-Xloggc".
The more complex solution does the above and also solves the multiple calls to hbase-env.sh so we can be sane about how all this works. Note that to fix this, hbase-daemon.sh just needs to read in HBASE_USE_GC_LOGFILE after sourcing hbase-env.sh and then update HBASE_OPTS. Oh and also not source hbase-env.sh in bin/hbase.
Even further, we might want to consider adding options just for cases where we don't need gc logging - i.e. the shell, the config reading tool, hcbk, etc. This is the hardest version to handle since the first couple will willy-nilly apply the gc options.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7091) support custom GC options in
hbase-env.sh
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499218#comment-13499218 ]
Lars Hofhansl commented on HBASE-7091:
--------------------------------------
Rereading your description, yes we should separate them.
> support custom GC options in hbase-env.sh
> -----------------------------------------
>
> Key: HBASE-7091
> URL: https://issues.apache.org/jira/browse/HBASE-7091
> Project: HBase
> Issue Type: Bug
> Components: scripts
> Affects Versions: 0.94.4
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Labels: newbie
> Fix For: 0.94.4
>
> Attachments: hbase-7091-v1.patch
>
>
> When running things like bin/start-hbase and bin/hbase-daemon.sh start [master|regionserver|etc] we end up setting HBASE_OPTS property a couple times via calling hbase-env.sh. This is generally not a problem for most cases, but when you want to set your own GC log properties, one would think you should set HBASE_GC_OPTS, which get added to HBASE_OPTS.
> NOPE! That would make too much sense.
> Running bin/hbase-daemons.sh will run bin/hbase-daemon.sh with the daemons it needs to start. Each time through hbase-daemon.sh we also call bin/hbase. This isn't a big deal except for each call to hbase-daemon.sh, we also source hbase-env.sh twice (once in the script and once in bin/hbase). This is important for my next point.
> Note that to turn on GC logging, you uncomment:
> {code}
> # export HBASE_OPTS="$HBASE_OPTS -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps $HBASE_GC_OPTS"
> {code}
> and then to log to a gc file for each server, you then uncomment:
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> in hbase-env.sh
> On the first pass through hbase-daemon.sh, HBASE_GC_OPTS isn't set, so HBASE_OPTS doesn't get anything funky, but we set HBASE_USE_GC_LOGFILE, which then sets HBASE_GC_OPTS to the log file (-Xloggc:...). Then in bin/hbase we again run hbase-env.sh, which now hs HBASE_GC_OPTS set, adding the GC file.
> This isn't a general problem because HBASE_OPTS is set without prefixing the existing HBASE_OPTS (eg. HBASE_OPTS="$HBASE_OPTS ..."), allowing easy updating. However, GC OPTS don't work the same and this is really odd behavior when you want to set your own GC opts, which can include turning on GC log rolling (yes, yes, they really are jvm opts, but they ought to support their own param, to help minimize clutter).
> The simple version of this patch will just add an idempotent GC option to hbase-env.sh and some comments that uncommenting
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> will lead to a custom gc log file per server (along with an example name), so you don't need to set "-Xloggc".
> The more complex solution does the above and also solves the multiple calls to hbase-env.sh so we can be sane about how all this works. Note that to fix this, hbase-daemon.sh just needs to read in HBASE_USE_GC_LOGFILE after sourcing hbase-env.sh and then update HBASE_OPTS. Oh and also not source hbase-env.sh in bin/hbase.
> Even further, we might want to consider adding options just for cases where we don't need gc logging - i.e. the shell, the config reading tool, hcbk, etc. This is the hardest version to handle since the first couple will willy-nilly apply the gc options.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-7091) support custom GC options in
hbase-env.sh
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates reassigned HBASE-7091:
----------------------------------
Assignee: Jesse Yates
> support custom GC options in hbase-env.sh
> -----------------------------------------
>
> Key: HBASE-7091
> URL: https://issues.apache.org/jira/browse/HBASE-7091
> Project: HBase
> Issue Type: Bug
> Components: scripts
> Affects Versions: 0.94.4
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Labels: newbie
> Fix For: 0.94.4
>
>
> When running things like bin/start-hbase and bin/hbase-daemon.sh start [master|regionserver|etc] we end up setting HBASE_OPTS property a couple times via calling hbase-env.sh. This is generally not a problem for most cases, but when you want to set your own GC log properties, one would think you should set HBASE_GC_OPTS, which get added to HBASE_OPTS.
> NOPE! That would make too much sense.
> Running bin/hbase-daemons.sh will run bin/hbase-daemon.sh with the daemons it needs to start. Each time through hbase-daemon.sh we also call bin/hbase. This isn't a big deal except for each call to hbase-daemon.sh, we also source hbase-env.sh twice (once in the script and once in bin/hbase). This is important for my next point.
> Note that to turn on GC logging, you uncomment:
> {code}
> # export HBASE_OPTS="$HBASE_OPTS -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps $HBASE_GC_OPTS"
> {code}
> and then to log to a gc file for each server, you then uncomment:
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> in hbase-env.sh
> On the first pass through hbase-daemon.sh, HBASE_GC_OPTS isn't set, so HBASE_OPTS doesn't get anything funky, but we set HBASE_USE_GC_LOGFILE, which then sets HBASE_GC_OPTS to the log file (-Xloggc:...). Then in bin/hbase we again run hbase-env.sh, which now hs HBASE_GC_OPTS set, adding the GC file.
> This isn't a general problem because HBASE_OPTS is set without prefixing the existing HBASE_OPTS (eg. HBASE_OPTS="$HBASE_OPTS ..."), allowing easy updating. However, GC OPTS don't work the same and this is really odd behavior when you want to set your own GC opts, which can include turning on GC log rolling (yes, yes, they really are jvm opts, but they ought to support their own param, to help minimize clutter).
> The simple version of this patch will just add an idempotent GC option to hbase-env.sh and some comments that uncommenting
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> will lead to a custom gc log file per server (along with an example name), so you don't need to set "-Xloggc".
> The more complex solution does the above and also solves the multiple calls to hbase-env.sh so we can be sane about how all this works. Note that to fix this, hbase-daemon.sh just needs to read in HBASE_USE_GC_LOGFILE after sourcing hbase-env.sh and then update HBASE_OPTS. Oh and also not source hbase-env.sh in bin/hbase.
> Even further, we might want to consider adding options just for cases where we don't need gc logging - i.e. the shell, the config reading tool, hcbk, etc. This is the hardest version to handle since the first couple will willy-nilly apply the gc options.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7091) support custom GC options in
hbase-env.sh
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499200#comment-13499200 ]
Lars Hofhansl commented on HBASE-7091:
--------------------------------------
Do we *need* to distinguish between server and client?
The client and server would typically not be the same machine, so having different GC settings in a single GC env variable seems reasonable.
> support custom GC options in hbase-env.sh
> -----------------------------------------
>
> Key: HBASE-7091
> URL: https://issues.apache.org/jira/browse/HBASE-7091
> Project: HBase
> Issue Type: Bug
> Components: scripts
> Affects Versions: 0.94.4
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Labels: newbie
> Fix For: 0.94.4
>
> Attachments: hbase-7091-v1.patch
>
>
> When running things like bin/start-hbase and bin/hbase-daemon.sh start [master|regionserver|etc] we end up setting HBASE_OPTS property a couple times via calling hbase-env.sh. This is generally not a problem for most cases, but when you want to set your own GC log properties, one would think you should set HBASE_GC_OPTS, which get added to HBASE_OPTS.
> NOPE! That would make too much sense.
> Running bin/hbase-daemons.sh will run bin/hbase-daemon.sh with the daemons it needs to start. Each time through hbase-daemon.sh we also call bin/hbase. This isn't a big deal except for each call to hbase-daemon.sh, we also source hbase-env.sh twice (once in the script and once in bin/hbase). This is important for my next point.
> Note that to turn on GC logging, you uncomment:
> {code}
> # export HBASE_OPTS="$HBASE_OPTS -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps $HBASE_GC_OPTS"
> {code}
> and then to log to a gc file for each server, you then uncomment:
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> in hbase-env.sh
> On the first pass through hbase-daemon.sh, HBASE_GC_OPTS isn't set, so HBASE_OPTS doesn't get anything funky, but we set HBASE_USE_GC_LOGFILE, which then sets HBASE_GC_OPTS to the log file (-Xloggc:...). Then in bin/hbase we again run hbase-env.sh, which now hs HBASE_GC_OPTS set, adding the GC file.
> This isn't a general problem because HBASE_OPTS is set without prefixing the existing HBASE_OPTS (eg. HBASE_OPTS="$HBASE_OPTS ..."), allowing easy updating. However, GC OPTS don't work the same and this is really odd behavior when you want to set your own GC opts, which can include turning on GC log rolling (yes, yes, they really are jvm opts, but they ought to support their own param, to help minimize clutter).
> The simple version of this patch will just add an idempotent GC option to hbase-env.sh and some comments that uncommenting
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> will lead to a custom gc log file per server (along with an example name), so you don't need to set "-Xloggc".
> The more complex solution does the above and also solves the multiple calls to hbase-env.sh so we can be sane about how all this works. Note that to fix this, hbase-daemon.sh just needs to read in HBASE_USE_GC_LOGFILE after sourcing hbase-env.sh and then update HBASE_OPTS. Oh and also not source hbase-env.sh in bin/hbase.
> Even further, we might want to consider adding options just for cases where we don't need gc logging - i.e. the shell, the config reading tool, hcbk, etc. This is the hardest version to handle since the first couple will willy-nilly apply the gc options.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7091) support custom GC options in
hbase-env.sh
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-7091:
-------------------------------
Fix Version/s: 0.94.4
> support custom GC options in hbase-env.sh
> -----------------------------------------
>
> Key: HBASE-7091
> URL: https://issues.apache.org/jira/browse/HBASE-7091
> Project: HBase
> Issue Type: Bug
> Components: scripts
> Affects Versions: 0.94.4
> Reporter: Jesse Yates
> Labels: newbie
> Fix For: 0.94.4
>
>
> When running things like bin/start-hbase and bin/hbase-daemon.sh start [master|regionserver|etc] we end up setting HBASE_OPTS property a couple times via calling hbase-env.sh. This is generally not a problem for most cases, but when you want to set your own GC log properties, one would think you should set HBASE_GC_OPTS, which get added to HBASE_OPTS.
> NOPE! That would make too much sense.
> Running bin/hbase-daemons.sh will run bin/hbase-daemon.sh with the daemons it needs to start. Each time through hbase-daemon.sh we also call bin/hbase. This isn't a big deal except for each call to hbase-daemon.sh, we also source hbase-env.sh twice (once in the script and once in bin/hbase). This is important for my next point.
> Note that to turn on GC logging, you uncomment:
> {code}
> # export HBASE_OPTS="$HBASE_OPTS -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps $HBASE_GC_OPTS"
> {code}
> and then to log to a gc file for each server, you then uncomment:
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> in hbase-env.sh
> On the first pass through hbase-daemon.sh, HBASE_GC_OPTS isn't set, so HBASE_OPTS doesn't get anything funky, but we set HBASE_USE_GC_LOGFILE, which then sets HBASE_GC_OPTS to the log file (-Xloggc:...). Then in bin/hbase we again run hbase-env.sh, which now hs HBASE_GC_OPTS set, adding the GC file.
> This isn't a general problem because HBASE_OPTS is set without prefixing the existing HBASE_OPTS (eg. HBASE_OPTS="$HBASE_OPTS ..."), allowing easy updating. However, GC OPTS don't work the same and this is really odd behavior when you want to set your own GC opts, which can include turning on GC log rolling (yes, yes, they really are jvm opts, but they ought to support their own param, to help minimize clutter).
> The simple version of this patch will just add an idempotent GC option to hbase-env.sh and some comments that uncommenting
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> will lead to a custom gc log file per server (along with an example name), so you don't need to set "-Xloggc".
> The more complex solution does the above and also solves the multiple calls to hbase-env.sh so we can be sane about how all this works. Note that to fix this, hbase-daemon.sh just needs to read in HBASE_USE_GC_LOGFILE after sourcing hbase-env.sh and then update HBASE_OPTS. Oh and also not source hbase-env.sh in bin/hbase.
> Even further, we might want to consider adding options just for cases where we don't need gc logging - i.e. the shell, the config reading tool, hcbk, etc. This is the hardest version to handle since the first couple will willy-nilly apply the gc options.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7091) support custom GC options in
hbase-env.sh
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499220#comment-13499220 ]
Lars Hofhansl commented on HBASE-7091:
--------------------------------------
BTW, you can just do:
{code}
for cmd in "shell" "hbck" "hlog" "hfile" "zkcli"; do
...
{code}
> support custom GC options in hbase-env.sh
> -----------------------------------------
>
> Key: HBASE-7091
> URL: https://issues.apache.org/jira/browse/HBASE-7091
> Project: HBase
> Issue Type: Bug
> Components: scripts
> Affects Versions: 0.94.4
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Labels: newbie
> Fix For: 0.94.4
>
> Attachments: hbase-7091-v1.patch
>
>
> When running things like bin/start-hbase and bin/hbase-daemon.sh start [master|regionserver|etc] we end up setting HBASE_OPTS property a couple times via calling hbase-env.sh. This is generally not a problem for most cases, but when you want to set your own GC log properties, one would think you should set HBASE_GC_OPTS, which get added to HBASE_OPTS.
> NOPE! That would make too much sense.
> Running bin/hbase-daemons.sh will run bin/hbase-daemon.sh with the daemons it needs to start. Each time through hbase-daemon.sh we also call bin/hbase. This isn't a big deal except for each call to hbase-daemon.sh, we also source hbase-env.sh twice (once in the script and once in bin/hbase). This is important for my next point.
> Note that to turn on GC logging, you uncomment:
> {code}
> # export HBASE_OPTS="$HBASE_OPTS -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps $HBASE_GC_OPTS"
> {code}
> and then to log to a gc file for each server, you then uncomment:
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> in hbase-env.sh
> On the first pass through hbase-daemon.sh, HBASE_GC_OPTS isn't set, so HBASE_OPTS doesn't get anything funky, but we set HBASE_USE_GC_LOGFILE, which then sets HBASE_GC_OPTS to the log file (-Xloggc:...). Then in bin/hbase we again run hbase-env.sh, which now hs HBASE_GC_OPTS set, adding the GC file.
> This isn't a general problem because HBASE_OPTS is set without prefixing the existing HBASE_OPTS (eg. HBASE_OPTS="$HBASE_OPTS ..."), allowing easy updating. However, GC OPTS don't work the same and this is really odd behavior when you want to set your own GC opts, which can include turning on GC log rolling (yes, yes, they really are jvm opts, but they ought to support their own param, to help minimize clutter).
> The simple version of this patch will just add an idempotent GC option to hbase-env.sh and some comments that uncommenting
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> will lead to a custom gc log file per server (along with an example name), so you don't need to set "-Xloggc".
> The more complex solution does the above and also solves the multiple calls to hbase-env.sh so we can be sane about how all this works. Note that to fix this, hbase-daemon.sh just needs to read in HBASE_USE_GC_LOGFILE after sourcing hbase-env.sh and then update HBASE_OPTS. Oh and also not source hbase-env.sh in bin/hbase.
> Even further, we might want to consider adding options just for cases where we don't need gc logging - i.e. the shell, the config reading tool, hcbk, etc. This is the hardest version to handle since the first couple will willy-nilly apply the gc options.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7091) support custom GC options in
hbase-env.sh
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-7091:
-------------------------------
Attachment: hbase-7091-v1.patch
Attaching first pass at a solution. It breaks the gc options into server and client options, which then get applied in hbase/bin/hbase depending on the $COMMAND passed to the script.
This is a little tricky because, as mentioned in the description) we actually source this script in basically every script in hbase/bin.
Tested locally and seems to work fine for a shell and a minicluster.
> support custom GC options in hbase-env.sh
> -----------------------------------------
>
> Key: HBASE-7091
> URL: https://issues.apache.org/jira/browse/HBASE-7091
> Project: HBase
> Issue Type: Bug
> Components: scripts
> Affects Versions: 0.94.4
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Labels: newbie
> Fix For: 0.94.4
>
> Attachments: hbase-7091-v1.patch
>
>
> When running things like bin/start-hbase and bin/hbase-daemon.sh start [master|regionserver|etc] we end up setting HBASE_OPTS property a couple times via calling hbase-env.sh. This is generally not a problem for most cases, but when you want to set your own GC log properties, one would think you should set HBASE_GC_OPTS, which get added to HBASE_OPTS.
> NOPE! That would make too much sense.
> Running bin/hbase-daemons.sh will run bin/hbase-daemon.sh with the daemons it needs to start. Each time through hbase-daemon.sh we also call bin/hbase. This isn't a big deal except for each call to hbase-daemon.sh, we also source hbase-env.sh twice (once in the script and once in bin/hbase). This is important for my next point.
> Note that to turn on GC logging, you uncomment:
> {code}
> # export HBASE_OPTS="$HBASE_OPTS -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps $HBASE_GC_OPTS"
> {code}
> and then to log to a gc file for each server, you then uncomment:
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> in hbase-env.sh
> On the first pass through hbase-daemon.sh, HBASE_GC_OPTS isn't set, so HBASE_OPTS doesn't get anything funky, but we set HBASE_USE_GC_LOGFILE, which then sets HBASE_GC_OPTS to the log file (-Xloggc:...). Then in bin/hbase we again run hbase-env.sh, which now hs HBASE_GC_OPTS set, adding the GC file.
> This isn't a general problem because HBASE_OPTS is set without prefixing the existing HBASE_OPTS (eg. HBASE_OPTS="$HBASE_OPTS ..."), allowing easy updating. However, GC OPTS don't work the same and this is really odd behavior when you want to set your own GC opts, which can include turning on GC log rolling (yes, yes, they really are jvm opts, but they ought to support their own param, to help minimize clutter).
> The simple version of this patch will just add an idempotent GC option to hbase-env.sh and some comments that uncommenting
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> will lead to a custom gc log file per server (along with an example name), so you don't need to set "-Xloggc".
> The more complex solution does the above and also solves the multiple calls to hbase-env.sh so we can be sane about how all this works. Note that to fix this, hbase-daemon.sh just needs to read in HBASE_USE_GC_LOGFILE after sourcing hbase-env.sh and then update HBASE_OPTS. Oh and also not source hbase-env.sh in bin/hbase.
> Even further, we might want to consider adding options just for cases where we don't need gc logging - i.e. the shell, the config reading tool, hcbk, etc. This is the hardest version to handle since the first couple will willy-nilly apply the gc options.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7091) support custom GC options in
hbase-env.sh
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499203#comment-13499203 ]
Jesse Yates commented on HBASE-7091:
------------------------------------
We don't need to, but I wouldn't be surprised if people commonly use the same config for both. This can can lead to a basically unusable shell (due to the way logging gets setup for the client), but the correct behavior for the server.
> support custom GC options in hbase-env.sh
> -----------------------------------------
>
> Key: HBASE-7091
> URL: https://issues.apache.org/jira/browse/HBASE-7091
> Project: HBase
> Issue Type: Bug
> Components: scripts
> Affects Versions: 0.94.4
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Labels: newbie
> Fix For: 0.94.4
>
> Attachments: hbase-7091-v1.patch
>
>
> When running things like bin/start-hbase and bin/hbase-daemon.sh start [master|regionserver|etc] we end up setting HBASE_OPTS property a couple times via calling hbase-env.sh. This is generally not a problem for most cases, but when you want to set your own GC log properties, one would think you should set HBASE_GC_OPTS, which get added to HBASE_OPTS.
> NOPE! That would make too much sense.
> Running bin/hbase-daemons.sh will run bin/hbase-daemon.sh with the daemons it needs to start. Each time through hbase-daemon.sh we also call bin/hbase. This isn't a big deal except for each call to hbase-daemon.sh, we also source hbase-env.sh twice (once in the script and once in bin/hbase). This is important for my next point.
> Note that to turn on GC logging, you uncomment:
> {code}
> # export HBASE_OPTS="$HBASE_OPTS -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps $HBASE_GC_OPTS"
> {code}
> and then to log to a gc file for each server, you then uncomment:
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> in hbase-env.sh
> On the first pass through hbase-daemon.sh, HBASE_GC_OPTS isn't set, so HBASE_OPTS doesn't get anything funky, but we set HBASE_USE_GC_LOGFILE, which then sets HBASE_GC_OPTS to the log file (-Xloggc:...). Then in bin/hbase we again run hbase-env.sh, which now hs HBASE_GC_OPTS set, adding the GC file.
> This isn't a general problem because HBASE_OPTS is set without prefixing the existing HBASE_OPTS (eg. HBASE_OPTS="$HBASE_OPTS ..."), allowing easy updating. However, GC OPTS don't work the same and this is really odd behavior when you want to set your own GC opts, which can include turning on GC log rolling (yes, yes, they really are jvm opts, but they ought to support their own param, to help minimize clutter).
> The simple version of this patch will just add an idempotent GC option to hbase-env.sh and some comments that uncommenting
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> will lead to a custom gc log file per server (along with an example name), so you don't need to set "-Xloggc".
> The more complex solution does the above and also solves the multiple calls to hbase-env.sh so we can be sane about how all this works. Note that to fix this, hbase-daemon.sh just needs to read in HBASE_USE_GC_LOGFILE after sourcing hbase-env.sh and then update HBASE_OPTS. Oh and also not source hbase-env.sh in bin/hbase.
> Even further, we might want to consider adding options just for cases where we don't need gc logging - i.e. the shell, the config reading tool, hcbk, etc. This is the hardest version to handle since the first couple will willy-nilly apply the gc options.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7091) support custom GC options in
hbase-env.sh
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499225#comment-13499225 ]
Jesse Yates commented on HBASE-7091:
------------------------------------
yeah, I could. I thought using the array would be cleaner, as its more obvious (at least to me) where I might go to update these things. Happy to switch it.
> support custom GC options in hbase-env.sh
> -----------------------------------------
>
> Key: HBASE-7091
> URL: https://issues.apache.org/jira/browse/HBASE-7091
> Project: HBase
> Issue Type: Bug
> Components: scripts
> Affects Versions: 0.94.4
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Labels: newbie
> Fix For: 0.94.4
>
> Attachments: hbase-7091-v1.patch
>
>
> When running things like bin/start-hbase and bin/hbase-daemon.sh start [master|regionserver|etc] we end up setting HBASE_OPTS property a couple times via calling hbase-env.sh. This is generally not a problem for most cases, but when you want to set your own GC log properties, one would think you should set HBASE_GC_OPTS, which get added to HBASE_OPTS.
> NOPE! That would make too much sense.
> Running bin/hbase-daemons.sh will run bin/hbase-daemon.sh with the daemons it needs to start. Each time through hbase-daemon.sh we also call bin/hbase. This isn't a big deal except for each call to hbase-daemon.sh, we also source hbase-env.sh twice (once in the script and once in bin/hbase). This is important for my next point.
> Note that to turn on GC logging, you uncomment:
> {code}
> # export HBASE_OPTS="$HBASE_OPTS -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps $HBASE_GC_OPTS"
> {code}
> and then to log to a gc file for each server, you then uncomment:
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> in hbase-env.sh
> On the first pass through hbase-daemon.sh, HBASE_GC_OPTS isn't set, so HBASE_OPTS doesn't get anything funky, but we set HBASE_USE_GC_LOGFILE, which then sets HBASE_GC_OPTS to the log file (-Xloggc:...). Then in bin/hbase we again run hbase-env.sh, which now hs HBASE_GC_OPTS set, adding the GC file.
> This isn't a general problem because HBASE_OPTS is set without prefixing the existing HBASE_OPTS (eg. HBASE_OPTS="$HBASE_OPTS ..."), allowing easy updating. However, GC OPTS don't work the same and this is really odd behavior when you want to set your own GC opts, which can include turning on GC log rolling (yes, yes, they really are jvm opts, but they ought to support their own param, to help minimize clutter).
> The simple version of this patch will just add an idempotent GC option to hbase-env.sh and some comments that uncommenting
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> will lead to a custom gc log file per server (along with an example name), so you don't need to set "-Xloggc".
> The more complex solution does the above and also solves the multiple calls to hbase-env.sh so we can be sane about how all this works. Note that to fix this, hbase-daemon.sh just needs to read in HBASE_USE_GC_LOGFILE after sourcing hbase-env.sh and then update HBASE_OPTS. Oh and also not source hbase-env.sh in bin/hbase.
> Even further, we might want to consider adding options just for cases where we don't need gc logging - i.e. the shell, the config reading tool, hcbk, etc. This is the hardest version to handle since the first couple will willy-nilly apply the gc options.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7091) support custom GC options in
hbase-env.sh
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499216#comment-13499216 ]
Lars Hofhansl commented on HBASE-7091:
--------------------------------------
Yeah probably fine. Just worried that it will lead to confusion.
> support custom GC options in hbase-env.sh
> -----------------------------------------
>
> Key: HBASE-7091
> URL: https://issues.apache.org/jira/browse/HBASE-7091
> Project: HBase
> Issue Type: Bug
> Components: scripts
> Affects Versions: 0.94.4
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Labels: newbie
> Fix For: 0.94.4
>
> Attachments: hbase-7091-v1.patch
>
>
> When running things like bin/start-hbase and bin/hbase-daemon.sh start [master|regionserver|etc] we end up setting HBASE_OPTS property a couple times via calling hbase-env.sh. This is generally not a problem for most cases, but when you want to set your own GC log properties, one would think you should set HBASE_GC_OPTS, which get added to HBASE_OPTS.
> NOPE! That would make too much sense.
> Running bin/hbase-daemons.sh will run bin/hbase-daemon.sh with the daemons it needs to start. Each time through hbase-daemon.sh we also call bin/hbase. This isn't a big deal except for each call to hbase-daemon.sh, we also source hbase-env.sh twice (once in the script and once in bin/hbase). This is important for my next point.
> Note that to turn on GC logging, you uncomment:
> {code}
> # export HBASE_OPTS="$HBASE_OPTS -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps $HBASE_GC_OPTS"
> {code}
> and then to log to a gc file for each server, you then uncomment:
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> in hbase-env.sh
> On the first pass through hbase-daemon.sh, HBASE_GC_OPTS isn't set, so HBASE_OPTS doesn't get anything funky, but we set HBASE_USE_GC_LOGFILE, which then sets HBASE_GC_OPTS to the log file (-Xloggc:...). Then in bin/hbase we again run hbase-env.sh, which now hs HBASE_GC_OPTS set, adding the GC file.
> This isn't a general problem because HBASE_OPTS is set without prefixing the existing HBASE_OPTS (eg. HBASE_OPTS="$HBASE_OPTS ..."), allowing easy updating. However, GC OPTS don't work the same and this is really odd behavior when you want to set your own GC opts, which can include turning on GC log rolling (yes, yes, they really are jvm opts, but they ought to support their own param, to help minimize clutter).
> The simple version of this patch will just add an idempotent GC option to hbase-env.sh and some comments that uncommenting
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> will lead to a custom gc log file per server (along with an example name), so you don't need to set "-Xloggc".
> The more complex solution does the above and also solves the multiple calls to hbase-env.sh so we can be sane about how all this works. Note that to fix this, hbase-daemon.sh just needs to read in HBASE_USE_GC_LOGFILE after sourcing hbase-env.sh and then update HBASE_OPTS. Oh and also not source hbase-env.sh in bin/hbase.
> Even further, we might want to consider adding options just for cases where we don't need gc logging - i.e. the shell, the config reading tool, hcbk, etc. This is the hardest version to handle since the first couple will willy-nilly apply the gc options.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7091) support custom GC options in
hbase-env.sh
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-7091:
-------------------------------
Affects Version/s: 0.94.4
> support custom GC options in hbase-env.sh
> -----------------------------------------
>
> Key: HBASE-7091
> URL: https://issues.apache.org/jira/browse/HBASE-7091
> Project: HBase
> Issue Type: Bug
> Components: scripts
> Affects Versions: 0.94.4
> Reporter: Jesse Yates
> Labels: newbie
> Fix For: 0.94.4
>
>
> When running things like bin/start-hbase and bin/hbase-daemon.sh start [master|regionserver|etc] we end up setting HBASE_OPTS property a couple times via calling hbase-env.sh. This is generally not a problem for most cases, but when you want to set your own GC log properties, one would think you should set HBASE_GC_OPTS, which get added to HBASE_OPTS.
> NOPE! That would make too much sense.
> Running bin/hbase-daemons.sh will run bin/hbase-daemon.sh with the daemons it needs to start. Each time through hbase-daemon.sh we also call bin/hbase. This isn't a big deal except for each call to hbase-daemon.sh, we also source hbase-env.sh twice (once in the script and once in bin/hbase). This is important for my next point.
> Note that to turn on GC logging, you uncomment:
> {code}
> # export HBASE_OPTS="$HBASE_OPTS -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps $HBASE_GC_OPTS"
> {code}
> and then to log to a gc file for each server, you then uncomment:
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> in hbase-env.sh
> On the first pass through hbase-daemon.sh, HBASE_GC_OPTS isn't set, so HBASE_OPTS doesn't get anything funky, but we set HBASE_USE_GC_LOGFILE, which then sets HBASE_GC_OPTS to the log file (-Xloggc:...). Then in bin/hbase we again run hbase-env.sh, which now hs HBASE_GC_OPTS set, adding the GC file.
> This isn't a general problem because HBASE_OPTS is set without prefixing the existing HBASE_OPTS (eg. HBASE_OPTS="$HBASE_OPTS ..."), allowing easy updating. However, GC OPTS don't work the same and this is really odd behavior when you want to set your own GC opts, which can include turning on GC log rolling (yes, yes, they really are jvm opts, but they ought to support their own param, to help minimize clutter).
> The simple version of this patch will just add an idempotent GC option to hbase-env.sh and some comments that uncommenting
> {code}
> # export HBASE_USE_GC_LOGFILE=true
> {code}
> will lead to a custom gc log file per server (along with an example name), so you don't need to set "-Xloggc".
> The more complex solution does the above and also solves the multiple calls to hbase-env.sh so we can be sane about how all this works. Note that to fix this, hbase-daemon.sh just needs to read in HBASE_USE_GC_LOGFILE after sourcing hbase-env.sh and then update HBASE_OPTS. Oh and also not source hbase-env.sh in bin/hbase.
> Even further, we might want to consider adding options just for cases where we don't need gc logging - i.e. the shell, the config reading tool, hcbk, etc. This is the hardest version to handle since the first couple will willy-nilly apply the gc options.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira