[jira] [Comment Edited] (KYLIN-1830) Put KYLIN_JVM_SETTINGS to kylin.properties

["Richard Calaba (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KYLIN-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359948#comment-15359948 ] 

Richard Calaba edited comment on KYLIN-1830 at 7/2/16 3:18 AM:
---------------------------------------------------------------

Okay - fair enough - but I prefer having configuration parameters for sizing in one location (/conf ) as we there other performance and execution env. related settings.

Thus I have found for you a generic solution you can use for ALL variables in setenv.sh - using this solution you can override the parameters from setenv.sh by values specified in the conf/kylin.properties file using same name of the environment variable.

See attached & updated setenv.sh and kylin.properties files (both from Kylin 1.5.2.1).

------------ Background info about implementation:

The trick is that I have defined 2 bash functions which try to read the property value override from conf/kylin.properties - and if found - it uses the value specified there ; if not found (or the value override is commented by #) then it uses the default value as specified in the bin/setenv.sh script.

Feel free to include it to the standard packaging - was thinking to provide patch here - but didn't find setenv.sh in source code - probably generated ?

Functions Defined - see setenv.sh attachment.

Example from setenv.sh - how the override of the variable KYLIN_JVM_SETTINGS:

Instead of:

     export KYLIN_JVM_SETTINGS="<default_value>"

Use:

     export_property_override KYLIN_JVM_SETTINGS "<default_value>"

I.e:

export_property_override KYLIN_JVM_SETTINGS "-Xms1024M -Xmx4096 -XX:MaxPermSize=512M -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$ -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"

Then in the conf/kylin.properties you can put line:

KYLIN_JVM_SETTINGS="-Xms1024M -Xmx16g -XX:MaxPermSize=512M -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$ -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"

Which will override the default value provided in setenv.sh

Tested and working. Enjoy!




was (Author: calaba@gmail.com):
Okay - fair enough - but I prefer having configuration parameters for sizing in one location (/conf ) as we there other performance and execution env. related settings.

Thus I have found for you a generic solution you can use for ALL variables in setenv.sh - using this solution you can override the parameters from setenv.sh by values specified in the conf/kylin.properties file using same name of the environment variable.

See attached & updated setenv.sh and kylin.properties files (both from Kylin 1.5.2.1).

------------ Background info about implementation:

The trick is that I have defined 2 bash functions which try to read the property value override from conf/kylin.properties - and if found - it uses the value specified there ; if not found (or the value override is commented by #) then it uses the default value as specified in the bin/setenv.sh script.

Feel free to include it to the standard packaging - was thinking to provide patch here - but didn't find setenv.sh in source code - probably generated ?

Functions Defined:

function parse_properties() {
        local param_value=$(awk -F "=" "(!/^(\$|[[:space:]]*#)/) && (/$2/) { idx = index(\$0,\"=\"); print substr(\$0,idx+1)}" "$1" | sed -e 's/^"//'  -e 's/"$//')
        if [[ -z "${param_value}" ]]; then
                echo $3
        else
                echo `eval echo "${param_value}"`
        fi
}

function export_property_override() {
# default Kylin property file location - for environment values override
        local kylin_property_file=${KYLIN_HOME}/conf/kylin.properties
        export "$1"="$(parse_properties "${kylin_property_file}" "$1" "$2" )"
}

# Example from setenv.sh - how the override of the variable KYLIN_JVM_SETTINGS:

Instead of:

     export KYLIN_JVM_SETTINGS="<default_value>"

Use:

     export_property_override KYLIN_JVM_SETTINGS "<default_value>"

I.e:

export_property_override KYLIN_JVM_SETTINGS "-Xms1024M -Xmx4096 -XX:MaxPermSize=512M -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$ -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"

Then in the conf/kylin.properties you can put line:

KYLIN_JVM_SETTINGS="-Xms1024M -Xmx16g -XX:MaxPermSize=512M -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$ -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"

Which will override the default value provided in setenv.sh

Tested and working. Enjoy!



> Put KYLIN_JVM_SETTINGS to kylin.properties
> ------------------------------------------
>
>                 Key: KYLIN-1830
>                 URL: https://issues.apache.org/jira/browse/KYLIN-1830
>             Project: Kylin
>          Issue Type: Improvement
>            Reporter: Richard Calaba
>            Priority: Minor
>         Attachments: kylin.properties, setenv.sh
>
>
> Currently is the KYLIN_JVM_SETTINGS variable stored in the ,/bin/setenv.sh ... which is not wrong, but as we have also some other memory specific setting in ./conf/kylin.properties file (like i.e kylin.job.mapreduce.default.reduce.input.mb or kylin.table.snapshot.max_mb) it might be good idea to have those performance and sizing related parameters in one location.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)