You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Pranav Murugappan (JIRA)" <ji...@apache.org> on 2017/10/24 10:10:00 UTC
[jira] [Updated] (SOLR-11233) GC_LOG_OPTS customisation is a little
confusing
[ https://issues.apache.org/jira/browse/SOLR-11233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pranav Murugappan updated SOLR-11233:
-------------------------------------
Attachment: SOLR-11233.patch
# SOLR-11233
## GC_LOG_OPTS customisation is a little confusing
https://issues.apache.org/jira/browse/SOLR-11233
Quoting from the recent release - https://github.com/apache/lucene-solr/blob/releases/lucene-solr/7.1.0/solr/bin/solr#L1801
```bash
1781 # Establish default GC logging opts if no env var set (otherwise init to sensible default)
1782 if [ -z ${GC_LOG_OPTS+x} ]; then
1783 if [[ "$JAVA_VER_NUM" < "9" ]] ; then
1784 GC_LOG_OPTS=('-verbose:gc' '-XX:+PrintHeapAtGC' '-XX:+PrintGCDetails' \
1785 '-XX:+PrintGCDateStamps' '-XX:+PrintGCTimeStamps' '-XX:+PrintTenuringDistribution' \
1786 '-XX:+PrintGCApplicationStoppedTime')
1787 else
1788 GC_LOG_OPTS=('-Xlog:gc*')
1789 fi
1790 else
1791 GC_LOG_OPTS=($GC_LOG_OPTS)
1792 fi
1793
```
```bash
1794 # if verbose gc logging enabled, setup the location of the log file and rotation
1795 if [ "$GC_LOG_OPTS" != "" ]; then
1796 if [[ "$JAVA_VER_NUM" < "9" ]] ; then
1797 gc_log_flag="-Xloggc"
1798 if [ "$JAVA_VENDOR" == "IBM J9" ]; then
1799 gc_log_flag="-Xverbosegclog"
1800 fi
1801 GC_LOG_OPTS+=("$gc_log_flag:$SOLR_LOGS_DIR/solr_gc.log" '-XX:+UseGCLogFileRotation' '-XX:NumberOfGCLogFiles=9' '-XX:GCLogFileSize=20M')
1802 else
1803 # http://openjdk.java.net/jeps/158
1804 for i in "${!GC_LOG_OPTS[@]}";
1805 do
1806 # for simplicity, we only look at the prefix '-Xlog:gc'
1807 # (if 'all' or multiple tags are used starting with anything other then 'gc' the user is on their own)
1808 # if a single additional ':' exists in param, then there is already an explicit output specifier
1809 GC_LOG_OPTS[$i]=$(echo ${GC_LOG_OPTS[$i]} | sed "s|^\(-Xlog:gc[^:]*$\)|\1:file=$SOLR_LOGS_DIR/solr_gc.log:time,uptime:filecount=9,filesize=20000|")
1810 done
1811 fi
1812 fi
```
Let's just talk about how it works for Java 8 now, as it seems to work as expected for Java 9. The first block of code checks if the user has set GC_LOG_OPTS environment variable. If he hasn't, it proceeds to set the default ones. If he has, it parses the options into an array (line 1791).
In the second block, the confusing part is line 1801. It appends the default options to the options provided by the user.
So for example, if I pass in ```GC_LOG_OPTS="-XX:NumberOfGCLogFiles=20"```, at the end of line 1801, `GC_LOG_OPTS` will be `('-XX:NumberOfGCLogFiles=20', "$gc_log_flag:$SOLR_LOGS_DIR/solr_gc.log" '-XX:+UseGCLogFileRotation' '-XX:NumberOfGCLogFiles=9' '-XX:GCLogFileSize=20M')`.
Which option will be actually selected is ambiguous and undeterministic. Checking it for GCLogFiles, `ps -ef | grep java | grep GCLogFiles` gave us a `9` which would mean it ignored user input. When we tested it for GCLogFileSize it showed both the values.
There are a couple of ways to solve the problem as mentioned in the linked JIRA issue.
We chose to go with a new GC_LOG_FILE_OPTS option. The way we've implemented it maintains existing behaviour for those who don't want to change any of their start scripts and provides an option to fix this ambiguous behaviour for those who want to change it. So, the users who want to fix it will now have to provide the GC log file options through the new GC_LOG_FILE_OPTS environment variable.
The changes are very subtle. We replace line 1801 with,
```bash
1801 if [ -z ${GC_LOG_FILE_OPTS+x} ]; then
1802 GC_LOG_FILE_OPTS=("$gc_log_flag:$SOLR_LOGS_DIR/solr_gc.log" '-XX:+UseGCLogFileRotation' '-XX:NumberOfGCLogFiles=9' '-XX:GCLogFileSize=20M')
1803 else
1804 GC_LOG_FILE_OPTS=($GC_LOG_FILE_OPTS)
1805 fi
1806 GC_LOG_OPTS+=$GC_LOG_FILE_OPTS
```
For Java 9, this ambiguous behavior doesn't seem to be present as the code (line 1809) only sets the default options either when the user hasn't provided any input or any specific file options.
> GC_LOG_OPTS customisation is a little confusing
> -----------------------------------------------
>
> Key: SOLR-11233
> URL: https://issues.apache.org/jira/browse/SOLR-11233
> Project: Solr
> Issue Type: Task
> Reporter: Christine Poerschke
> Priority: Minor
> Attachments: SOLR-11233.patch
>
>
> {{GC_LOG_OPTS}} customisation is currently supported but (pre Java 9) the customised settings are supplemented e.g. https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.6.0/solr/bin/solr#L1713
> {code}
> GC_LOG_OPTS+=("$gc_log_flag:$SOLR_LOGS_DIR/solr_gc.log" '-XX:+UseGCLogFileRotation' '-XX:NumberOfGCLogFiles=9' '-XX:GCLogFileSize=20M')
> {code}
> This seems unexpected and confusing. Some ideas for making it less confusing:
> * a new {{GC_LOG_FILE_OPTS}} option
> ** the new option can be customised but if unset it would default to existing behaviour
> * use customised GC_LOG_OPTS 'as is'
> ** this would be a change to existing behaviour i.e. the [solr#L1713|https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.6.0/solr/bin/solr#L1713] settings mentioned above would no longer be appended
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org