You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by t spam <sp...@gmail.com> on 2020/07/24 06:23:46 UTC
JsonLayout breaks logging?
Hi,
I'm having difficulty configuring JsonLayout for appenders. I have the
following in my log4j2.xml:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout>
<Pattern>
%d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard}
%X{replica} %X{core}] %c{1.} %m%n
</Pattern>
</PatternLayout>
</Console>
<RollingFile
name="RollingFile"
fileName="${sys:solr.log.dir}/solr.log"
filePattern="${sys:solr.log.dir}/solr.log.%i" >
<JsonLayout complete="false" compact="true" eventEol="true" />
<Policies>
<OnStartupTriggeringPolicy />
<SizeBasedTriggeringPolicy size="32 MB"/>
</Policies>
<DefaultRolloverStrategy max="10"/>
</RollingFile>
<RollingFile
name="SlowFile"
fileName="${sys:solr.log.dir}/solr_slow_requests.log"
filePattern="${sys:solr.log.dir}/solr_slow_requests.log.%i" >
<PatternLayout>
<Pattern>
%d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard}
%X{replica} %X{core}] %c{1.} %m%n
</Pattern>
</PatternLayout>
<Policies>
<OnStartupTriggeringPolicy />
<SizeBasedTriggeringPolicy size="32 MB"/>
</Policies>
<DefaultRolloverStrategy max="10"/>
</RollingFile>
</Appenders>
<Loggers>
<Logger name="org.apache.hadoop" level="warn"/>
<Logger name="org.apache.solr.update.LoggingInfoStream" level="off"/>
<Logger name="org.apache.zookeeper" level="warn"/>
<Logger name="org.apache.solr.core.SolrCore.SlowRequest" level="info"
additivity="false">
<AppenderRef ref="SlowFile"/>
</Logger>
<Root level="debug">
<AppenderRef ref="RollingFile"/>
<AppenderRef ref="STDOUT"/>
</Root>
</Loggers>
</Configuration>
For some reason it just stops logging anything. I only get the solr_gc.log
and not the expected solr.log. I see an old thread mentioning the exact
same issue (except that solr.log isn't even created in my case) but it
wasn't resolved there. See here:
http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201809.mbox/%3cDB5PR0101MB211949C76C5C39472ECDFF87F4050@DB5PR0101MB2119.eurprd01.prod.exchangelabs.com%3e
I see this behaviour with both SOLR 7 and 8.
Am I missing something? Help is much appreciated.
- Tijmen
This was the original message:
-----Ursprüngliche Nachricht-----
Von: Shawn Heisey <ap...@elyograg.org>
Gesendet: Freitag, 7. September 2018 17:58
An: solr-user@lucene.apache.org
Betreff: Re: Solr 7.4 and log4j2 JSONLayout
On 9/6/2018 7:46 AM, Michael Aleythe, Sternwald wrote:
> I'm trying to edit the log4j2 logging configuration for solr. The goal is to get a log
file in json format. I configured the the JSONLayout for this purpose
inside the rollingFile
appender in the log4j2.xml. After this solr stops logging entirely.
Solr.log file is empty.
Only the solr-8983-console.log file contains 10 lines. The line
"2018-09-06 13:22:25.378:INFO:oejs.Server:main:
Started @2814ms" is the last one.
> My first guess was that the jackson-core and jackson-databind jars were missing, but
that did not fix the problem.
As Varun said, jackson is already included in Solr. You won't need to
add any jars for that.
> Does anyone know where to find error-messages or exceptions that point me towards whats
going wrong here?
Start Solr in the foreground, with the -f option. This will cause
Solr to log to the console.
When Solr is started in the background, it suppresses console logging.
I see that you have changed the logfile rollover size to 1MB. If your
Solr server sees much
traffic, this is going to result in an extremely fast rollover, which
may mean that you lose
access to logged events VERY quickly. This will especially be the
case with JSON logging
-- each event will take up a lot more space.
Thanks,
Shawn
Re: JsonLayout breaks logging?
Posted by t spam <sp...@gmail.com>.
Hi, is there anyone out there that could give me at least a direction? I
just can't seem to figure this one out. Kind regards,
Tijmen
On Wed, Jul 29, 2020 at 9:13 AM t spam <sp...@gmail.com> wrote:
>
>> Hi Naz and other solr-users (now with solr-user in to),
>>
>> Excuse my ignorance here (just getting started) but let's take the
>> techproducts example. As you proposed I included the latest jackson-core
>> and jackson-databind jars in the "solr install dir/lib/" directory:
>>
>> [tijmen@solr-1 solr-7.7.3]$ ls -la lib/
>> total 1732
>> drwxrwxr-x. 2 tijmen tijmen 72 Jul 25 08:06 .
>> drwxr-xr-x. 10 tijmen tijmen 212 Jul 25 07:45 ..
>> -rw-rw-r--. 1 tijmen tijmen 351575 Jul 25 07:48 jackson-core-2.11.1.jar
>> -rw-rw-r--. 1 tijmen tijmen 1419800 Jul 25 07:48
>> jackson-databind-2.11.1.jar
>>
>> I then added a lib directive to include the jackson jars
>> in: example/techproducts/solr/techproducts/conf/solrconfig.xml
>>
>> <lib dir="${solr.install.dir:../../../..}/contrib/extraction/lib"
>> regex=".*\.jar" />
>> <lib dir="${solr.install.dir:../../../..}/dist/"
>> regex="solr-cell-\d.*\.jar" />
>>
>> <lib dir="${solr.install.dir:../../../..}/contrib/clustering/lib/"
>> regex=".*\.jar" />
>> <lib dir="${solr.install.dir:../../../..}/dist/"
>> regex="solr-clustering-\d.*\.jar" />
>>
>> <lib dir="${solr.install.dir:../../../..}/contrib/langid/lib/"
>> regex=".*\.jar" />
>> <lib dir="${solr.install.dir:../../../..}/dist/"
>> regex="solr-langid-\d.*\.jar" />
>>
>> <lib dir="${solr.install.dir:../../../..}/dist/"
>> regex="solr-ltr-\d.*\.jar" />
>>
>> <lib dir="${solr.install.dir:../../../..}/contrib/velocity/lib"
>> regex=".*\.jar" />
>> <lib dir="${solr.install.dir:../../../..}/dist/"
>> regex="solr-velocity-\d.*\.jar" />
>>
>> <lib dir="${solr.install.dir:../../../..}/lib/" regex="jackson-.*\.jar"
>> />
>>
>> I start solr using:
>>
>> [tijmen@solr-1 solr-7.7.3]$ ./bin/solr stop -e techproducts
>>
>> Unfortunately I get the same result. Solr starts but no logging.
>>
>> Whenever I remove the JsonLayout from the log4j2.xml it starts logging as
>> expected.
>>
>> Thanks,
>>
>> Tijmen
>>
>> On Fri, Jul 24, 2020 at 6:48 PM Naz S <sn...@gmail.com> wrote:
>>
>>> Hi Tijmen,
>>>
>>> If you use maven, for example, you can add dependencies in pom.xml.
>>>
>>> For example,
>>> <dependencies>
>>> <dependency>
>>> <groupId>com.fasterxml.jackson.core</groupId>
>>> <artifactId>jackson-core</artifactId>
>>> <version>2.11.1</version>
>>> </dependency>
>>> <dependency>
>>> <groupId>com.fasterxml.jackson.core</groupId>
>>> <artifactId>jackson-databind</artifactId>
>>> <version>2.11.1</version>
>>> </dependency>
>>> </dependencies>
>>>
>>> On Fri, Jul 24, 2020 at 1:37 PM t spam <sp...@gmail.com> wrote:
>>>
>>>> Hi Naz,
>>>>
>>>> Could you give me some directions in where or how I should provide
>>>> these dependencies? I can see these dependencies are already in various
>>>> places by default:
>>>>
>>>> [tijmen@solr-1 solr-7.7.3]$ find . -name jackson*
>>>> ./contrib/clustering/lib/jackson-annotations-2.9.8.jar
>>>> ./contrib/clustering/lib/jackson-databind-2.9.8.jar
>>>> ./contrib/prometheus-exporter/lib/jackson-annotations-2.9.8.jar
>>>> ./contrib/prometheus-exporter/lib/jackson-core-2.9.8.jar
>>>> ./contrib/prometheus-exporter/lib/jackson-databind-2.9.8.jar
>>>> ./contrib/prometheus-exporter/lib/jackson-jq-0.0.8.jar
>>>> ./licenses/jackson-annotations-2.9.8.jar.sha1
>>>> ./licenses/jackson-annotations-LICENSE-ASL.txt
>>>> ./licenses/jackson-annotations-NOTICE.txt
>>>> ./licenses/jackson-core-2.9.8.jar.sha1
>>>> ./licenses/jackson-core-LICENSE-ASL.txt
>>>> ./licenses/jackson-core-NOTICE.txt
>>>> ./licenses/jackson-core-asl-1.9.13.jar.sha1
>>>> ./licenses/jackson-core-asl-LICENSE-ASL.txt
>>>> ./licenses/jackson-core-asl-NOTICE.txt
>>>> ./licenses/jackson-databind-2.9.8.jar.sha1
>>>> ./licenses/jackson-databind-LICENSE-ASL.txt
>>>> ./licenses/jackson-databind-NOTICE.txt
>>>> ./licenses/jackson-dataformat-smile-2.9.8.jar.sha1
>>>> ./licenses/jackson-dataformat-smile-LICENSE-ASL.txt
>>>> ./licenses/jackson-dataformat-smile-NOTICE.txt
>>>> ./licenses/jackson-jq-0.0.8.jar.sha1
>>>> ./licenses/jackson-jq-LICENSE-ASL.txt
>>>> ./licenses/jackson-jq-NOTICE.txt
>>>> ./licenses/jackson-mapper-asl-1.9.13.jar.sha1
>>>> ./licenses/jackson-mapper-asl-LICENSE-ASL.txt
>>>> ./licenses/jackson-mapper-asl-NOTICE.txt
>>>> ./server/solr-webapp/webapp/WEB-INF/lib/jackson-annotations-2.9.8.jar
>>>> ./server/solr-webapp/webapp/WEB-INF/lib/jackson-core-2.9.8.jar
>>>> ./server/solr-webapp/webapp/WEB-INF/lib/jackson-core-asl-1.9.13.jar
>>>> ./server/solr-webapp/webapp/WEB-INF/lib/jackson-databind-2.9.8.jar
>>>>
>>>> ./server/solr-webapp/webapp/WEB-INF/lib/jackson-dataformat-smile-2.9.8.jar
>>>> ./server/solr-webapp/webapp/WEB-INF/lib/jackson-mapper-asl-1.9.13.jar
>>>>
>>>> Thanks for your time.
>>>>
>>>> Tijmen
>>>>
>>>> On Fri, Jul 24, 2020 at 1:16 PM Naz S <sn...@gmail.com> wrote:
>>>>
>>>>>
>>>>> You should explicitly provide the jackson dependencies: jackson-core,
>>>>> jackson-databind and/or jackson-annotations.
>>>>>
>>>>> On Fri, Jul 24, 2020 at 8:24 AM t spam <sp...@gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm having difficulty configuring JsonLayout for appenders. I have the
>>>>>> following in my log4j2.xml:
>>>>>>
>>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>>> <Configuration>
>>>>>> <Appenders>
>>>>>>
>>>>>> <Console name="STDOUT" target="SYSTEM_OUT">
>>>>>> <PatternLayout>
>>>>>> <Pattern>
>>>>>> %d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection}
>>>>>> %X{shard}
>>>>>> %X{replica} %X{core}] %c{1.} %m%n
>>>>>> </Pattern>
>>>>>> </PatternLayout>
>>>>>> </Console>
>>>>>>
>>>>>> <RollingFile
>>>>>> name="RollingFile"
>>>>>> fileName="${sys:solr.log.dir}/solr.log"
>>>>>> filePattern="${sys:solr.log.dir}/solr.log.%i" >
>>>>>> <JsonLayout complete="false" compact="true" eventEol="true" />
>>>>>> <Policies>
>>>>>> <OnStartupTriggeringPolicy />
>>>>>> <SizeBasedTriggeringPolicy size="32 MB"/>
>>>>>> </Policies>
>>>>>> <DefaultRolloverStrategy max="10"/>
>>>>>> </RollingFile>
>>>>>>
>>>>>> <RollingFile
>>>>>> name="SlowFile"
>>>>>> fileName="${sys:solr.log.dir}/solr_slow_requests.log"
>>>>>> filePattern="${sys:solr.log.dir}/solr_slow_requests.log.%i" >
>>>>>> <PatternLayout>
>>>>>> <Pattern>
>>>>>> %d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection}
>>>>>> %X{shard}
>>>>>> %X{replica} %X{core}] %c{1.} %m%n
>>>>>> </Pattern>
>>>>>> </PatternLayout>
>>>>>> <Policies>
>>>>>> <OnStartupTriggeringPolicy />
>>>>>> <SizeBasedTriggeringPolicy size="32 MB"/>
>>>>>> </Policies>
>>>>>> <DefaultRolloverStrategy max="10"/>
>>>>>> </RollingFile>
>>>>>>
>>>>>> </Appenders>
>>>>>> <Loggers>
>>>>>> <Logger name="org.apache.hadoop" level="warn"/>
>>>>>> <Logger name="org.apache.solr.update.LoggingInfoStream"
>>>>>> level="off"/>
>>>>>> <Logger name="org.apache.zookeeper" level="warn"/>
>>>>>> <Logger name="org.apache.solr.core.SolrCore.SlowRequest"
>>>>>> level="info"
>>>>>> additivity="false">
>>>>>> <AppenderRef ref="SlowFile"/>
>>>>>> </Logger>
>>>>>>
>>>>>> <Root level="debug">
>>>>>> <AppenderRef ref="RollingFile"/>
>>>>>> <AppenderRef ref="STDOUT"/>
>>>>>> </Root>
>>>>>> </Loggers>
>>>>>> </Configuration>
>>>>>>
>>>>>> For some reason it just stops logging anything. I only get the
>>>>>> solr_gc.log
>>>>>> and not the expected solr.log. I see an old thread mentioning the
>>>>>> exact
>>>>>> same issue (except that solr.log isn't even created in my case) but it
>>>>>> wasn't resolved there. See here:
>>>>>>
>>>>>> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201809.mbox/%3cDB5PR0101MB211949C76C5C39472ECDFF87F4050@DB5PR0101MB2119.eurprd01.prod.exchangelabs.com%3e
>>>>>>
>>>>>> I see this behaviour with both SOLR 7 and 8.
>>>>>>
>>>>>> Am I missing something? Help is much appreciated.
>>>>>>
>>>>>> - Tijmen
>>>>>>
>>>>>> This was the original message:
>>>>>>
>>>>>> -----Ursprüngliche Nachricht-----
>>>>>> Von: Shawn Heisey <ap...@elyograg.org>
>>>>>> Gesendet: Freitag, 7. September 2018 17:58
>>>>>> An: solr-user@lucene.apache.org
>>>>>> Betreff: Re: Solr 7.4 and log4j2 JSONLayout
>>>>>>
>>>>>> On 9/6/2018 7:46 AM, Michael Aleythe, Sternwald wrote:
>>>>>> > I'm trying to edit the log4j2 logging configuration for solr. The
>>>>>> goal is to get a log
>>>>>> file in json format. I configured the the JSONLayout for this purpose
>>>>>> inside the rollingFile
>>>>>> appender in the log4j2.xml. After this solr stops logging entirely.
>>>>>> Solr.log file is empty.
>>>>>> Only the solr-8983-console.log file contains 10 lines. The line
>>>>>> "2018-09-06 13:22:25.378:INFO:oejs.Server:main:
>>>>>> Started @2814ms" is the last one.
>>>>>> > My first guess was that the jackson-core and jackson-databind jars
>>>>>> were missing, but
>>>>>> that did not fix the problem.
>>>>>>
>>>>>> As Varun said, jackson is already included in Solr. You won't need to
>>>>>> add any jars for that.
>>>>>>
>>>>>> > Does anyone know where to find error-messages or exceptions that
>>>>>> point me towards whats
>>>>>> going wrong here?
>>>>>>
>>>>>> Start Solr in the foreground, with the -f option. This will cause
>>>>>> Solr to log to the console.
>>>>>> When Solr is started in the background, it suppresses console logging.
>>>>>>
>>>>>> I see that you have changed the logfile rollover size to 1MB. If your
>>>>>> Solr server sees much
>>>>>> traffic, this is going to result in an extremely fast rollover, which
>>>>>> may mean that you lose
>>>>>> access to logged events VERY quickly. This will especially be the
>>>>>> case with JSON logging
>>>>>> -- each event will take up a lot more space.
>>>>>>
>>>>>> Thanks,
>>>>>> Shawn
>>>>>>
>>>>>
Re: JsonLayout breaks logging?
Posted by t spam <sp...@gmail.com>.
>
>
> Hi Naz and other solr-users (now with solr-user in to),
>
> Excuse my ignorance here (just getting started) but let's take the
> techproducts example. As you proposed I included the latest jackson-core
> and jackson-databind jars in the "solr install dir/lib/" directory:
>
> [tijmen@solr-1 solr-7.7.3]$ ls -la lib/
> total 1732
> drwxrwxr-x. 2 tijmen tijmen 72 Jul 25 08:06 .
> drwxr-xr-x. 10 tijmen tijmen 212 Jul 25 07:45 ..
> -rw-rw-r--. 1 tijmen tijmen 351575 Jul 25 07:48 jackson-core-2.11.1.jar
> -rw-rw-r--. 1 tijmen tijmen 1419800 Jul 25 07:48
> jackson-databind-2.11.1.jar
>
> I then added a lib directive to include the jackson jars
> in: example/techproducts/solr/techproducts/conf/solrconfig.xml
>
> <lib dir="${solr.install.dir:../../../..}/contrib/extraction/lib"
> regex=".*\.jar" />
> <lib dir="${solr.install.dir:../../../..}/dist/"
> regex="solr-cell-\d.*\.jar" />
>
> <lib dir="${solr.install.dir:../../../..}/contrib/clustering/lib/"
> regex=".*\.jar" />
> <lib dir="${solr.install.dir:../../../..}/dist/"
> regex="solr-clustering-\d.*\.jar" />
>
> <lib dir="${solr.install.dir:../../../..}/contrib/langid/lib/"
> regex=".*\.jar" />
> <lib dir="${solr.install.dir:../../../..}/dist/"
> regex="solr-langid-\d.*\.jar" />
>
> <lib dir="${solr.install.dir:../../../..}/dist/"
> regex="solr-ltr-\d.*\.jar" />
>
> <lib dir="${solr.install.dir:../../../..}/contrib/velocity/lib"
> regex=".*\.jar" />
> <lib dir="${solr.install.dir:../../../..}/dist/"
> regex="solr-velocity-\d.*\.jar" />
>
> <lib dir="${solr.install.dir:../../../..}/lib/" regex="jackson-.*\.jar"
> />
>
> I start solr using:
>
> [tijmen@solr-1 solr-7.7.3]$ ./bin/solr stop -e techproducts
>
> Unfortunately I get the same result. Solr starts but no logging.
>
> Whenever I remove the JsonLayout from the log4j2.xml it starts logging as
> expected.
>
> Thanks,
>
> Tijmen
>
> On Fri, Jul 24, 2020 at 6:48 PM Naz S <sn...@gmail.com> wrote:
>
>> Hi Tijmen,
>>
>> If you use maven, for example, you can add dependencies in pom.xml.
>>
>> For example,
>> <dependencies>
>> <dependency>
>> <groupId>com.fasterxml.jackson.core</groupId>
>> <artifactId>jackson-core</artifactId>
>> <version>2.11.1</version>
>> </dependency>
>> <dependency>
>> <groupId>com.fasterxml.jackson.core</groupId>
>> <artifactId>jackson-databind</artifactId>
>> <version>2.11.1</version>
>> </dependency>
>> </dependencies>
>>
>> On Fri, Jul 24, 2020 at 1:37 PM t spam <sp...@gmail.com> wrote:
>>
>>> Hi Naz,
>>>
>>> Could you give me some directions in where or how I should provide these
>>> dependencies? I can see these dependencies are already in various places by
>>> default:
>>>
>>> [tijmen@solr-1 solr-7.7.3]$ find . -name jackson*
>>> ./contrib/clustering/lib/jackson-annotations-2.9.8.jar
>>> ./contrib/clustering/lib/jackson-databind-2.9.8.jar
>>> ./contrib/prometheus-exporter/lib/jackson-annotations-2.9.8.jar
>>> ./contrib/prometheus-exporter/lib/jackson-core-2.9.8.jar
>>> ./contrib/prometheus-exporter/lib/jackson-databind-2.9.8.jar
>>> ./contrib/prometheus-exporter/lib/jackson-jq-0.0.8.jar
>>> ./licenses/jackson-annotations-2.9.8.jar.sha1
>>> ./licenses/jackson-annotations-LICENSE-ASL.txt
>>> ./licenses/jackson-annotations-NOTICE.txt
>>> ./licenses/jackson-core-2.9.8.jar.sha1
>>> ./licenses/jackson-core-LICENSE-ASL.txt
>>> ./licenses/jackson-core-NOTICE.txt
>>> ./licenses/jackson-core-asl-1.9.13.jar.sha1
>>> ./licenses/jackson-core-asl-LICENSE-ASL.txt
>>> ./licenses/jackson-core-asl-NOTICE.txt
>>> ./licenses/jackson-databind-2.9.8.jar.sha1
>>> ./licenses/jackson-databind-LICENSE-ASL.txt
>>> ./licenses/jackson-databind-NOTICE.txt
>>> ./licenses/jackson-dataformat-smile-2.9.8.jar.sha1
>>> ./licenses/jackson-dataformat-smile-LICENSE-ASL.txt
>>> ./licenses/jackson-dataformat-smile-NOTICE.txt
>>> ./licenses/jackson-jq-0.0.8.jar.sha1
>>> ./licenses/jackson-jq-LICENSE-ASL.txt
>>> ./licenses/jackson-jq-NOTICE.txt
>>> ./licenses/jackson-mapper-asl-1.9.13.jar.sha1
>>> ./licenses/jackson-mapper-asl-LICENSE-ASL.txt
>>> ./licenses/jackson-mapper-asl-NOTICE.txt
>>> ./server/solr-webapp/webapp/WEB-INF/lib/jackson-annotations-2.9.8.jar
>>> ./server/solr-webapp/webapp/WEB-INF/lib/jackson-core-2.9.8.jar
>>> ./server/solr-webapp/webapp/WEB-INF/lib/jackson-core-asl-1.9.13.jar
>>> ./server/solr-webapp/webapp/WEB-INF/lib/jackson-databind-2.9.8.jar
>>>
>>> ./server/solr-webapp/webapp/WEB-INF/lib/jackson-dataformat-smile-2.9.8.jar
>>> ./server/solr-webapp/webapp/WEB-INF/lib/jackson-mapper-asl-1.9.13.jar
>>>
>>> Thanks for your time.
>>>
>>> Tijmen
>>>
>>> On Fri, Jul 24, 2020 at 1:16 PM Naz S <sn...@gmail.com> wrote:
>>>
>>>>
>>>> You should explicitly provide the jackson dependencies: jackson-core,
>>>> jackson-databind and/or jackson-annotations.
>>>>
>>>> On Fri, Jul 24, 2020 at 8:24 AM t spam <sp...@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I'm having difficulty configuring JsonLayout for appenders. I have the
>>>>> following in my log4j2.xml:
>>>>>
>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>> <Configuration>
>>>>> <Appenders>
>>>>>
>>>>> <Console name="STDOUT" target="SYSTEM_OUT">
>>>>> <PatternLayout>
>>>>> <Pattern>
>>>>> %d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection}
>>>>> %X{shard}
>>>>> %X{replica} %X{core}] %c{1.} %m%n
>>>>> </Pattern>
>>>>> </PatternLayout>
>>>>> </Console>
>>>>>
>>>>> <RollingFile
>>>>> name="RollingFile"
>>>>> fileName="${sys:solr.log.dir}/solr.log"
>>>>> filePattern="${sys:solr.log.dir}/solr.log.%i" >
>>>>> <JsonLayout complete="false" compact="true" eventEol="true" />
>>>>> <Policies>
>>>>> <OnStartupTriggeringPolicy />
>>>>> <SizeBasedTriggeringPolicy size="32 MB"/>
>>>>> </Policies>
>>>>> <DefaultRolloverStrategy max="10"/>
>>>>> </RollingFile>
>>>>>
>>>>> <RollingFile
>>>>> name="SlowFile"
>>>>> fileName="${sys:solr.log.dir}/solr_slow_requests.log"
>>>>> filePattern="${sys:solr.log.dir}/solr_slow_requests.log.%i" >
>>>>> <PatternLayout>
>>>>> <Pattern>
>>>>> %d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection}
>>>>> %X{shard}
>>>>> %X{replica} %X{core}] %c{1.} %m%n
>>>>> </Pattern>
>>>>> </PatternLayout>
>>>>> <Policies>
>>>>> <OnStartupTriggeringPolicy />
>>>>> <SizeBasedTriggeringPolicy size="32 MB"/>
>>>>> </Policies>
>>>>> <DefaultRolloverStrategy max="10"/>
>>>>> </RollingFile>
>>>>>
>>>>> </Appenders>
>>>>> <Loggers>
>>>>> <Logger name="org.apache.hadoop" level="warn"/>
>>>>> <Logger name="org.apache.solr.update.LoggingInfoStream"
>>>>> level="off"/>
>>>>> <Logger name="org.apache.zookeeper" level="warn"/>
>>>>> <Logger name="org.apache.solr.core.SolrCore.SlowRequest"
>>>>> level="info"
>>>>> additivity="false">
>>>>> <AppenderRef ref="SlowFile"/>
>>>>> </Logger>
>>>>>
>>>>> <Root level="debug">
>>>>> <AppenderRef ref="RollingFile"/>
>>>>> <AppenderRef ref="STDOUT"/>
>>>>> </Root>
>>>>> </Loggers>
>>>>> </Configuration>
>>>>>
>>>>> For some reason it just stops logging anything. I only get the
>>>>> solr_gc.log
>>>>> and not the expected solr.log. I see an old thread mentioning the exact
>>>>> same issue (except that solr.log isn't even created in my case) but it
>>>>> wasn't resolved there. See here:
>>>>>
>>>>> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201809.mbox/%3cDB5PR0101MB211949C76C5C39472ECDFF87F4050@DB5PR0101MB2119.eurprd01.prod.exchangelabs.com%3e
>>>>>
>>>>> I see this behaviour with both SOLR 7 and 8.
>>>>>
>>>>> Am I missing something? Help is much appreciated.
>>>>>
>>>>> - Tijmen
>>>>>
>>>>> This was the original message:
>>>>>
>>>>> -----Ursprüngliche Nachricht-----
>>>>> Von: Shawn Heisey <ap...@elyograg.org>
>>>>> Gesendet: Freitag, 7. September 2018 17:58
>>>>> An: solr-user@lucene.apache.org
>>>>> Betreff: Re: Solr 7.4 and log4j2 JSONLayout
>>>>>
>>>>> On 9/6/2018 7:46 AM, Michael Aleythe, Sternwald wrote:
>>>>> > I'm trying to edit the log4j2 logging configuration for solr. The
>>>>> goal is to get a log
>>>>> file in json format. I configured the the JSONLayout for this purpose
>>>>> inside the rollingFile
>>>>> appender in the log4j2.xml. After this solr stops logging entirely.
>>>>> Solr.log file is empty.
>>>>> Only the solr-8983-console.log file contains 10 lines. The line
>>>>> "2018-09-06 13:22:25.378:INFO:oejs.Server:main:
>>>>> Started @2814ms" is the last one.
>>>>> > My first guess was that the jackson-core and jackson-databind jars
>>>>> were missing, but
>>>>> that did not fix the problem.
>>>>>
>>>>> As Varun said, jackson is already included in Solr. You won't need to
>>>>> add any jars for that.
>>>>>
>>>>> > Does anyone know where to find error-messages or exceptions that
>>>>> point me towards whats
>>>>> going wrong here?
>>>>>
>>>>> Start Solr in the foreground, with the -f option. This will cause
>>>>> Solr to log to the console.
>>>>> When Solr is started in the background, it suppresses console logging.
>>>>>
>>>>> I see that you have changed the logfile rollover size to 1MB. If your
>>>>> Solr server sees much
>>>>> traffic, this is going to result in an extremely fast rollover, which
>>>>> may mean that you lose
>>>>> access to logged events VERY quickly. This will especially be the
>>>>> case with JSON logging
>>>>> -- each event will take up a lot more space.
>>>>>
>>>>> Thanks,
>>>>> Shawn
>>>>>
>>>>
Re: JsonLayout breaks logging?
Posted by t spam <sp...@gmail.com>.
Hi Naz,
Could you give me some directions in where or how I should provide these
dependencies? I can see these dependencies are already in various places by
default:
[tijmen@solr-1 solr-7.7.3]$ find . -name jackson*
./contrib/clustering/lib/jackson-annotations-2.9.8.jar
./contrib/clustering/lib/jackson-databind-2.9.8.jar
./contrib/prometheus-exporter/lib/jackson-annotations-2.9.8.jar
./contrib/prometheus-exporter/lib/jackson-core-2.9.8.jar
./contrib/prometheus-exporter/lib/jackson-databind-2.9.8.jar
./contrib/prometheus-exporter/lib/jackson-jq-0.0.8.jar
./licenses/jackson-annotations-2.9.8.jar.sha1
./licenses/jackson-annotations-LICENSE-ASL.txt
./licenses/jackson-annotations-NOTICE.txt
./licenses/jackson-core-2.9.8.jar.sha1
./licenses/jackson-core-LICENSE-ASL.txt
./licenses/jackson-core-NOTICE.txt
./licenses/jackson-core-asl-1.9.13.jar.sha1
./licenses/jackson-core-asl-LICENSE-ASL.txt
./licenses/jackson-core-asl-NOTICE.txt
./licenses/jackson-databind-2.9.8.jar.sha1
./licenses/jackson-databind-LICENSE-ASL.txt
./licenses/jackson-databind-NOTICE.txt
./licenses/jackson-dataformat-smile-2.9.8.jar.sha1
./licenses/jackson-dataformat-smile-LICENSE-ASL.txt
./licenses/jackson-dataformat-smile-NOTICE.txt
./licenses/jackson-jq-0.0.8.jar.sha1
./licenses/jackson-jq-LICENSE-ASL.txt
./licenses/jackson-jq-NOTICE.txt
./licenses/jackson-mapper-asl-1.9.13.jar.sha1
./licenses/jackson-mapper-asl-LICENSE-ASL.txt
./licenses/jackson-mapper-asl-NOTICE.txt
./server/solr-webapp/webapp/WEB-INF/lib/jackson-annotations-2.9.8.jar
./server/solr-webapp/webapp/WEB-INF/lib/jackson-core-2.9.8.jar
./server/solr-webapp/webapp/WEB-INF/lib/jackson-core-asl-1.9.13.jar
./server/solr-webapp/webapp/WEB-INF/lib/jackson-databind-2.9.8.jar
./server/solr-webapp/webapp/WEB-INF/lib/jackson-dataformat-smile-2.9.8.jar
./server/solr-webapp/webapp/WEB-INF/lib/jackson-mapper-asl-1.9.13.jar
Thanks for your time.
Tijmen
On Fri, Jul 24, 2020 at 1:16 PM Naz S <sn...@gmail.com> wrote:
>
> You should explicitly provide the jackson dependencies: jackson-core,
> jackson-databind and/or jackson-annotations.
>
> On Fri, Jul 24, 2020 at 8:24 AM t spam <sp...@gmail.com> wrote:
>
>> Hi,
>>
>> I'm having difficulty configuring JsonLayout for appenders. I have the
>> following in my log4j2.xml:
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <Configuration>
>> <Appenders>
>>
>> <Console name="STDOUT" target="SYSTEM_OUT">
>> <PatternLayout>
>> <Pattern>
>> %d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard}
>> %X{replica} %X{core}] %c{1.} %m%n
>> </Pattern>
>> </PatternLayout>
>> </Console>
>>
>> <RollingFile
>> name="RollingFile"
>> fileName="${sys:solr.log.dir}/solr.log"
>> filePattern="${sys:solr.log.dir}/solr.log.%i" >
>> <JsonLayout complete="false" compact="true" eventEol="true" />
>> <Policies>
>> <OnStartupTriggeringPolicy />
>> <SizeBasedTriggeringPolicy size="32 MB"/>
>> </Policies>
>> <DefaultRolloverStrategy max="10"/>
>> </RollingFile>
>>
>> <RollingFile
>> name="SlowFile"
>> fileName="${sys:solr.log.dir}/solr_slow_requests.log"
>> filePattern="${sys:solr.log.dir}/solr_slow_requests.log.%i" >
>> <PatternLayout>
>> <Pattern>
>> %d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard}
>> %X{replica} %X{core}] %c{1.} %m%n
>> </Pattern>
>> </PatternLayout>
>> <Policies>
>> <OnStartupTriggeringPolicy />
>> <SizeBasedTriggeringPolicy size="32 MB"/>
>> </Policies>
>> <DefaultRolloverStrategy max="10"/>
>> </RollingFile>
>>
>> </Appenders>
>> <Loggers>
>> <Logger name="org.apache.hadoop" level="warn"/>
>> <Logger name="org.apache.solr.update.LoggingInfoStream" level="off"/>
>> <Logger name="org.apache.zookeeper" level="warn"/>
>> <Logger name="org.apache.solr.core.SolrCore.SlowRequest" level="info"
>> additivity="false">
>> <AppenderRef ref="SlowFile"/>
>> </Logger>
>>
>> <Root level="debug">
>> <AppenderRef ref="RollingFile"/>
>> <AppenderRef ref="STDOUT"/>
>> </Root>
>> </Loggers>
>> </Configuration>
>>
>> For some reason it just stops logging anything. I only get the solr_gc.log
>> and not the expected solr.log. I see an old thread mentioning the exact
>> same issue (except that solr.log isn't even created in my case) but it
>> wasn't resolved there. See here:
>>
>> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201809.mbox/%3cDB5PR0101MB211949C76C5C39472ECDFF87F4050@DB5PR0101MB2119.eurprd01.prod.exchangelabs.com%3e
>>
>> I see this behaviour with both SOLR 7 and 8.
>>
>> Am I missing something? Help is much appreciated.
>>
>> - Tijmen
>>
>> This was the original message:
>>
>> -----Ursprüngliche Nachricht-----
>> Von: Shawn Heisey <ap...@elyograg.org>
>> Gesendet: Freitag, 7. September 2018 17:58
>> An: solr-user@lucene.apache.org
>> Betreff: Re: Solr 7.4 and log4j2 JSONLayout
>>
>> On 9/6/2018 7:46 AM, Michael Aleythe, Sternwald wrote:
>> > I'm trying to edit the log4j2 logging configuration for solr. The goal
>> is to get a log
>> file in json format. I configured the the JSONLayout for this purpose
>> inside the rollingFile
>> appender in the log4j2.xml. After this solr stops logging entirely.
>> Solr.log file is empty.
>> Only the solr-8983-console.log file contains 10 lines. The line
>> "2018-09-06 13:22:25.378:INFO:oejs.Server:main:
>> Started @2814ms" is the last one.
>> > My first guess was that the jackson-core and jackson-databind jars were
>> missing, but
>> that did not fix the problem.
>>
>> As Varun said, jackson is already included in Solr. You won't need to
>> add any jars for that.
>>
>> > Does anyone know where to find error-messages or exceptions that point
>> me towards whats
>> going wrong here?
>>
>> Start Solr in the foreground, with the -f option. This will cause
>> Solr to log to the console.
>> When Solr is started in the background, it suppresses console logging.
>>
>> I see that you have changed the logfile rollover size to 1MB. If your
>> Solr server sees much
>> traffic, this is going to result in an extremely fast rollover, which
>> may mean that you lose
>> access to logged events VERY quickly. This will especially be the
>> case with JSON logging
>> -- each event will take up a lot more space.
>>
>> Thanks,
>> Shawn
>>
>
Re: JsonLayout breaks logging?
Posted by Naz S <sn...@gmail.com>.
You should explicitly provide the jackson dependencies: jackson-core,
jackson-databind and/or jackson-annotations.
On Fri, Jul 24, 2020 at 8:24 AM t spam <sp...@gmail.com> wrote:
> Hi,
>
> I'm having difficulty configuring JsonLayout for appenders. I have the
> following in my log4j2.xml:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <Configuration>
> <Appenders>
>
> <Console name="STDOUT" target="SYSTEM_OUT">
> <PatternLayout>
> <Pattern>
> %d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard}
> %X{replica} %X{core}] %c{1.} %m%n
> </Pattern>
> </PatternLayout>
> </Console>
>
> <RollingFile
> name="RollingFile"
> fileName="${sys:solr.log.dir}/solr.log"
> filePattern="${sys:solr.log.dir}/solr.log.%i" >
> <JsonLayout complete="false" compact="true" eventEol="true" />
> <Policies>
> <OnStartupTriggeringPolicy />
> <SizeBasedTriggeringPolicy size="32 MB"/>
> </Policies>
> <DefaultRolloverStrategy max="10"/>
> </RollingFile>
>
> <RollingFile
> name="SlowFile"
> fileName="${sys:solr.log.dir}/solr_slow_requests.log"
> filePattern="${sys:solr.log.dir}/solr_slow_requests.log.%i" >
> <PatternLayout>
> <Pattern>
> %d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard}
> %X{replica} %X{core}] %c{1.} %m%n
> </Pattern>
> </PatternLayout>
> <Policies>
> <OnStartupTriggeringPolicy />
> <SizeBasedTriggeringPolicy size="32 MB"/>
> </Policies>
> <DefaultRolloverStrategy max="10"/>
> </RollingFile>
>
> </Appenders>
> <Loggers>
> <Logger name="org.apache.hadoop" level="warn"/>
> <Logger name="org.apache.solr.update.LoggingInfoStream" level="off"/>
> <Logger name="org.apache.zookeeper" level="warn"/>
> <Logger name="org.apache.solr.core.SolrCore.SlowRequest" level="info"
> additivity="false">
> <AppenderRef ref="SlowFile"/>
> </Logger>
>
> <Root level="debug">
> <AppenderRef ref="RollingFile"/>
> <AppenderRef ref="STDOUT"/>
> </Root>
> </Loggers>
> </Configuration>
>
> For some reason it just stops logging anything. I only get the solr_gc.log
> and not the expected solr.log. I see an old thread mentioning the exact
> same issue (except that solr.log isn't even created in my case) but it
> wasn't resolved there. See here:
>
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201809.mbox/%3cDB5PR0101MB211949C76C5C39472ECDFF87F4050@DB5PR0101MB2119.eurprd01.prod.exchangelabs.com%3e
>
> I see this behaviour with both SOLR 7 and 8.
>
> Am I missing something? Help is much appreciated.
>
> - Tijmen
>
> This was the original message:
>
> -----Ursprüngliche Nachricht-----
> Von: Shawn Heisey <ap...@elyograg.org>
> Gesendet: Freitag, 7. September 2018 17:58
> An: solr-user@lucene.apache.org
> Betreff: Re: Solr 7.4 and log4j2 JSONLayout
>
> On 9/6/2018 7:46 AM, Michael Aleythe, Sternwald wrote:
> > I'm trying to edit the log4j2 logging configuration for solr. The goal
> is to get a log
> file in json format. I configured the the JSONLayout for this purpose
> inside the rollingFile
> appender in the log4j2.xml. After this solr stops logging entirely.
> Solr.log file is empty.
> Only the solr-8983-console.log file contains 10 lines. The line
> "2018-09-06 13:22:25.378:INFO:oejs.Server:main:
> Started @2814ms" is the last one.
> > My first guess was that the jackson-core and jackson-databind jars were
> missing, but
> that did not fix the problem.
>
> As Varun said, jackson is already included in Solr. You won't need to
> add any jars for that.
>
> > Does anyone know where to find error-messages or exceptions that point
> me towards whats
> going wrong here?
>
> Start Solr in the foreground, with the -f option. This will cause
> Solr to log to the console.
> When Solr is started in the background, it suppresses console logging.
>
> I see that you have changed the logfile rollover size to 1MB. If your
> Solr server sees much
> traffic, this is going to result in an extremely fast rollover, which
> may mean that you lose
> access to logged events VERY quickly. This will especially be the
> case with JSON logging
> -- each event will take up a lot more space.
>
> Thanks,
> Shawn
>