How for distributed search only log collective search response

[Koorosh Vakhshoori <Ko...@synopsys.com>]

  In my use case, I have a number of shards where a query would run as distributed search.  I am not using Solr Cloud, I have just a Solr server. Now, when the search runs, I see one entry for each shard query as well as the finally collective search query response. As the results, I am ending up with a very noisy log. I don't care about individual shard queries, just the aggregate result. Is there a way to configure Solr so it would only log the final collective response? I believe this use case also applies to Solr Cloud.

  Looking at the Solr code, class SolrCore, I see the following lines is performing the logging:

    if (rsp.getToLog().size() > 0) {
      if (requestLog.isInfoEnabled()) {
        requestLog.info(rsp.getToLogAsString(logid));
      }

  I was thinking of adding a flag that filter the distributed logs by looking at the 'params' and check for 'isShard=true' and if present don't log it.

  Any suggestion or comment? Is this something people would be interested in?

Regards,

Koorosh

RE: How for distributed search only log collective search response

[Koorosh Vakhshoori <Ko...@synopsys.com>]

It turns out there is a better way to do this. It does not require code change in Solr, if you are using log4j. However, you need to migrate to log4j.xml file format. The solution is to use the filter feature. Here is what my console appender looks like with the filter:

    <appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%-5p - %C - %d{yyyy-MM-dd HH:mm:ss.SSS}: %m\n"/>
        </layout>
        <filter class="org.apache.log4j.varia.StringMatchFilter">
            <param name="StringToMatch" value="isShard=" />
            <param name="acceptOnMatch" value="false"/>
        </filter>
    </appender>

Regards,

Koorosh