[jira] [Comment Edited] (SOLR-3284) StreamingUpdateSolrServer swallows exceptions

["Mike Sokolov (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292507#comment-13292507 ] 

Mike Sokolov edited comment on SOLR-3284 at 6/10/12 12:14 PM:
--------------------------------------------------------------

Another approach we have been using in a similar parallel writing situation is to have the thread pool maintain a fixed-size list of exceptions.  Whenever a worker throws one, it gets put on the list.  Then at certain barrier conditions (flush, exception list full), the exceptions are collected together and re-thrown using an umbrella exception that wraps them.  At the same time, all worker threads are terminated. You do need flush(), or can you rely on the user calling commit() (and flush() internally then)?

This enables writing to continue even in the face of errors, but the errors do get reported eventually.  This makes the system to be more robust in the face of transient failure conditions. 
                
      was (Author: sokolov):
    Another approach we have been using in a similar parallel writing situation is to have the thread pool maintain a fixed-size list of exceptions.  Whenever a worker throws one, it gets put on the list.  Then at certain barrier conditions (flush, exception list full), the exceptions are collected together and re-thrown using an umbrella exception that wraps them.  You do need flush(), or can you rely on the user calling commit() (and flush() internally then)?

This enables writing to continue even in the face of errors, but the errors do get reported eventually.  This makes the system to be more robust in the face of transient failure conditions. 
                  
> StreamingUpdateSolrServer swallows exceptions
> ---------------------------------------------
>
>                 Key: SOLR-3284
>                 URL: https://issues.apache.org/jira/browse/SOLR-3284
>             Project: Solr
>          Issue Type: Improvement
>          Components: clients - java
>    Affects Versions: 3.5, 4.0
>            Reporter: Shawn Heisey
>         Attachments: SOLR-3284.patch
>
>
> StreamingUpdateSolrServer eats exceptions thrown by lower level code, such as HttpClient, when doing adds.  It may happen with other methods, though I know that query and deleteByQuery will throw exceptions.  I believe that this is a result of the queue/Runner design.  That's what makes SUSS perform better, but it means you sacrifice the ability to programmatically determine that there was a problem with your update.  All errors are logged via slf4j, but that's not terribly helpful except with determining what went wrong after the fact.
> When using CommonsHttpSolrServer, I've been able to rely on getting an exception thrown by pretty much any error, letting me use try/catch to detect problems.
> There's probably enough dependent code out there that it would not be a good idea to change the design of SUSS, unless there were alternate constructors or additional methods available to configure new/old behavior.  Fixing this is probably not trivial, so it's probably a better idea to come up with a new server object based on CHSS.  This is outside my current skillset.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org