You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Bill Au (JIRA)" <ji...@apache.org> on 2007/09/10 19:28:29 UTC
[jira] Created: (SOLR-352) UpdateRequest is duplicating commit and
optimize requests
UpdateRequest is duplicating commit and optimize requests
---------------------------------------------------------
Key: SOLR-352
URL: https://issues.apache.org/jira/browse/SOLR-352
Project: Solr
Issue Type: Bug
Components: clients - java
Affects Versions: 1.3
Reporter: Bill Au
Assignee: Bill Au
Priority: Minor
UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-352) UpdateRequest is duplicating commit
and optimize requests
Posted by "Bill Au (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526497 ]
Bill Au commented on SOLR-352:
------------------------------
I have opened SOLR-353 for setting default overwrite=true in UpdateRequest.
> UpdateRequest is duplicating commit and optimize requests
> ---------------------------------------------------------
>
> Key: SOLR-352
> URL: https://issues.apache.org/jira/browse/SOLR-352
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Bill Au
> Assignee: Bill Au
> Priority: Minor
> Fix For: 1.3
>
> Attachments: solr-352.patch, solr-352.patch
>
>
> UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-352) UpdateRequest is duplicating commit and
optimize requests
Posted by "Bill Au (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Au resolved SOLR-352.
--------------------------
Resolution: Fixed
Fix Version/s: 1.3
Patch commited (Committed revision 574608).
> UpdateRequest is duplicating commit and optimize requests
> ---------------------------------------------------------
>
> Key: SOLR-352
> URL: https://issues.apache.org/jira/browse/SOLR-352
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Bill Au
> Assignee: Bill Au
> Priority: Minor
> Fix For: 1.3
>
> Attachments: solr-352.patch, solr-352.patch
>
>
> UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-352) UpdateRequest is duplicating commit and
optimize requests
Posted by "Bill Au (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Au updated SOLR-352:
-------------------------
Attachment: solr-352.patch
Patch to remove commit/optimize query args since the request already contain an commit/optimize XML message in the POST body.
> UpdateRequest is duplicating commit and optimize requests
> ---------------------------------------------------------
>
> Key: SOLR-352
> URL: https://issues.apache.org/jira/browse/SOLR-352
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Bill Au
> Assignee: Bill Au
> Priority: Minor
> Attachments: solr-352.patch
>
>
> UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-352) UpdateRequest is duplicating commit
and optimize requests
Posted by "Bill Au (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526463 ]
Bill Au commented on SOLR-352:
------------------------------
Thanks Ryan. I got it to work but I have to add
req.setOverwrite(true);
Otherwise, a got an exception from Solr:
org.apache.solr.common.SolrException: unsupported param combo:add:,allowDups=false,overwritePending=false,overwriteCommitted=false
unsupported param combo:add:,allowDups=false,overwritePending=false,overwriteCommitted=false
request: http://cn-ewr1-dev40-pi2.cnet.com:7905/solr/update?commit=true&waitFlush=false&waitSearcher=false&wt=xml&version=2.2
So it looks like the combination of default values of allowDups, overwriteCommitted, and overwritePending (all false) in UpdateRequest is not supported by Solr. Should we change the default to something that is supported (setting overwirte to true)? I can open a separate bug and take care of that.
> UpdateRequest is duplicating commit and optimize requests
> ---------------------------------------------------------
>
> Key: SOLR-352
> URL: https://issues.apache.org/jira/browse/SOLR-352
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Bill Au
> Assignee: Bill Au
> Priority: Minor
> Attachments: solr-352.patch, solr-352.patch
>
>
> UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-352) UpdateRequest is duplicating commit
and optimize requests
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526284 ]
Ryan McKinley commented on SOLR-352:
------------------------------------
Note, in solr, this is starting the commit with a request parameter, not <commit/>
/update?commit=true with post data:
<add>
...
</add>
> UpdateRequest is duplicating commit and optimize requests
> ---------------------------------------------------------
>
> Key: SOLR-352
> URL: https://issues.apache.org/jira/browse/SOLR-352
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Bill Au
> Assignee: Bill Au
> Priority: Minor
> Attachments: solr-352.patch, solr-352.patch
>
>
> UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-352) UpdateRequest is duplicating commit and
optimize requests
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryan McKinley updated SOLR-352:
-------------------------------
Attachment: solr-352.patch
Dooh, thanks Bill
This patch is slightly different. Rather then getting rid of the parameters for commit or optimize, it always uses the parameter version and does not send xml.
This way we can continue to support calling <commit> immediatly after <add> (without sending a second request)
ryan
> UpdateRequest is duplicating commit and optimize requests
> ---------------------------------------------------------
>
> Key: SOLR-352
> URL: https://issues.apache.org/jira/browse/SOLR-352
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Bill Au
> Assignee: Bill Au
> Priority: Minor
> Attachments: solr-352.patch, solr-352.patch
>
>
> UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-352) UpdateRequest is duplicating commit
and optimize requests
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526469 ]
Ryan McKinley commented on SOLR-352:
------------------------------------
yes, lets make the default overwrite=true
I removed the overwritePending/overwriteCommited options so that adding SOLR-60 is a non issue for solrj.
Can you open a new issue?
> UpdateRequest is duplicating commit and optimize requests
> ---------------------------------------------------------
>
> Key: SOLR-352
> URL: https://issues.apache.org/jira/browse/SOLR-352
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Bill Au
> Assignee: Bill Au
> Priority: Minor
> Fix For: 1.3
>
> Attachments: solr-352.patch, solr-352.patch
>
>
> UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-352) UpdateRequest is duplicating commit
and optimize requests
Posted by "Bill Au (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526228 ]
Bill Au commented on SOLR-352:
------------------------------
My bad. Somehow I didn't pick up the patch to ClientUtils.java. After taking care of that, both commit and optimize works as expected and only once. I will commit the latest patch tomorrow morning (so that other folks will have a chance to review/comment).
> UpdateRequest is duplicating commit and optimize requests
> ---------------------------------------------------------
>
> Key: SOLR-352
> URL: https://issues.apache.org/jira/browse/SOLR-352
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Bill Au
> Assignee: Bill Au
> Priority: Minor
> Attachments: solr-352.patch, solr-352.patch
>
>
> UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-352) UpdateRequest is duplicating commit
and optimize requests
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526224 ]
Ryan McKinley commented on SOLR-352:
------------------------------------
did you apply the whole patch?
It includes a null check at line 59 of ClientUtils.java.
//--------------------------------------------------------------------------
Index: client/java/solrj/src/org/apache/solr/client/solrj/util/ClientUtils.java
===================================================================
--- client/java/solrj/src/org/apache/solr/client/solrj/util/ClientUtils.java (revision 574315)
+++ client/java/solrj/src/org/apache/solr/client/solrj/util/ClientUtils.java (working copy)
@@ -56,9 +56,12 @@
*/
public static Collection<ContentStream> toContentStreams( final String str, final String contentType )
{
+ if( str == null )
+ return null;
+
+ ArrayList<ContentStream> streams = new ArrayList<ContentStream>( 1 );
ContentStreamBase ccc = new ContentStreamBase.StringStream( str );
ccc.setContentType( contentType );
- ArrayList<ContentStream> streams = new ArrayList<ContentStream>();
streams.add( ccc );
return streams;
}
> UpdateRequest is duplicating commit and optimize requests
> ---------------------------------------------------------
>
> Key: SOLR-352
> URL: https://issues.apache.org/jira/browse/SOLR-352
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Bill Au
> Assignee: Bill Au
> Priority: Minor
> Attachments: solr-352.patch, solr-352.patch
>
>
> UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-352) UpdateRequest is duplicating commit
and optimize requests
Posted by "Bill Au (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526271 ]
Bill Au commented on SOLR-352:
------------------------------
Ryan, how does one call <commit> immediately after <add> with a single request?
> UpdateRequest is duplicating commit and optimize requests
> ---------------------------------------------------------
>
> Key: SOLR-352
> URL: https://issues.apache.org/jira/browse/SOLR-352
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Bill Au
> Assignee: Bill Au
> Priority: Minor
> Attachments: solr-352.patch, solr-352.patch
>
>
> UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (SOLR-352) UpdateRequest is duplicating commit
and optimize requests
Posted by Ryan McKinley <ry...@gmail.com>.
did you apply the whole patch?
It also added a null check at line 59 of ClientUtils.java.
//--------------------------------------------------------------------------
Index:
client/java/solrj/src/org/apache/solr/client/solrj/util/ClientUtils.java
===================================================================
---
client/java/solrj/src/org/apache/solr/client/solrj/util/ClientUtils.java
(revision 574315)
+++
client/java/solrj/src/org/apache/solr/client/solrj/util/ClientUtils.java
(working copy)
@@ -56,9 +56,12 @@
*/
public static Collection<ContentStream> toContentStreams( final
String str, final String contentType )
{
+ if( str == null )
+ return null;
+
+ ArrayList<ContentStream> streams = new ArrayList<ContentStream>( 1 );
ContentStreamBase ccc = new ContentStreamBase.StringStream( str );
ccc.setContentType( contentType );
- ArrayList<ContentStream> streams = new ArrayList<ContentStream>();
streams.add( ccc );
return streams;
}
Bill Au (JIRA) wrote:
> [ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526213 ]
>
> Bill Au commented on SOLR-352:
> ------------------------------
>
> Ryan, I am gettign a NullPointerException when I tried you patch:
>
> java.lang.NullPointerException
> at org.apache.solr.common.util.ContentStreamBase$StringStream.<init>(ContentStreamBase.java:137)
> at org.apache.solr.client.solrj.util.ClientUtils.toContentStreams(ClientUtils.java:59)
> at org.apache.solr.client.solrj.request.UpdateRequest.getContentStreams(UpdateRequest.java:134)
> at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:113)
> at org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequest.java:199)
> at org.apache.solr.client.solrj.impl.BaseSolrServer.commit(BaseSolrServer.java:79)
> at org.apache.solr.client.solrj.impl.BaseSolrServer.commit(BaseSolrServer.java:68)
> at _jsp._solrjCommit__jsp._jspService(solrjCommit.jsp:12)
> at com.caucho.jsp.JavaPage.service(JavaPage.java:60)
> at com.caucho.jsp.Page.pageservice(Page.java:570)
> at com.caucho.server.dispatch.PageFilterChain.doFilter(PageFilterChain.java:179)
> at com.caucho.server.cache.CacheFilterChain.doFilter(CacheFilterChain.java:209)
> at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:173)
> at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:229)
> at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:274)
> at com.caucho.server.port.TcpConnection.run(TcpConnection.java:511)
> at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:520)
> at com.caucho.util.ThreadPool.run(ThreadPool.java:442)
> at java.lang.Thread.run(Thread.java:595)
>
>> UpdateRequest is duplicating commit and optimize requests
>> ---------------------------------------------------------
>>
>> Key: SOLR-352
>> URL: https://issues.apache.org/jira/browse/SOLR-352
>> Project: Solr
>> Issue Type: Bug
>> Components: clients - java
>> Affects Versions: 1.3
>> Reporter: Bill Au
>> Assignee: Bill Au
>> Priority: Minor
>> Attachments: solr-352.patch, solr-352.patch
>>
>>
>> UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args.
>
[jira] Commented: (SOLR-352) UpdateRequest is duplicating commit
and optimize requests
Posted by "Bill Au (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526213 ]
Bill Au commented on SOLR-352:
------------------------------
Ryan, I am gettign a NullPointerException when I tried you patch:
java.lang.NullPointerException
at org.apache.solr.common.util.ContentStreamBase$StringStream.<init>(ContentStreamBase.java:137)
at org.apache.solr.client.solrj.util.ClientUtils.toContentStreams(ClientUtils.java:59)
at org.apache.solr.client.solrj.request.UpdateRequest.getContentStreams(UpdateRequest.java:134)
at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:113)
at org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequest.java:199)
at org.apache.solr.client.solrj.impl.BaseSolrServer.commit(BaseSolrServer.java:79)
at org.apache.solr.client.solrj.impl.BaseSolrServer.commit(BaseSolrServer.java:68)
at _jsp._solrjCommit__jsp._jspService(solrjCommit.jsp:12)
at com.caucho.jsp.JavaPage.service(JavaPage.java:60)
at com.caucho.jsp.Page.pageservice(Page.java:570)
at com.caucho.server.dispatch.PageFilterChain.doFilter(PageFilterChain.java:179)
at com.caucho.server.cache.CacheFilterChain.doFilter(CacheFilterChain.java:209)
at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:173)
at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:229)
at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:274)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:511)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:520)
at com.caucho.util.ThreadPool.run(ThreadPool.java:442)
at java.lang.Thread.run(Thread.java:595)
> UpdateRequest is duplicating commit and optimize requests
> ---------------------------------------------------------
>
> Key: SOLR-352
> URL: https://issues.apache.org/jira/browse/SOLR-352
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Bill Au
> Assignee: Bill Au
> Priority: Minor
> Attachments: solr-352.patch, solr-352.patch
>
>
> UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-352) UpdateRequest is duplicating commit
and optimize requests
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526283 ]
Ryan McKinley commented on SOLR-352:
------------------------------------
I just added some docs to:
http://wiki.apache.org/solr/Solrj
Essentially, you can work with the UpdateRequest directly and use options that are not available in the SolrServer interface.
SolrServer server = ...
UpdateRequest req = new UpdateRequest();
req.setAction( UpdateRequest.ACTION.COMMIT, false, false );
req.add( docs );
UpdateResponse rsp = req.process( server );
> UpdateRequest is duplicating commit and optimize requests
> ---------------------------------------------------------
>
> Key: SOLR-352
> URL: https://issues.apache.org/jira/browse/SOLR-352
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Bill Au
> Assignee: Bill Au
> Priority: Minor
> Attachments: solr-352.patch, solr-352.patch
>
>
> UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.