You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Jayson Minard (JIRA)" <ji...@apache.org> on 2009/05/13 00:01:46 UTC
[jira] Created: (SOLR-1162) SolrJ does not maintain order of
operations when using an UpdateRequest object to send them in bulk
SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
---------------------------------------------------------------------------------------------------
Key: SOLR-1162
URL: https://issues.apache.org/jira/browse/SOLR-1162
Project: Solr
Issue Type: Bug
Components: clients - java
Affects Versions: 1.3
Reporter: Jayson Minard
In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ does not maintain order of
operations when using an UpdateRequest object to send them in bulk
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708790#action_12708790 ]
Jayson Minard commented on SOLR-1162:
-------------------------------------
Current patch (12-May-2009 11:30PM) is not tested with live server, only against the unit test TestUpdateRequestCodec.
> SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
> ---------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ does not maintain order of
operations when using an UpdateRequest object to send them in bulk
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708785#action_12708785 ]
Noble Paul commented on SOLR-1162:
----------------------------------
isn't it better to use multiple UpdateRequest to go in one Request and you can have a method which takes in an array of UpdateRequests.
> SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
> ---------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ UpdateRequest does not maintain
order of operations when sending mixed types of changes (updates, delete
id, delete query, update iterator)
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709390#action_12709390 ]
Jayson Minard commented on SOLR-1162:
-------------------------------------
UpdateRequest is used on both ends of the stream potentially so we would want to break out the server pieces to not use the same un-ordered object for binary codec. I could go either way, just never saw a need for unordered to remain but then again there is a whole community out there that might!
> SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-1162) SolrJ UpdateRequest does not maintain
order of operations when sending mixed types of changes (updates, delete
id, delete query, update iterator)
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Noble Paul reassigned SOLR-1162:
--------------------------------
Assignee: Noble Paul
> SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Assignee: Noble Paul
> Fix For: 1.5
>
> Attachments: Solr-1162.patch, Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ does not maintain order of
operations when using an UpdateRequest object to send them in bulk
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708792#action_12708792 ]
Jayson Minard commented on SOLR-1162:
-------------------------------------
Also (for Noble Paul)...
UpdateRequest is the invoker of itself via the process method, there is nothing to which you can pass it as a list.
UpdateRequest req = new UpdateRequest();
... add things
req.process(solrServer);
> SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
> ---------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ UpdateRequest does not maintain
order of operations when sending mixed types of changes (updates, delete
id, delete query, update iterator)
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709503#action_12709503 ]
Jayson Minard commented on SOLR-1162:
-------------------------------------
Sounds good. I'll watch SOLR-1164 for updates.
> SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch, Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1162) SolrJ does not maintain order of
operations when using an UpdateRequest object to send them in bulk
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1162:
--------------------------------
Attachment: Solr-1162.patch
Updated patch to fix commitWithin parameter in UpdateRequest and failing unit test.
> SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
> ---------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ UpdateRequest does not maintain
order of operations when sending mixed types of changes (updates, delete
id, delete query, update iterator)
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709465#action_12709465 ]
Noble Paul commented on SOLR-1162:
----------------------------------
I shall update SOLR-1164 and you can take it over from there
> SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch, Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ does not maintain order of
operations when using an UpdateRequest object to send them in bulk
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708830#action_12708830 ]
Noble Paul commented on SOLR-1162:
----------------------------------
I took a look at the patch . Looks fine , there are a lot of changes.
But I am still not convinced of the usecase , considering that you can achieve the same with multiple requests easily. Could you help with a usecase
> SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
> ---------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ does not maintain order of
operations when using an UpdateRequest object to send them in bulk
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708930#action_12708930 ]
Jayson Minard commented on SOLR-1162:
-------------------------------------
Multiple requests are less efficient than sending large batches together.
To be the most efficient with large requests, every user of SolrJ UpdateRequest would need to write the same logic... Place adds into UpdateRequest until you hit the first non-add, then send the UpdateRequest and start writing your deletes until you hit a non-delete, then flush the UpdateRequest and keep adding your new transaction type until you hit the first ... In that case they should avoid using UpdateRequest altogether as calling the SolrServer directly is just as "easy." If we are going to batch on their behalf why wouldn't we do it correctly and be predictable with our ordering. I'm sure if JDBC batches did not maintain order, there would be havoc to pay...
Besides that, it isn't clear to users of UpdateRequest as to the order of operations, so someone doing an Add doc 1, Delete doc 1, Add doc 1 may not end up with the expected outcome. It turns into Add doc 1, Add doc 1, Delete doc1 when streaming and similary for XML version of the transaction. If I did a Delete Query *:* then Add doc1, Add doc 2 I end up with no docs as the delete query comes last, but I (the user) does not know that.
I've written code to work around UpdateRequest ordering and I usually end up only using it for commitWithin or having a commit tacked on the end of the request due to the above issues.
> SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
> ---------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1162) SolrJ UpdateRequest does not maintain
order of operations when sending mixed types of changes (updates, delete
id, delete query, update iterator)
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar updated SOLR-1162:
----------------------------------------
Fix Version/s: 1.5
> SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Fix For: 1.5
>
> Attachments: Solr-1162.patch, Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ UpdateRequest does not maintain
order of operations when sending mixed types of changes (updates, delete
id, delete query, update iterator)
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709446#action_12709446 ]
Jayson Minard commented on SOLR-1162:
-------------------------------------
Sounds good, didn't like that returned object in the old method anyway...
I'll update the unmarshal and unit test that used that object to ensure it was accurate as well.
What do you want to do about the deserialization? I think it was unsafe before in that it looked up the lists by names and made assumptions about the parms being first.
> SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ does not maintain order of
operations when using an UpdateRequest object to send them in bulk
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708789#action_12708789 ]
Jayson Minard commented on SOLR-1162:
-------------------------------------
In response to Noble Paul:
An UpdateRequest currently contains: a list of adds, a single add via iterator, a list of delete by ids, a list of delete queries. So you can pile them all into the same object currently and then have no idea what order they will actually execute. If you want to stream adds, you cannot intermix deletes or other actions in the same stream. So if UpdateRequest is going to allow a set of different actions it should at least maintain the order in which they were added and execute them similarly.
UpdateRequest is the current batching model, so it should be correct.
> SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
> ---------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ UpdateRequest does not maintain
order of operations when sending mixed types of changes (updates, delete
id, delete query, update iterator)
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776829#action_12776829 ]
Jayson Minard commented on SOLR-1162:
-------------------------------------
Yikes, set to 1.5 -- I better revamp this to current code.
> SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Fix For: 1.5
>
> Attachments: Solr-1162.patch, Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1162) SolrJ UpdateRequest does not maintain
order of operations when sending mixed types of changes (updates, delete
id, delete query, update iterator)
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1162:
--------------------------------
Attachment: Solr-1162.patch
Updated patch to remove return value from the unmarshall call as suggested above.
> SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch, Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ UpdateRequest does not maintain
order of operations when sending mixed types of changes (updates, delete
id, delete query, update iterator)
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709392#action_12709392 ]
Noble Paul commented on SOLR-1162:
----------------------------------
we need a revamp of the JavaBinUpdateRequestCodec .
The way the deserialization is done is not safe .
{code:java}
if (name.equals("params")) {
NamedList parmsAsList = (NamedList)val;
ModifiableSolrParams parms = namedListToSolrParams(parmsAsList);
updateRequest.setParams(parms);
}
else if (name.equals("delById")) {
List<String> delIds = (List<String>)val;
streamer.deleteById(delIds);
}
else if (name.equals("delByQ")) {
String delQuery = (String)val;
streamer.deleteByQuery(delQuery);
}
{code}
may break if there are other objects with these names . I can provide a true streaming unmarshaller for JavabinCodec
Let us change the method
{code:java}
public UpdateRequest unmarshal(InputStream is, final StreamingDocumentHandler handler)
{code}
to
{code:java}
public void unmarshal(InputStream is, final StreamingDocumentHandler handler)
{code}
and make the handler mandatory. so it does not have to worry about the type of Object
> SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1162) SolrJ does not maintain order of
operations when using an UpdateRequest object to send them in bulk
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1162:
--------------------------------
Issue Type: Improvement (was: Bug)
> SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
> ---------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1162) SolrJ UpdateRequest does not maintain
order of operations when sending mixed types of changes (updates, delete
id, delete query, update iterator)
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1162:
--------------------------------
Description:
In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
was:In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
Summary: SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator) (was: SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk)
> SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ does not maintain order of
operations when using an UpdateRequest object to send them in bulk
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709037#action_12709037 ]
Jayson Minard commented on SOLR-1162:
-------------------------------------
Outside of UpdateRequest if you hand construct a binary codec stream to send to Solr (without this patch) your order of actions would not be maintained within the stream. So the binary streaming update handler is "broken" in this regard as well.
So this patch actually resolves two issue: UpdateRequest does not serialize into xml or binary stream the actions in-order. Nor does BinaryUpdateRequestHandler nor did the underlaying JavaBinUpdateRequestCodec so these all become unusable for mixed style actions since none can maintain order. UpdateRequest is just one of the clients to that server-side code that also has the same issue.
> SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
> ---------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1162) SolrJ does not maintain order of
operations when using an UpdateRequest object to send them in bulk
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1162:
--------------------------------
Attachment: Solr-1162.patch
First round of patch, changed the UpdateRequest to maintain one list of different types of request items, then changed XML serialization to keep the same order, and the same for binary codec. Then updated the streaming side of the binary update handler to stream all request types (not just doc adds) and updated the unit test to verify order is maintained both when streaming or not streaming using the binary codec.
> SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
> ---------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ UpdateRequest does not maintain
order of operations when sending mixed types of changes (updates, delete
id, delete query, update iterator)
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709279#action_12709279 ]
Noble Paul commented on SOLR-1162:
----------------------------------
Jayson, I see your point, Instead of changing the current UpdateRequest , isn't it better to add a new one OrderedUpdateRequest for this purpose?
> SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ does not maintain order of
operations when using an UpdateRequest object to send them in bulk
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708655#action_12708655 ]
Jayson Minard commented on SOLR-1162:
-------------------------------------
Working on this patch now...
> SolrJ does not maintain order of operations when using an UpdateRequest object to send them in bulk
> ---------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1162) SolrJ UpdateRequest does not maintain
order of operations when sending mixed types of changes (updates, delete
id, delete query, update iterator)
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709046#action_12709046 ]
Yonik Seeley commented on SOLR-1162:
------------------------------------
I haven't looked at the patch, but I agree with the rational. One would not be able to use multiple requests in parallel because of uncertain ordering of their execution.
> SolrJ UpdateRequest does not maintain order of operations when sending mixed types of changes (updates, delete id, delete query, update iterator)
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1162
> URL: https://issues.apache.org/jira/browse/SOLR-1162
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1162.patch, Solr-1162.patch
>
>
> In SolrJ UpdateRequest object it maintains separate lists of documents to add, delete, and delete queries so that the order of those operations is not known to the caller. It really should execute the items in the same order they were added to the UpdateRequest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.