[jira] [Updated] (RYA-313) Rya Mongo Blows up on Large result sets

["Aaron Mihalik (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/RYA-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aaron Mihalik updated RYA-313:
------------------------------
    Description: 
Simple queries that return a lot of results fail because mongo is trying to send all of the results back at once.  For instance, if I have a lot of data and run something like:

{noformat}
SELECT * WHERE 
{
	?s a ?t.
}
{noformat}

I will get this exception.

{noformat}
Caused by: com.mongodb.MongoCommandException: Command failed with error 16389: 'aggregation result exceeds maximum document size (16MB)' on server localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "aggregation result exceeds maximum document size (16MB)", "code" : 16389 }
{noformat}

I think we need to toss in a "AggregationOptions with Batch = 1000", but I couldn't get that to work immediately.  Somebody with more mongo experience needs to look at this.

[Here is the line of code|https://github.com/apache/incubator-rya/blob/master/dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/RyaStatementBindingSetCursorIterator.java#L114]

  was:
Simple queries that return a lot of results fail because mongo is trying to send all of the results back at once.  For instance, if I have a lot of data and run something like:

'''
SELECT * WHERE 
{
	?s a ?t.
}
'''

I will get this exception.

Caused by: com.mongodb.MongoCommandException: Command failed with error 16389: 'aggregation result exceeds maximum document size (16MB)' on server localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "aggregation result exceeds maximum document size (16MB)", "code" : 16389 }

I think we need to toss in a "AggregationOptions with Batch = 1000", but I couldn't get that to work immediately.  Somebody with more mongo experience needs to look at this.

[Here is the line of code|https://github.com/apache/incubator-rya/blob/master/dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/RyaStatementBindingSetCursorIterator.java#L114
]


> Rya Mongo Blows up on Large result sets
> ---------------------------------------
>
>                 Key: RYA-313
>                 URL: https://issues.apache.org/jira/browse/RYA-313
>             Project: Rya
>          Issue Type: Bug
>          Components: dao
>    Affects Versions: 3.2.10
>         Environment: Mongo DB with Rya 3.2.11-SNAPSHOT with a lot of data in Rya
>            Reporter: Aaron Mihalik
>            Assignee: Andrew Smith
>
> Simple queries that return a lot of results fail because mongo is trying to send all of the results back at once.  For instance, if I have a lot of data and run something like:
> {noformat}
> SELECT * WHERE 
> {
> 	?s a ?t.
> }
> {noformat}
> I will get this exception.
> {noformat}
> Caused by: com.mongodb.MongoCommandException: Command failed with error 16389: 'aggregation result exceeds maximum document size (16MB)' on server localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "aggregation result exceeds maximum document size (16MB)", "code" : 16389 }
> {noformat}
> I think we need to toss in a "AggregationOptions with Batch = 1000", but I couldn't get that to work immediately.  Somebody with more mongo experience needs to look at this.
> [Here is the line of code|https://github.com/apache/incubator-rya/blob/master/dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/RyaStatementBindingSetCursorIterator.java#L114]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)