You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2021/12/07 12:53:44 UTC
[GitHub] [shardingsphere] xuejmnet opened a new issue #13942: ConnectionMode.MEMORY_STRICTLY Should Use StreamQueryResult If Use Page Query
xuejmnet opened a new issue #13942:
URL: https://github.com/apache/shardingsphere/issues/13942
```
ConnectionMode.MEMORY_STRICTLY == connectionMode ? new JDBCStreamQueryResult(resultSet) : new JDBCMemoryQueryResult(resultSet)
```
is one node sql query result,but if this group has more than one query parallel,sub node parallel should use stream query result,if use memory query result in page query, `limit 10,10` will rewrite `limit 0,20` if parallel query result use memory result not stream query result,memory will has more than scan rows.for example: `sql.skip(10).take(10) `in one data base 2 table will route, if ConnectionMode is CONNECTION_STRICTLY sharding sphere will use MemoryQueryResult, memory has 40 total rows,
```
public abstract class JDBCExecutorCallback<T> implements ExecutorCallback<JDBCExecutionUnit, T> {
@Override
public final Collection<T> execute(final Collection<JDBCExecutionUnit> executionUnits, final boolean isTrunkThread, final Map<String, Object> dataMap) throws SQLException {
// TODO It is better to judge whether need sane result before execute, can avoid exception thrown
Collection<T> result = new LinkedList<>();
for (JDBCExecutionUnit each : executionUnits) {
//once query memory add 20 rows
//once query memory add 20 rows
//once query memory add 20 rows
T executeResult = execute(each, isTrunkThread, dataMap);
if (null != executeResult) {
result.add(executeResult);
}
}
//once query should return stream ,memory merge in foreach end
//and should input last memory result
// first only stream merge array do merge
//second and then should last memory result + current stream merge array do merge
if(connectionMode==ConnectionMode.CONNECTION_STRICTLY ){
(lastMemoryResult+result).merge()
}
return result;
}
}
```
so ConnectionMode is CONNECTION_STRICTLY still use StreamResult,but with other connection parallel result should combine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] xuejmnet commented on issue #13942: ConnectionMode.MEMORY_STRICTLY Should Use StreamQueryResult If Use Page Query
Posted by GitBox <gi...@apache.org>.
xuejmnet commented on issue #13942:
URL: https://github.com/apache/shardingsphere/issues/13942#issuecomment-987899742
select * from order limit 10,10 ,now has 20rows for every node,memory is large
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #13942: ConnectionMode.MEMORY_STRICTLY Should Use StreamQueryResult If Use Page Query
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #13942:
URL: https://github.com/apache/shardingsphere/issues/13942#issuecomment-988421585
> I'm looking forward to ShardingSphere better :)
@xuejmnet Thank you. We will try our best to optimize performance, and you are welcome to participate in the community.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #13942: ConnectionMode.MEMORY_STRICTLY Should Use StreamQueryResult If Use Page Query
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #13942:
URL: https://github.com/apache/shardingsphere/issues/13942#issuecomment-988386546
Hi @xuejmnet I like this optimization, it seems to be effective in reducing memory usage.
But this optimization may require a lot of code adjustments, because at present, the results of all nodes are queried first, and then merged. To achieve this optimization, execution and merging need to be processed together.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] xuejmnet commented on issue #13942: ConnectionMode.MEMORY_STRICTLY Should Use StreamQueryResult If Use Page Query
Posted by GitBox <gi...@apache.org>.
xuejmnet commented on issue #13942:
URL: https://github.com/apache/shardingsphere/issues/13942#issuecomment-987893647
## now

## optimize

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] xuejmnet commented on issue #13942: ConnectionMode.MEMORY_STRICTLY Should Use StreamQueryResult If Use Page Query
Posted by GitBox <gi...@apache.org>.
xuejmnet commented on issue #13942:
URL: https://github.com/apache/shardingsphere/issues/13942#issuecomment-988389638
I'm looking forward to ShardingSphere better :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] xuejmnet closed issue #13942: ConnectionMode.MEMORY_STRICTLY Should Use StreamQueryResult If Use Page Query
Posted by GitBox <gi...@apache.org>.
xuejmnet closed issue #13942:
URL: https://github.com/apache/shardingsphere/issues/13942
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org