You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openjpa.apache.org by "Vermeulen (JIRA)" <ji...@apache.org> on 2011/01/07 17:01:07 UTC
[jira] Created: (OPENJPA-1920) N+1 select with entities that
contain Maps (even with FetchType.EAGER)
N+1 select with entities that contain Maps (even with FetchType.EAGER)
----------------------------------------------------------------------
Key: OPENJPA-1920
URL: https://issues.apache.org/jira/browse/OPENJPA-1920
Project: OpenJPA
Issue Type: Improvement
Components: kernel
Affects Versions: 2.0.0
Reporter: Vermeulen
When I have an entity that contains a Map, e.g.:
@Entity
public class TestEntity {
@ElementCollection(fetch = FetchType.EAGER)
Map<String, String> strings = new HashMap<String, String>();
}
And I select all existing entities
String query = "SELECT z FROM " + TestEntity.class.getSimpleName()
+ " z";
List<TestEntity> result = em.createQuery(query, TestEntity.class)
.getResultList();
n + 1 selects are performed which can be seen by using <property name="openjpa.Log" value="DefaultLevel=TRACE" />
917 testPU TRACE [main] openjpa.Query - Executing query: SELECT z FROM TestEntity z
918 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 17507279 SELECT t0.id, t0.name FROM TestEntity t0
918 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> [0 ms] spent
919 testPU TRACE [main] openjpa.jdbc.SQLDiag - load field: 'strings' for oid=entities.TestEntity-1 class entities.TestEntity
919 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 7493991 SELECT t0.KEY0, t0.value FROM TestEntity_strings t0 WHERE t0.TESTENTITY_ID = ? [params=(long) 1]
919 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> [0 ms] spent
920 testPU TRACE [main] openjpa.jdbc.SQLDiag - load field: 'strings' for oid=entities.TestEntity-51 class entities.TestEntity
920 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 7200207 SELECT t0.KEY0, t0.value FROM TestEntity_strings t0 WHERE t0.TESTENTITY_ID = ? [params=(long) 51]
etc.....
It doesn't matter whether or not I use <property name="openjpa.jdbc.EagerFetchMode" value="parallel"/> or <property name="openjpa.jdbc.EagerFetchMode" value="join"/>.
This is extremely inefficient when I wish to load a list of products that have their name set in multiple languages by using a Map from language to String.
As a workaround I can turn the Map into a List and search the List myself for the right entry.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (OPENJPA-1920) N+1 select with entities that
contain Maps (even with FetchType.EAGER)
Posted by "Rick Curtis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12995878#comment-12995878 ]
Rick Curtis commented on OPENJPA-1920:
--------------------------------------
Ignore my previous comment, I wasn't running the correct scenario.
> N+1 select with entities that contain Maps (even with FetchType.EAGER)
> ----------------------------------------------------------------------
>
> Key: OPENJPA-1920
> URL: https://issues.apache.org/jira/browse/OPENJPA-1920
> Project: OpenJPA
> Issue Type: Improvement
> Components: kernel
> Affects Versions: 2.0.0
> Reporter: Vermeulen
>
> When I have an entity that contains a Map, e.g.:
> @Entity
> public class TestEntity {
> @ElementCollection(fetch = FetchType.EAGER)
> Map<String, String> strings = new HashMap<String, String>();
> }
> And I select all existing entities
> String query = "SELECT z FROM " + TestEntity.class.getSimpleName()
> + " z";
> List<TestEntity> result = em.createQuery(query, TestEntity.class)
> .getResultList();
> n + 1 selects are performed which can be seen by using <property name="openjpa.Log" value="DefaultLevel=TRACE" />
> 917 testPU TRACE [main] openjpa.Query - Executing query: SELECT z FROM TestEntity z
> 918 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 17507279 SELECT t0.id, t0.name FROM TestEntity t0
> 918 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> [0 ms] spent
> 919 testPU TRACE [main] openjpa.jdbc.SQLDiag - load field: 'strings' for oid=entities.TestEntity-1 class entities.TestEntity
> 919 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 7493991 SELECT t0.KEY0, t0.value FROM TestEntity_strings t0 WHERE t0.TESTENTITY_ID = ? [params=(long) 1]
> 919 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> [0 ms] spent
> 920 testPU TRACE [main] openjpa.jdbc.SQLDiag - load field: 'strings' for oid=entities.TestEntity-51 class entities.TestEntity
> 920 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 7200207 SELECT t0.KEY0, t0.value FROM TestEntity_strings t0 WHERE t0.TESTENTITY_ID = ? [params=(long) 51]
> etc.....
> It doesn't matter whether or not I use <property name="openjpa.jdbc.EagerFetchMode" value="parallel"/> or <property name="openjpa.jdbc.EagerFetchMode" value="join"/>.
> This is extremely inefficient when I wish to load a list of products that have their name set in multiple languages by using a Map from language to String.
> As a workaround I can turn the Map into a List and search the List myself for the right entry.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OPENJPA-1920) N+1 select with entities that
contain Maps (even with FetchType.EAGER)
Posted by "Vermeulen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493954#comment-13493954 ]
Vermeulen commented on OPENJPA-1920:
------------------------------------
I used the Eclipse debugger which I also used for the related OPENJPA-2296.
When JDBCStoreManager.createEagerSelects considers the TestEntity.strings field for parallel select, the expression fms[i].supportsSelect(sel, Select.EAGER_PARALLEL, sm, this, fetch) returns 0 so it is ignored for efficient eager fetching.
Does this mean that this kind of mapping does not support parallel fetch? Why not? Is this easy to add? Why doesn't it do join fetch instead?
> N+1 select with entities that contain Maps (even with FetchType.EAGER)
> ----------------------------------------------------------------------
>
> Key: OPENJPA-1920
> URL: https://issues.apache.org/jira/browse/OPENJPA-1920
> Project: OpenJPA
> Issue Type: Improvement
> Components: kernel
> Affects Versions: 2.0.0
> Reporter: Vermeulen
>
> When I have an entity that contains a Map, e.g.:
> @Entity
> public class TestEntity {
> @ElementCollection(fetch = FetchType.EAGER)
> Map<String, String> strings = new HashMap<String, String>();
> }
> And I select all existing entities
> String query = "SELECT z FROM " + TestEntity.class.getSimpleName()
> + " z";
> List<TestEntity> result = em.createQuery(query, TestEntity.class)
> .getResultList();
> n + 1 selects are performed which can be seen by using <property name="openjpa.Log" value="DefaultLevel=TRACE" />
> 917 testPU TRACE [main] openjpa.Query - Executing query: SELECT z FROM TestEntity z
> 918 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 17507279 SELECT t0.id, t0.name FROM TestEntity t0
> 918 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> [0 ms] spent
> 919 testPU TRACE [main] openjpa.jdbc.SQLDiag - load field: 'strings' for oid=entities.TestEntity-1 class entities.TestEntity
> 919 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 7493991 SELECT t0.KEY0, t0.value FROM TestEntity_strings t0 WHERE t0.TESTENTITY_ID = ? [params=(long) 1]
> 919 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> [0 ms] spent
> 920 testPU TRACE [main] openjpa.jdbc.SQLDiag - load field: 'strings' for oid=entities.TestEntity-51 class entities.TestEntity
> 920 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 7200207 SELECT t0.KEY0, t0.value FROM TestEntity_strings t0 WHERE t0.TESTENTITY_ID = ? [params=(long) 51]
> etc.....
> It doesn't matter whether or not I use <property name="openjpa.jdbc.EagerFetchMode" value="parallel"/> or <property name="openjpa.jdbc.EagerFetchMode" value="join"/>.
> This is extremely inefficient when I wish to load a list of products that have their name set in multiple languages by using a Map from language to String.
> As a workaround I can turn the Map into a List and search the List myself for the right entry.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (OPENJPA-1920) N+1 select with entities
that contain Maps (even with FetchType.EAGER)
Posted by "Vermeulen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493954#comment-13493954 ]
Vermeulen edited comment on OPENJPA-1920 at 11/9/12 11:54 AM:
--------------------------------------------------------------
I used the Eclipse debugger which I also used for the related OPENJPA-2296.
When JDBCStoreManager.createEagerSelects considers the TestEntity.strings field for parallel select, the expression fms[i].supportsSelect(sel, Select.EAGER_PARALLEL, sm, this, fetch) returns 0 so it is ignored for efficient eager fetching.
Does this mean that this kind of mapping does not support parallel fetch? Why not? Is this easy to add? Why doesn't it do join fetch instead?
(EDIT: from the debugger I can see that this fetch mode is also considered inside JDBCStoreManager.createEagerSelects but somehow also rejected)
was (Author: slowstrider):
I used the Eclipse debugger which I also used for the related OPENJPA-2296.
When JDBCStoreManager.createEagerSelects considers the TestEntity.strings field for parallel select, the expression fms[i].supportsSelect(sel, Select.EAGER_PARALLEL, sm, this, fetch) returns 0 so it is ignored for efficient eager fetching.
Does this mean that this kind of mapping does not support parallel fetch? Why not? Is this easy to add? Why doesn't it do join fetch instead?
> N+1 select with entities that contain Maps (even with FetchType.EAGER)
> ----------------------------------------------------------------------
>
> Key: OPENJPA-1920
> URL: https://issues.apache.org/jira/browse/OPENJPA-1920
> Project: OpenJPA
> Issue Type: Improvement
> Components: kernel
> Affects Versions: 2.0.0
> Reporter: Vermeulen
>
> When I have an entity that contains a Map, e.g.:
> @Entity
> public class TestEntity {
> @ElementCollection(fetch = FetchType.EAGER)
> Map<String, String> strings = new HashMap<String, String>();
> }
> And I select all existing entities
> String query = "SELECT z FROM " + TestEntity.class.getSimpleName()
> + " z";
> List<TestEntity> result = em.createQuery(query, TestEntity.class)
> .getResultList();
> n + 1 selects are performed which can be seen by using <property name="openjpa.Log" value="DefaultLevel=TRACE" />
> 917 testPU TRACE [main] openjpa.Query - Executing query: SELECT z FROM TestEntity z
> 918 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 17507279 SELECT t0.id, t0.name FROM TestEntity t0
> 918 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> [0 ms] spent
> 919 testPU TRACE [main] openjpa.jdbc.SQLDiag - load field: 'strings' for oid=entities.TestEntity-1 class entities.TestEntity
> 919 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 7493991 SELECT t0.KEY0, t0.value FROM TestEntity_strings t0 WHERE t0.TESTENTITY_ID = ? [params=(long) 1]
> 919 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> [0 ms] spent
> 920 testPU TRACE [main] openjpa.jdbc.SQLDiag - load field: 'strings' for oid=entities.TestEntity-51 class entities.TestEntity
> 920 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 7200207 SELECT t0.KEY0, t0.value FROM TestEntity_strings t0 WHERE t0.TESTENTITY_ID = ? [params=(long) 51]
> etc.....
> It doesn't matter whether or not I use <property name="openjpa.jdbc.EagerFetchMode" value="parallel"/> or <property name="openjpa.jdbc.EagerFetchMode" value="join"/>.
> This is extremely inefficient when I wish to load a list of products that have their name set in multiple languages by using a Map from language to String.
> As a workaround I can turn the Map into a List and search the List myself for the right entry.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OPENJPA-1920) N+1 select with entities that
contain Maps (even with FetchType.EAGER)
Posted by "Vermeulen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vermeulen updated OPENJPA-1920:
-------------------------------
Attachment: testeagermapfetching.zip
Added OpenJPA unit tests that show the problem.
Also added a test to the existing TestParallelEagerMap. This does NOT suffer from the N+1 select problem by using a MapKey annotation and a map value entity that has a map key column.
So using a MapKey annotaiton may be a good workaround for people that encounter this problem.
Also tried if the change to RelationFieldStrategy I mentioned in OPENJPA-2296 fixes the N+1 select problem, but it does not, so OPENJPA-2296 may have a different cause.
> N+1 select with entities that contain Maps (even with FetchType.EAGER)
> ----------------------------------------------------------------------
>
> Key: OPENJPA-1920
> URL: https://issues.apache.org/jira/browse/OPENJPA-1920
> Project: OpenJPA
> Issue Type: Improvement
> Components: kernel
> Affects Versions: 2.0.0
> Reporter: Vermeulen
> Attachments: testeagermapfetching.zip
>
>
> When I have an entity that contains a Map, e.g.:
> @Entity
> public class TestEntity {
> @ElementCollection(fetch = FetchType.EAGER)
> Map<String, String> strings = new HashMap<String, String>();
> }
> And I select all existing entities
> String query = "SELECT z FROM " + TestEntity.class.getSimpleName()
> + " z";
> List<TestEntity> result = em.createQuery(query, TestEntity.class)
> .getResultList();
> n + 1 selects are performed which can be seen by using <property name="openjpa.Log" value="DefaultLevel=TRACE" />
> 917 testPU TRACE [main] openjpa.Query - Executing query: SELECT z FROM TestEntity z
> 918 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 17507279 SELECT t0.id, t0.name FROM TestEntity t0
> 918 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> [0 ms] spent
> 919 testPU TRACE [main] openjpa.jdbc.SQLDiag - load field: 'strings' for oid=entities.TestEntity-1 class entities.TestEntity
> 919 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 7493991 SELECT t0.KEY0, t0.value FROM TestEntity_strings t0 WHERE t0.TESTENTITY_ID = ? [params=(long) 1]
> 919 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> [0 ms] spent
> 920 testPU TRACE [main] openjpa.jdbc.SQLDiag - load field: 'strings' for oid=entities.TestEntity-51 class entities.TestEntity
> 920 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 7200207 SELECT t0.KEY0, t0.value FROM TestEntity_strings t0 WHERE t0.TESTENTITY_ID = ? [params=(long) 51]
> etc.....
> It doesn't matter whether or not I use <property name="openjpa.jdbc.EagerFetchMode" value="parallel"/> or <property name="openjpa.jdbc.EagerFetchMode" value="join"/>.
> This is extremely inefficient when I wish to load a list of products that have their name set in multiple languages by using a Map from language to String.
> As a workaround I can turn the Map into a List and search the List myself for the right entry.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (OPENJPA-1920) N+1 select with entities that
contain Maps (even with FetchType.EAGER)
Posted by "Rick Curtis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12995392#comment-12995392 ]
Rick Curtis commented on OPENJPA-1920:
--------------------------------------
Try using a @PersistentMap rather than a @ElementCollection to see if that resolves the performance issue.
> N+1 select with entities that contain Maps (even with FetchType.EAGER)
> ----------------------------------------------------------------------
>
> Key: OPENJPA-1920
> URL: https://issues.apache.org/jira/browse/OPENJPA-1920
> Project: OpenJPA
> Issue Type: Improvement
> Components: kernel
> Affects Versions: 2.0.0
> Reporter: Vermeulen
>
> When I have an entity that contains a Map, e.g.:
> @Entity
> public class TestEntity {
> @ElementCollection(fetch = FetchType.EAGER)
> Map<String, String> strings = new HashMap<String, String>();
> }
> And I select all existing entities
> String query = "SELECT z FROM " + TestEntity.class.getSimpleName()
> + " z";
> List<TestEntity> result = em.createQuery(query, TestEntity.class)
> .getResultList();
> n + 1 selects are performed which can be seen by using <property name="openjpa.Log" value="DefaultLevel=TRACE" />
> 917 testPU TRACE [main] openjpa.Query - Executing query: SELECT z FROM TestEntity z
> 918 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 17507279 SELECT t0.id, t0.name FROM TestEntity t0
> 918 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> [0 ms] spent
> 919 testPU TRACE [main] openjpa.jdbc.SQLDiag - load field: 'strings' for oid=entities.TestEntity-1 class entities.TestEntity
> 919 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 7493991 SELECT t0.KEY0, t0.value FROM TestEntity_strings t0 WHERE t0.TESTENTITY_ID = ? [params=(long) 1]
> 919 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> [0 ms] spent
> 920 testPU TRACE [main] openjpa.jdbc.SQLDiag - load field: 'strings' for oid=entities.TestEntity-51 class entities.TestEntity
> 920 testPU TRACE [main] openjpa.jdbc.SQL - <t 4837279, conn 6040101> executing prepstmnt 7200207 SELECT t0.KEY0, t0.value FROM TestEntity_strings t0 WHERE t0.TESTENTITY_ID = ? [params=(long) 51]
> etc.....
> It doesn't matter whether or not I use <property name="openjpa.jdbc.EagerFetchMode" value="parallel"/> or <property name="openjpa.jdbc.EagerFetchMode" value="join"/>.
> This is extremely inefficient when I wish to load a list of products that have their name set in multiple languages by using a Map from language to String.
> As a workaround I can turn the Map into a List and search the List myself for the right entry.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira