You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openjpa.apache.org by "Fay Wang (JIRA)" <ji...@apache.org> on 2010/06/14 22:35:12 UTC
[jira] Created: (OPENJPA-1699) Streaming Lob support in DB2
Streaming Lob support in DB2
----------------------------
Key: OPENJPA-1699
URL: https://issues.apache.org/jira/browse/OPENJPA-1699
Project: OpenJPA
Issue Type: Bug
Components: jdbc
Affects Versions: 2.0.0
Reporter: Fay Wang
Assignee: Fay Wang
Fix For: 2.0.0
Streaming Lob support in DB2
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (OPENJPA-1699) Streaming Lob support in DB2
Posted by "Michael Dick (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Dick updated OPENJPA-1699:
----------------------------------
Fix Version/s: 2.1.0
(was: 2.0.0)
> Streaming Lob support in DB2
> ----------------------------
>
> Key: OPENJPA-1699
> URL: https://issues.apache.org/jira/browse/OPENJPA-1699
> Project: OpenJPA
> Issue Type: Bug
> Components: jdbc
> Affects Versions: 2.0.0
> Reporter: Fay Wang
> Assignee: Fay Wang
> Fix For: 2.1.0
>
>
> Streaming Lob support in DB2
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (OPENJPA-1699) Streaming Lob support in DB2
Posted by "Heath Thomann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Heath Thomann updated OPENJPA-1699:
-----------------------------------
Attachment: OPENJPA-1699-DB2-1.2.x.patch
I'm attaching patch named OPENJPA-1699-DB2-1.2.x.patch which is for 1.2.x. This is basically a back port of the changes made in truck via OJ-130, with one additional fix to LobFieldStrategy for 1.2.x.
Thanks,
Heath
> Streaming Lob support in DB2
> ----------------------------
>
> Key: OPENJPA-1699
> URL: https://issues.apache.org/jira/browse/OPENJPA-1699
> Project: OpenJPA
> Issue Type: New Feature
> Components: jdbc
> Affects Versions: 2.0.0
> Reporter: Fay Wang
> Assignee: Fay Wang
> Fix For: 2.1.0
>
> Attachments: OPENJPA-1699-DB2-1.2.x.patch
>
>
> Streaming Lob support in DB2
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (OPENJPA-1699) Streaming Lob support in DB2
Posted by "Milosz Tylenda (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Milosz Tylenda reopened OPENJPA-1699:
-------------------------------------
The fullyMaterializedLobData trick should be documented under "Known issues with DB2" section, like it is done with other database-specific LOB streaming limitations (Oracle and MySQL for example).
> Streaming Lob support in DB2
> ----------------------------
>
> Key: OPENJPA-1699
> URL: https://issues.apache.org/jira/browse/OPENJPA-1699
> Project: OpenJPA
> Issue Type: Improvement
> Components: jdbc
> Affects Versions: 2.0.0
> Reporter: Fay Wang
> Assignee: Fay Wang
> Fix For: 1.2.3, 1.3.0, 2.0.2, 2.1.0
>
> Attachments: OPENJPA-1699-DB2-1.2.x.patch, OPENJPA-1699-DB2-1.3.x.patch, OPENJPA-1699-DB2-2.0.x.patch
>
>
> Streaming Lob support in DB2
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (OPENJPA-1699) Streaming Lob support in DB2
Posted by "Milosz Tylenda (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Milosz Tylenda updated OPENJPA-1699:
------------------------------------
Issue Type: New Feature (was: Bug)
> Streaming Lob support in DB2
> ----------------------------
>
> Key: OPENJPA-1699
> URL: https://issues.apache.org/jira/browse/OPENJPA-1699
> Project: OpenJPA
> Issue Type: New Feature
> Components: jdbc
> Affects Versions: 2.0.0
> Reporter: Fay Wang
> Assignee: Fay Wang
> Fix For: 2.1.0
>
>
> Streaming Lob support in DB2
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (OPENJPA-1699) Streaming Lob support in DB2
Posted by "Heath Thomann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Heath Thomann updated OPENJPA-1699:
-----------------------------------
Attachment: (was: OPENJPA-1699-DB2-1.2.x.patch)
> Streaming Lob support in DB2
> ----------------------------
>
> Key: OPENJPA-1699
> URL: https://issues.apache.org/jira/browse/OPENJPA-1699
> Project: OpenJPA
> Issue Type: New Feature
> Components: jdbc
> Affects Versions: 2.0.0
> Reporter: Fay Wang
> Assignee: Fay Wang
> Fix For: 2.1.0
>
> Attachments: OPENJPA-1699-DB2-1.2.x.patch
>
>
> Streaming Lob support in DB2
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (OPENJPA-1699) Streaming Lob support in DB2
Posted by "Heath Thomann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Heath Thomann updated OPENJPA-1699:
-----------------------------------
Attachment: OPENJPA-1699-DB2-2.0.x.patch
I'm attaching patch named OPENJPA-1699-DB2-2.0.x.patch which is for 2.0.x. This is basically a back port of the changes made in truck via OJ-130.
Thanks,
Heath
> Streaming Lob support in DB2
> ----------------------------
>
> Key: OPENJPA-1699
> URL: https://issues.apache.org/jira/browse/OPENJPA-1699
> Project: OpenJPA
> Issue Type: New Feature
> Components: jdbc
> Affects Versions: 2.0.0
> Reporter: Fay Wang
> Assignee: Fay Wang
> Fix For: 2.1.0
>
> Attachments: OPENJPA-1699-DB2-1.2.x.patch, OPENJPA-1699-DB2-2.0.x.patch
>
>
> Streaming Lob support in DB2
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (OPENJPA-1699) Streaming Lob support in DB2
Posted by "Fay Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887491#action_12887491 ]
Fay Wang commented on OPENJPA-1699:
-----------------------------------
In the case of very large lob, DB2 JCC driver will automatically use progressive streaming to retrieve the Lob data. With progressiveStreaming, the inputStream retrieved (see red below) must be materialized before the next iteration of call to rs.next().
ResultSet rs = pstmt.executeQuery(...);
while (rs.next()) {
InputStream is = rs.getBinaryInputStream(1);
writeToFile(is);
}
rs.close();
However, in OpenJPA, we iterate through the whole resultSet and set the InputStream to respective entities:
ResultSet rs = pstmt.executeQuery(...);
while (rs.next()) {
InputStream is = rs.getBinaryInputStream(1);
entity.setInputStream(is);
}
rs.close();
...
Once the entity is returned from OpenJPA, the application will get Lob Closed exception when trying to process the InputStream:
writeToFile(entity.getInputStream()); <== Lob is closed Exception
The work around is to force fullyMaterializedLobData to true (see below) in the url, so that the DB2 JCC driver will materialize the whole Lob data into memory before returning to the application:
<property name="openjpa.ConnectionProperties"
value="DriverClassName=com.ibm.db2.jcc.DB2Driver,Url=jdbc:db2://localhost:50000/demodb:fullyMaterializeLobData=true;progressiveStreaming=NO;,Username=db2user,Password=passw0rd" />
> Streaming Lob support in DB2
> ----------------------------
>
> Key: OPENJPA-1699
> URL: https://issues.apache.org/jira/browse/OPENJPA-1699
> Project: OpenJPA
> Issue Type: New Feature
> Components: jdbc
> Affects Versions: 2.0.0
> Reporter: Fay Wang
> Assignee: Fay Wang
> Fix For: 2.1.0
>
>
> Streaming Lob support in DB2
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (OPENJPA-1699) Streaming Lob support in DB2
Posted by "Milosz Tylenda (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879480#action_12879480 ]
Milosz Tylenda commented on OPENJPA-1699:
-----------------------------------------
Now the patch looks much better to me. I am just still curious why the instanceof checks are needed, I don't remember seeing them in other dictionaries which support LOB streaming.
> Streaming Lob support in DB2
> ----------------------------
>
> Key: OPENJPA-1699
> URL: https://issues.apache.org/jira/browse/OPENJPA-1699
> Project: OpenJPA
> Issue Type: New Feature
> Components: jdbc
> Affects Versions: 2.0.0
> Reporter: Fay Wang
> Assignee: Fay Wang
> Fix For: 2.1.0
>
>
> Streaming Lob support in DB2
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (OPENJPA-1699) Streaming Lob support in DB2
Posted by "Michael Dick (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Dick updated OPENJPA-1699:
----------------------------------
Fix Version/s: 2.0.2
1.3.0
1.2.3
Issue Type: Improvement (was: New Feature)
Thanks for the patches Fay and Heath.
> Streaming Lob support in DB2
> ----------------------------
>
> Key: OPENJPA-1699
> URL: https://issues.apache.org/jira/browse/OPENJPA-1699
> Project: OpenJPA
> Issue Type: Improvement
> Components: jdbc
> Affects Versions: 2.0.0
> Reporter: Fay Wang
> Assignee: Fay Wang
> Fix For: 1.2.3, 1.3.0, 2.0.2, 2.1.0
>
> Attachments: OPENJPA-1699-DB2-1.2.x.patch, OPENJPA-1699-DB2-1.3.x.patch, OPENJPA-1699-DB2-2.0.x.patch
>
>
> Streaming Lob support in DB2
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (OPENJPA-1699) Streaming Lob support in DB2
Posted by "Heath Thomann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Heath Thomann updated OPENJPA-1699:
-----------------------------------
Comment: was deleted
(was: I'm attaching patch named OPENJPA-1699-DB2-1.2.x.patch which is for 1.2.x. This is basically a back port of the changes made in truck via OJ-130, with one additional fix to LobFieldStrategy for 1.2.x.
Thanks,
Heath)
> Streaming Lob support in DB2
> ----------------------------
>
> Key: OPENJPA-1699
> URL: https://issues.apache.org/jira/browse/OPENJPA-1699
> Project: OpenJPA
> Issue Type: New Feature
> Components: jdbc
> Affects Versions: 2.0.0
> Reporter: Fay Wang
> Assignee: Fay Wang
> Fix For: 2.1.0
>
> Attachments: OPENJPA-1699-DB2-1.2.x.patch
>
>
> Streaming Lob support in DB2
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (OPENJPA-1699) Streaming Lob support in DB2
Posted by "Michael Dick (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Dick resolved OPENJPA-1699.
-----------------------------------
Resolution: Fixed
> Streaming Lob support in DB2
> ----------------------------
>
> Key: OPENJPA-1699
> URL: https://issues.apache.org/jira/browse/OPENJPA-1699
> Project: OpenJPA
> Issue Type: Improvement
> Components: jdbc
> Affects Versions: 2.0.0
> Reporter: Fay Wang
> Assignee: Fay Wang
> Fix For: 1.2.3, 1.3.0, 2.0.2, 2.1.0
>
> Attachments: OPENJPA-1699-DB2-1.2.x.patch, OPENJPA-1699-DB2-1.3.x.patch, OPENJPA-1699-DB2-2.0.x.patch
>
>
> Streaming Lob support in DB2
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (OPENJPA-1699) Streaming Lob support in DB2
Posted by "Heath Thomann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Heath Thomann updated OPENJPA-1699:
-----------------------------------
Attachment: OPENJPA-1699-DB2-1.3.x.patch
I'm attaching patch named OPENJPA-1699-DB2-1.3.x.patch which is for 1.3.x. This is basically a back port of the changes made in truck via OJ-130.
Thanks,
Heath
> Streaming Lob support in DB2
> ----------------------------
>
> Key: OPENJPA-1699
> URL: https://issues.apache.org/jira/browse/OPENJPA-1699
> Project: OpenJPA
> Issue Type: New Feature
> Components: jdbc
> Affects Versions: 2.0.0
> Reporter: Fay Wang
> Assignee: Fay Wang
> Fix For: 2.1.0
>
> Attachments: OPENJPA-1699-DB2-1.2.x.patch, OPENJPA-1699-DB2-1.3.x.patch, OPENJPA-1699-DB2-2.0.x.patch
>
>
> Streaming Lob support in DB2
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (OPENJPA-1699) Streaming Lob support in DB2
Posted by "Heath Thomann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Heath Thomann updated OPENJPA-1699:
-----------------------------------
Attachment: OPENJPA-1699-DB2-1.2.x.patch
I'm attaching patch named OPENJPA-1699-DB2-1.2.x.patch which is for 1.2.x. This is basically a back port of the changes made in truck via OJ-130, with one additional fix to LobFieldStrategy for 1.2.x.
Thanks,
Heath
> Streaming Lob support in DB2
> ----------------------------
>
> Key: OPENJPA-1699
> URL: https://issues.apache.org/jira/browse/OPENJPA-1699
> Project: OpenJPA
> Issue Type: New Feature
> Components: jdbc
> Affects Versions: 2.0.0
> Reporter: Fay Wang
> Assignee: Fay Wang
> Fix For: 2.1.0
>
> Attachments: OPENJPA-1699-DB2-1.2.x.patch
>
>
> Streaming Lob support in DB2
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.