[jira] [Updated] (DERBY-5236) Client driver silently truncates strings that exceed 32KB

["Knut Anders Hatlen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/DERBY-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Knut Anders Hatlen updated DERBY-5236:
--------------------------------------

    Attachment: d5236-1a-client-fetch-complete.diff

The problem with strings that take 65510 bytes is indeed related to the string being split in three parts on transfer from the server. The client already knows that values can be split, and handles that by sending a CNTQRY command to the server when it detects that it needs more data, but it implicitly assumes that each value is only split once. That assumption is valid currently, since we never send strings long enough to span three blocks. This changed with the server-side fix to stop truncating the strings, and the client stops sending CNTQRY commands too early, and there's not enough data in the receive buffer to construct the string when ResultSet.getString() is called.

Just changing an "if" statement to a "while" statement in NetCursor.skipFdocaBytes() appears to be enough to get the client to send CNTQRY commands until the entire value has been retreived. This pattern is repeated in many methods in NetCursor. I think only skipFdocaBytes() will ever need to send more than one CNTQRY, even when the truncation is fixed on the server, but for completeness and consistency I factored out the common code into a helper method so that all of them now call CNTQRY in a loop. See the attached patch d5236-1a-client-fetch-complete.diff.

With the d5236-1a patch in combination with the write-full-string.diff patch, strings up to 65535 (2^16-1) bytes can be fetched. Larger strings will still cause problems because they overflow the two-byte length field.

> Client driver silently truncates strings that exceed 32KB
> ---------------------------------------------------------
>
>                 Key: DERBY-5236
>                 URL: https://issues.apache.org/jira/browse/DERBY-5236
>             Project: Derby
>          Issue Type: Bug
>          Components: Network Client
>    Affects Versions: 10.8.1.2
>            Reporter: Knut Anders Hatlen
>            Assignee: Knut Anders Hatlen
>         Attachments: d5236-1a-client-fetch-complete.diff, repro.diff, write-full-string.diff
>
>
> Can be seen with this JUnit test case that retrieves a VARCHAR value with 20000 characters. With the client driver, the string is truncated to 10900 characters (32700 bytes when encoded in UTF-8).
>     public void testLongColumn() throws SQLException {
>         PreparedStatement ps = prepareStatement(
>                 "values cast(? as varchar(20000))");
>         char[] chars = new char[20000];
>         Arrays.fill(chars, '\u4e10');
>         String str = new String(chars);
>         ps.setString(1, str);
>         JDBC.assertSingleValueResultSet(ps.executeQuery(), str);
>     }

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira