[jira] Commented: (DIRSERVER-1319) Cannot save String values larger than 64 Kb

["Felix Knecht (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DIRSERVER-1319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12874411#action_12874411 ] 

Felix Knecht commented on DIRSERVER-1319:
-----------------------------------------

http://www.coderanch.com/t/382385/java/java/String-StringBuffer-OutOfMemoryError


public class WriteUTFAssistant {
  
   /*
    * In UTF every char may have 1-3 bytes size 
    * Max writeUTF bytes / max char symbol size
    */
   public static final int BLOCK_SIZE = 65535 / 3;
   
   /*
    * This method write string into stream in spite of its length
    * Standard DataOutput.writeUTF() contract tells us than we
    * cannot write more than 65535 bytes.
    * This method splits string to substrings and write them.
    * Forum discussion
    * <a href="http://forum.java.sun.com/thread.jspa?messageID=9580692" target="_blank" rel="nofollow">http://forum.java.sun.com/thread.jspa?messageID=9580692</a>
    */
   public static void writeUTFSafely(DataOutputStream outputStream,
                                     String stringToWrite) throws IOException {
       
       int strLength = stringToWrite.length();
       int blocksAmount = strLength / BLOCK_SIZE;
              
       if(blocksAmount == 0)
       {
          outputStream.writeInt(1);
          outputStream.writeUTF(stringToWrite);          
       }
       else 
       {  
          if (strLength % BLOCK_SIZE > 0) 
          {
             ++blocksAmount;
          }
          String[] strArray = new String[blocksAmount];   
          
          for (int i = 0; i < blocksAmount; ++i)
          {
              int beginIndex = i * BLOCK_SIZE;
              int lastIndex = beginIndex + BLOCK_SIZE;
              
              if (lastIndex > strLength)
              {
                 strArray[i] = stringToWrite.substring(beginIndex, strLength);
              }                  
              else
              {
                 strArray[i] = stringToWrite.substring(beginIndex, lastIndex);
              }   
          }
          outputStream.writeInt(blocksAmount);
          for (int i = 0; i < blocksAmount; i++)
          {
             outputStream.writeUTF(strArray[i]);
          }   
       }
   }
 
   /*
    * This method reads UTF String which 
    * we write with writeUTF mehtod of this class
    */
   public static String readUTFSafely(DataInputStream inputStream) throws IOException
   {
       int blocksAmount = inputStream.readInt();
       StringBuffer buffer = new StringBuffer(blocksAmount * BLOCK_SIZE);
       //String[] string = new String[blocksAmount * BLOCK_SIZE];
       String result = "";
       for (int i = 0; i < blocksAmount; ++i)
       {
          result += inputStream.readUTF();
          //string[i] = inputStream.readUTF();
           //buffer.append(inputStream.readUTF());
       }
       return result;
       //return buffer.toString();
   }
 
}

> Cannot save String values larger than 64 Kb 
> --------------------------------------------
>
>                 Key: DIRSERVER-1319
>                 URL: https://issues.apache.org/jira/browse/DIRSERVER-1319
>             Project: Directory ApacheDS
>          Issue Type: Bug
>    Affects Versions: 1.5.4
>            Reporter: Emmanuel Lecharny
>             Fix For: 2.0.0-RC1
>
>
> We can't save String values into the server due to the way the entries are serialized : we are using a OutputStream.writeUTF() which is limited to 65535 chars.
> We have two options :
> - we save data splitting them in smaller chuncks
> - we save a reference to a file, considering the data as a stream.
> The second option would be way better, but is more complicated to implement. In any case, both system change the way the serialization works.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.