[jira] [Commented] (OODT-483) NumberFormatException when using RSS service to view transfers for large files

["Chris A. Mattmann (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OODT-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429154#comment-13429154 ] 

Chris A. Mattmann commented on OODT-483:
----------------------------------------

Ross, +1 this makes a ton of sense. Feel free to use Review Board http://reviews.apache.org/ to post up a review and link it to this JIRA issue. Brian Foster, Ricky, Sheryl, Tom, Paul R. and me and others are all active consumers of Review Board reviews, so that would be a great thing.

Review Board or not, +1 to commit this patch and this is a critical fix.

Thanks!

                
> NumberFormatException when using RSS service to view transfers for large files
> ------------------------------------------------------------------------------
>
>                 Key: OODT-483
>                 URL: https://issues.apache.org/jira/browse/OODT-483
>             Project: OODT
>          Issue Type: Bug
>          Components: file manager
>            Reporter: Ross Laidlaw
>            Assignee: Ross Laidlaw
>            Priority: Minor
>              Labels: gsoc2012
>             Fix For: 0.5
>
>         Attachments: OODT-483.rlaidlaw.2012-08-06.patch.txt
>
>
> I used File Manager to ingest a large file and then attempted to view the progress of the transfer using the 'cas-product' RSS web application (RSSProductTransferServlet [1]).  This caused a NumberFormatException as follows:
> {code}
> java.lang.NumberFormatException: For input string: "11100111001101000000000000"
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> java.lang.Long.parseLong(Long.java:422)
> java.lang.Long.parseLong(Long.java:468)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusFromXmlRpc(XmlRpcStructFactory.java:86)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusesFromXmlRpc(XmlRpcStructFactory.java:112)
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.getCurrentFileTransfers(XmlRpcFileManagerClient.java:398)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doIt(RSSProductTransferServlet.java:151)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doGet(RSSProductTransferServlet.java:138)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
> {code}
> From further testing I found that this exception occurs for files approximately 10MB in size or larger.
> This exception occurs because the XmlRpcStructFactory class is attempting to set the value of a long variable using a binary representation (i.e. treating the binary number as a decimal number).  The size of the binary representation exceeds the maximum capacity of the long because it is not being converted to a decimal representation.
> I traced the problem to the following methods in class XmlRpcStructFactory [2]:
> {code}
> public static Hashtable<String, Object> getXmlRpcFileTransferStatus(FileTransferStatus status)
> {
>   Hashtable<String, Object> statusHash = new Hashtable<String, Object>();
>   statusHash.put("bytesTransferred",Long.toBinaryString(status.getBytesTransferred()));
>   statusHash.put("parentProduct", getXmlRpcProduct(status.getParentProduct()));
>   statusHash.put("fileRef", getXmlRpcReference(status.getFileRef()));
>   return statusHash;
> }
> public static FileTransferStatus getFileTransferStatusFromXmlRpc(Hashtable<String, Object> statusHash) 
> {
>   FileTransferStatus status = new FileTransferStatus();
>   status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
>   status.setParentProduct(getProductFromXmlRpc((Hashtable<String, Object>) statusHash.get("parentProduct")));
>   status.setFileRef(getReferenceFromXmlRpc((Hashtable<String, Object>)
> statusHash.get("fileRef")));
>   return status;
> }
> {code}
> In the getXmlRpcFileTransferStatus method shown above, the following line converts the value returned by 'status.getBytesTransferred()' to a binary representation and stores it as a String in the 'statusHash' Hashtable:
> {code}
>   statusHash.put("bytesTransferred", Long.toBinaryString(status.getBytesTransferred()));
> {code}
> But when the value is retrieved from statusHash in method getFileTransferStatusFromXmlRpc, it isn't assumed to be a binary number:
> {code}
>   status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> {code}
> For large files, the binary representation exceeds the capacity of a long.  In the example above, the input string "11100111001101000000000000" converted directly to a long (i.e. treated as a decimal) will exceed the maximum size of a long (9,223,372,036,854,775,807).
> A simple fix for this would be to add a radix argument to Long.parseLong in method getFileTransferStatusFromXmlRpc, as follows:
> {code}
>   status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString(), 2));{code}
> I tested this out on a large file (over 100MB) and it seemed to solve the problem.  Would this be an acceptable fix?  I'll attach a patch to this issue for reference.
> An alternative solution would be not to store a binary representation in the statusHash Hashtable.  But I'm assuming there's a reason why it's converted to binary for the hash.  
> [1] http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductTransferServlet.java
> [2] http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira