You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@oodt.apache.org by "Ross Laidlaw (JIRA)" <ji...@apache.org> on 2012/08/06 12:55:05 UTC
[jira] [Created] (OODT-483) NumberFormatException when using RSS
service to view transfers for large files
Ross Laidlaw created OODT-483:
---------------------------------
Summary: NumberFormatException when using RSS service to view transfers for large files
Key: OODT-483
URL: https://issues.apache.org/jira/browse/OODT-483
Project: OODT
Issue Type: Bug
Components: file manager
Reporter: Ross Laidlaw
Assignee: Ross Laidlaw
Priority: Minor
Fix For: 0.5
I used File Manager to ingest a large file and then attempted to view the progress of the transfer using the 'cas-product' RSS web application (RSSProductTransferServlet [1]). This caused a NumberFormatException as follows:
{code}
java.lang.NumberFormatException: For input string: "11100111001101000000000000"
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
java.lang.Long.parseLong(Long.java:422)
java.lang.Long.parseLong(Long.java:468)
org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusFromXmlRpc(XmlRpcStructFactory.java:86)
org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusesFromXmlRpc(XmlRpcStructFactory.java:112)
org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.getCurrentFileTransfers(XmlRpcFileManagerClient.java:398)
org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doIt(RSSProductTransferServlet.java:151)
org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doGet(RSSProductTransferServlet.java:138)
javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
{code}
>From further testing I found that this exception occurs for files approximately 10MB in size or larger.
This exception occurs because the XmlRpcStructFactory class is attempting to set the value of a long variable using a binary representation (i.e. treating the binary number as a decimal number). The size of the binary representation exceeds the maximum capacity of the long because it is not being converted to a decimal representation.
I traced the problem to the following methods in class XmlRpcStructFactory [2]:
{code}
public static Hashtable<String, Object> getXmlRpcFileTransferStatus(FileTransferStatus status)
{
Hashtable<String, Object> statusHash = new Hashtable<String, Object>();
statusHash.put("bytesTransferred",Long.toBinaryString(status.getBytesTransferred()));
statusHash.put("parentProduct", getXmlRpcProduct(status.getParentProduct()));
statusHash.put("fileRef", getXmlRpcReference(status.getFileRef()));
return statusHash;
}
public static FileTransferStatus getFileTransferStatusFromXmlRpc(Hashtable<String, Object> statusHash)
{
FileTransferStatus status = new FileTransferStatus();
status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
status.setParentProduct(getProductFromXmlRpc((Hashtable<String, Object>) statusHash.get("parentProduct")));
status.setFileRef(getReferenceFromXmlRpc((Hashtable<String, Object>)
statusHash.get("fileRef")));
return status;
}
{code}
In the getXmlRpcFileTransferStatus method shown above, the following line converts the value returned by 'status.getBytesTransferred()' to a binary representation and stores it as a String in the 'statusHash' Hashtable:
{code}
statusHash.put("bytesTransferred", Long.toBinaryString(status.getBytesTransferred()));
{code}
But when the value is retrieved from statusHash in method getFileTransferStatusFromXmlRpc, it isn't assumed to be a binary number:
{code}
status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
{code}
For large files, the binary representation exceeds the capacity of a long. In the example above, the input string "11100111001101000000000000" converted directly to a long (i.e. treated as a decimal) will exceed the maximum size of a long (9,223,372,036,854,775,807).
A simple fix for this would be to add a radix argument to Long.parseLong in method getFileTransferStatusFromXmlRpc, as follows:
{code}
status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString(), 2));{code}
I tested this out on a large file (over 100MB) and it seemed to solve the problem. Would this be an acceptable fix? I'll attach a patch to this issue for reference.
An alternative solution would be not to store a binary representation in the statusHash Hashtable. But I'm assuming there's a reason why it's converted to binary for the hash.
[1] http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductTransferServlet.java
[2] http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OODT-483) NumberFormatException when using RSS
service to view transfers for large files
Posted by "Ross Laidlaw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ross Laidlaw updated OODT-483:
------------------------------
Attachment: (was: OODT-483.rlaidlaw.2012-08-06.patch.txt)
> NumberFormatException when using RSS service to view transfers for large files
> ------------------------------------------------------------------------------
>
> Key: OODT-483
> URL: https://issues.apache.org/jira/browse/OODT-483
> Project: OODT
> Issue Type: Bug
> Components: file manager
> Reporter: Ross Laidlaw
> Assignee: Ross Laidlaw
> Priority: Minor
> Labels: gsoc2012
> Fix For: 0.5
>
>
> I used File Manager to ingest a large file and then attempted to view the progress of the transfer using the 'cas-product' RSS web application (RSSProductTransferServlet [1]). This caused a NumberFormatException as follows:
> {code}
> java.lang.NumberFormatException: For input string: "11100111001101000000000000"
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> java.lang.Long.parseLong(Long.java:422)
> java.lang.Long.parseLong(Long.java:468)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusFromXmlRpc(XmlRpcStructFactory.java:86)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusesFromXmlRpc(XmlRpcStructFactory.java:112)
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.getCurrentFileTransfers(XmlRpcFileManagerClient.java:398)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doIt(RSSProductTransferServlet.java:151)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doGet(RSSProductTransferServlet.java:138)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
> {code}
> From further testing I found that this exception occurs for files approximately 10MB in size or larger.
> This exception occurs because the XmlRpcStructFactory class is attempting to set the value of a long variable using a binary representation (i.e. treating the binary number as a decimal number). The size of the binary representation exceeds the maximum capacity of the long because it is not being converted to a decimal representation.
> I traced the problem to the following methods in class XmlRpcStructFactory [2]:
> {code}
> public static Hashtable<String, Object> getXmlRpcFileTransferStatus(FileTransferStatus status)
> {
> Hashtable<String, Object> statusHash = new Hashtable<String, Object>();
> statusHash.put("bytesTransferred",Long.toBinaryString(status.getBytesTransferred()));
> statusHash.put("parentProduct", getXmlRpcProduct(status.getParentProduct()));
> statusHash.put("fileRef", getXmlRpcReference(status.getFileRef()));
> return statusHash;
> }
> public static FileTransferStatus getFileTransferStatusFromXmlRpc(Hashtable<String, Object> statusHash)
> {
> FileTransferStatus status = new FileTransferStatus();
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> status.setParentProduct(getProductFromXmlRpc((Hashtable<String, Object>) statusHash.get("parentProduct")));
> status.setFileRef(getReferenceFromXmlRpc((Hashtable<String, Object>)
> statusHash.get("fileRef")));
> return status;
> }
> {code}
> In the getXmlRpcFileTransferStatus method shown above, the following line converts the value returned by 'status.getBytesTransferred()' to a binary representation and stores it as a String in the 'statusHash' Hashtable:
> {code}
> statusHash.put("bytesTransferred", Long.toBinaryString(status.getBytesTransferred()));
> {code}
> But when the value is retrieved from statusHash in method getFileTransferStatusFromXmlRpc, it isn't assumed to be a binary number:
> {code}
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> {code}
> For large files, the binary representation exceeds the capacity of a long. In the example above, the input string "11100111001101000000000000" converted directly to a long (i.e. treated as a decimal) will exceed the maximum size of a long (9,223,372,036,854,775,807).
> A simple fix for this would be to add a radix argument to Long.parseLong in method getFileTransferStatusFromXmlRpc, as follows:
> {code}
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString(), 2));{code}
> I tested this out on a large file (over 100MB) and it seemed to solve the problem. Would this be an acceptable fix? I'll attach a patch to this issue for reference.
> An alternative solution would be not to store a binary representation in the statusHash Hashtable. But I'm assuming there's a reason why it's converted to binary for the hash.
> [1] http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductTransferServlet.java
> [2] http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OODT-483) NumberFormatException when using RSS
service to view transfers for large files
Posted by "Ross Laidlaw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ross Laidlaw updated OODT-483:
------------------------------
Attachment: OODT-483.rlaidlaw.2012-08-06.patch.2.txt
OODT-483.rlaidlaw.2012-08-06.patch.1.txt
Two patches, each an alternative solution:
1) in getXmlRpcFileTransferStatus use 'toString' instead of 'toBinaryString' to store a decimal representation of the number in the string (in statusHash):
{code}
statusHash.put("bytesTransferred", Long.toString(status.getBytesTransferred()));
{code}
2) in getFileTransferStatusFromXmlRpc use a radix argument for parseLong to specify that the "bytesTransferred" string (from statusHash) represents a binary number:
{code} status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString(), 2));
{code}
> NumberFormatException when using RSS service to view transfers for large files
> ------------------------------------------------------------------------------
>
> Key: OODT-483
> URL: https://issues.apache.org/jira/browse/OODT-483
> Project: OODT
> Issue Type: Bug
> Components: file manager
> Reporter: Ross Laidlaw
> Assignee: Ross Laidlaw
> Priority: Minor
> Labels: gsoc2012
> Fix For: 0.5
>
> Attachments: OODT-483.rlaidlaw.2012-08-06.patch.1.txt, OODT-483.rlaidlaw.2012-08-06.patch.2.txt
>
>
> I used File Manager to ingest a large file and then attempted to view the progress of the transfer using the 'cas-product' RSS web application (RSSProductTransferServlet [1]). This caused a NumberFormatException as follows:
> {code}
> java.lang.NumberFormatException: For input string: "11100111001101000000000000"
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> java.lang.Long.parseLong(Long.java:422)
> java.lang.Long.parseLong(Long.java:468)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusFromXmlRpc(XmlRpcStructFactory.java:86)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusesFromXmlRpc(XmlRpcStructFactory.java:112)
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.getCurrentFileTransfers(XmlRpcFileManagerClient.java:398)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doIt(RSSProductTransferServlet.java:151)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doGet(RSSProductTransferServlet.java:138)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
> {code}
> From further testing I found that this exception occurs for files approximately 10MB in size or larger.
> This exception occurs because the XmlRpcStructFactory class is attempting to set the value of a long variable using a binary representation (i.e. treating the binary number as a decimal number). The size of the binary representation exceeds the maximum capacity of the long because it is not being converted to a decimal representation.
> I traced the problem to the following methods in class XmlRpcStructFactory [2]:
> {code}
> public static Hashtable<String, Object> getXmlRpcFileTransferStatus(FileTransferStatus status)
> {
> Hashtable<String, Object> statusHash = new Hashtable<String, Object>();
> statusHash.put("bytesTransferred",Long.toBinaryString(status.getBytesTransferred()));
> statusHash.put("parentProduct", getXmlRpcProduct(status.getParentProduct()));
> statusHash.put("fileRef", getXmlRpcReference(status.getFileRef()));
> return statusHash;
> }
> public static FileTransferStatus getFileTransferStatusFromXmlRpc(Hashtable<String, Object> statusHash)
> {
> FileTransferStatus status = new FileTransferStatus();
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> status.setParentProduct(getProductFromXmlRpc((Hashtable<String, Object>) statusHash.get("parentProduct")));
> status.setFileRef(getReferenceFromXmlRpc((Hashtable<String, Object>)
> statusHash.get("fileRef")));
> return status;
> }
> {code}
> In the getXmlRpcFileTransferStatus method shown above, the following line converts the value returned by 'status.getBytesTransferred()' to a binary representation and stores it as a String in the 'statusHash' Hashtable:
> {code}
> statusHash.put("bytesTransferred", Long.toBinaryString(status.getBytesTransferred()));
> {code}
> But when the value is retrieved from statusHash in method getFileTransferStatusFromXmlRpc, it isn't assumed to be a binary number:
> {code}
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> {code}
> For large files, the binary representation exceeds the capacity of a long. In the example above, the input string "11100111001101000000000000" converted directly to a long (i.e. treated as a decimal) will exceed the maximum size of a long (9,223,372,036,854,775,807).
> A simple fix for this would be to add a radix argument to Long.parseLong in method getFileTransferStatusFromXmlRpc, as follows:
> {code}
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString(), 2));{code}
> I tested this out on a large file (over 100MB) and it seemed to solve the problem. Would this be an acceptable fix? I'll attach a patch to this issue for reference.
> An alternative solution would be not to store a binary representation in the statusHash Hashtable. But I'm assuming there's a reason why it's converted to binary for the hash.
> [1] http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductTransferServlet.java
> [2] http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (OODT-483) NumberFormatException when using RSS
service to view transfers for large files
Posted by "Ross Laidlaw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ross Laidlaw closed OODT-483.
-----------------------------
> NumberFormatException when using RSS service to view transfers for large files
> ------------------------------------------------------------------------------
>
> Key: OODT-483
> URL: https://issues.apache.org/jira/browse/OODT-483
> Project: OODT
> Issue Type: Bug
> Components: file manager
> Reporter: Ross Laidlaw
> Assignee: Ross Laidlaw
> Priority: Minor
> Labels: gsoc2012
> Fix For: 0.5
>
> Attachments: OODT-483.rlaidlaw.2012-08-06.patch.1.txt, OODT-483.rlaidlaw.2012-08-06.patch.2.txt
>
>
> I used File Manager to ingest a large file and then attempted to view the progress of the transfer using the 'cas-product' RSS web application (RSSProductTransferServlet [1]). This caused a NumberFormatException as follows:
> {code}
> java.lang.NumberFormatException: For input string: "11100111001101000000000000"
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> java.lang.Long.parseLong(Long.java:422)
> java.lang.Long.parseLong(Long.java:468)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusFromXmlRpc(XmlRpcStructFactory.java:86)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusesFromXmlRpc(XmlRpcStructFactory.java:112)
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.getCurrentFileTransfers(XmlRpcFileManagerClient.java:398)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doIt(RSSProductTransferServlet.java:151)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doGet(RSSProductTransferServlet.java:138)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
> {code}
> From further testing I found that this exception occurs for files approximately 10MB in size or larger.
> This exception occurs because the XmlRpcStructFactory class is attempting to set the value of a long variable using a binary representation (i.e. treating the binary number as a decimal number). The size of the binary representation exceeds the maximum capacity of the long because it is not being converted to a decimal representation.
> I traced the problem to the following methods in class XmlRpcStructFactory [2]:
> {code}
> public static Hashtable<String, Object> getXmlRpcFileTransferStatus(FileTransferStatus status)
> {
> Hashtable<String, Object> statusHash = new Hashtable<String, Object>();
> statusHash.put("bytesTransferred",Long.toBinaryString(status.getBytesTransferred()));
> statusHash.put("parentProduct", getXmlRpcProduct(status.getParentProduct()));
> statusHash.put("fileRef", getXmlRpcReference(status.getFileRef()));
> return statusHash;
> }
> public static FileTransferStatus getFileTransferStatusFromXmlRpc(Hashtable<String, Object> statusHash)
> {
> FileTransferStatus status = new FileTransferStatus();
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> status.setParentProduct(getProductFromXmlRpc((Hashtable<String, Object>) statusHash.get("parentProduct")));
> status.setFileRef(getReferenceFromXmlRpc((Hashtable<String, Object>)
> statusHash.get("fileRef")));
> return status;
> }
> {code}
> In the getXmlRpcFileTransferStatus method shown above, the following line converts the value returned by 'status.getBytesTransferred()' to a binary representation and stores it as a String in the 'statusHash' Hashtable:
> {code}
> statusHash.put("bytesTransferred", Long.toBinaryString(status.getBytesTransferred()));
> {code}
> But when the value is retrieved from statusHash in method getFileTransferStatusFromXmlRpc, it isn't assumed to be a binary number:
> {code}
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> {code}
> For large files, the binary representation exceeds the capacity of a long. In the example above, the input string "11100111001101000000000000" converted directly to a long (i.e. treated as a decimal) will exceed the maximum size of a long (9,223,372,036,854,775,807).
> A simple fix for this would be to add a radix argument to Long.parseLong in method getFileTransferStatusFromXmlRpc, as follows:
> {code}
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString(), 2));{code}
> I tested this out on a large file (over 100MB) and it seemed to solve the problem. Would this be an acceptable fix? I'll attach a patch to this issue for reference.
> An alternative solution would be not to store a binary representation in the statusHash Hashtable. But I'm assuming there's a reason why it's converted to binary for the hash.
> [1] http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductTransferServlet.java
> [2] http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OODT-483) NumberFormatException when using RSS
service to view transfers for large files
Posted by "Ross Laidlaw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ross Laidlaw updated OODT-483:
------------------------------
Attachment: OODT-483.rlaidlaw.2012-08-06.patch.txt
Here's a simple patch that adds a radix argument to the Long.parseLong() call in method getFileTransferStatusFromXmlRpc. The second argument is the radix argument and it has a value of 2. This specifies that the first argument is in binary (base 2) format.
> NumberFormatException when using RSS service to view transfers for large files
> ------------------------------------------------------------------------------
>
> Key: OODT-483
> URL: https://issues.apache.org/jira/browse/OODT-483
> Project: OODT
> Issue Type: Bug
> Components: file manager
> Reporter: Ross Laidlaw
> Assignee: Ross Laidlaw
> Priority: Minor
> Labels: gsoc2012
> Fix For: 0.5
>
> Attachments: OODT-483.rlaidlaw.2012-08-06.patch.txt
>
>
> I used File Manager to ingest a large file and then attempted to view the progress of the transfer using the 'cas-product' RSS web application (RSSProductTransferServlet [1]). This caused a NumberFormatException as follows:
> {code}
> java.lang.NumberFormatException: For input string: "11100111001101000000000000"
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> java.lang.Long.parseLong(Long.java:422)
> java.lang.Long.parseLong(Long.java:468)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusFromXmlRpc(XmlRpcStructFactory.java:86)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusesFromXmlRpc(XmlRpcStructFactory.java:112)
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.getCurrentFileTransfers(XmlRpcFileManagerClient.java:398)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doIt(RSSProductTransferServlet.java:151)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doGet(RSSProductTransferServlet.java:138)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
> {code}
> From further testing I found that this exception occurs for files approximately 10MB in size or larger.
> This exception occurs because the XmlRpcStructFactory class is attempting to set the value of a long variable using a binary representation (i.e. treating the binary number as a decimal number). The size of the binary representation exceeds the maximum capacity of the long because it is not being converted to a decimal representation.
> I traced the problem to the following methods in class XmlRpcStructFactory [2]:
> {code}
> public static Hashtable<String, Object> getXmlRpcFileTransferStatus(FileTransferStatus status)
> {
> Hashtable<String, Object> statusHash = new Hashtable<String, Object>();
> statusHash.put("bytesTransferred",Long.toBinaryString(status.getBytesTransferred()));
> statusHash.put("parentProduct", getXmlRpcProduct(status.getParentProduct()));
> statusHash.put("fileRef", getXmlRpcReference(status.getFileRef()));
> return statusHash;
> }
> public static FileTransferStatus getFileTransferStatusFromXmlRpc(Hashtable<String, Object> statusHash)
> {
> FileTransferStatus status = new FileTransferStatus();
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> status.setParentProduct(getProductFromXmlRpc((Hashtable<String, Object>) statusHash.get("parentProduct")));
> status.setFileRef(getReferenceFromXmlRpc((Hashtable<String, Object>)
> statusHash.get("fileRef")));
> return status;
> }
> {code}
> In the getXmlRpcFileTransferStatus method shown above, the following line converts the value returned by 'status.getBytesTransferred()' to a binary representation and stores it as a String in the 'statusHash' Hashtable:
> {code}
> statusHash.put("bytesTransferred", Long.toBinaryString(status.getBytesTransferred()));
> {code}
> But when the value is retrieved from statusHash in method getFileTransferStatusFromXmlRpc, it isn't assumed to be a binary number:
> {code}
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> {code}
> For large files, the binary representation exceeds the capacity of a long. In the example above, the input string "11100111001101000000000000" converted directly to a long (i.e. treated as a decimal) will exceed the maximum size of a long (9,223,372,036,854,775,807).
> A simple fix for this would be to add a radix argument to Long.parseLong in method getFileTransferStatusFromXmlRpc, as follows:
> {code}
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString(), 2));{code}
> I tested this out on a large file (over 100MB) and it seemed to solve the problem. Would this be an acceptable fix? I'll attach a patch to this issue for reference.
> An alternative solution would be not to store a binary representation in the statusHash Hashtable. But I'm assuming there's a reason why it's converted to binary for the hash.
> [1] http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductTransferServlet.java
> [2] http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-483) NumberFormatException when using RSS
service to view transfers for large files
Posted by "Chris A. Mattmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429154#comment-13429154 ]
Chris A. Mattmann commented on OODT-483:
----------------------------------------
Ross, +1 this makes a ton of sense. Feel free to use Review Board http://reviews.apache.org/ to post up a review and link it to this JIRA issue. Brian Foster, Ricky, Sheryl, Tom, Paul R. and me and others are all active consumers of Review Board reviews, so that would be a great thing.
Review Board or not, +1 to commit this patch and this is a critical fix.
Thanks!
> NumberFormatException when using RSS service to view transfers for large files
> ------------------------------------------------------------------------------
>
> Key: OODT-483
> URL: https://issues.apache.org/jira/browse/OODT-483
> Project: OODT
> Issue Type: Bug
> Components: file manager
> Reporter: Ross Laidlaw
> Assignee: Ross Laidlaw
> Priority: Minor
> Labels: gsoc2012
> Fix For: 0.5
>
> Attachments: OODT-483.rlaidlaw.2012-08-06.patch.txt
>
>
> I used File Manager to ingest a large file and then attempted to view the progress of the transfer using the 'cas-product' RSS web application (RSSProductTransferServlet [1]). This caused a NumberFormatException as follows:
> {code}
> java.lang.NumberFormatException: For input string: "11100111001101000000000000"
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> java.lang.Long.parseLong(Long.java:422)
> java.lang.Long.parseLong(Long.java:468)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusFromXmlRpc(XmlRpcStructFactory.java:86)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusesFromXmlRpc(XmlRpcStructFactory.java:112)
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.getCurrentFileTransfers(XmlRpcFileManagerClient.java:398)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doIt(RSSProductTransferServlet.java:151)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doGet(RSSProductTransferServlet.java:138)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
> {code}
> From further testing I found that this exception occurs for files approximately 10MB in size or larger.
> This exception occurs because the XmlRpcStructFactory class is attempting to set the value of a long variable using a binary representation (i.e. treating the binary number as a decimal number). The size of the binary representation exceeds the maximum capacity of the long because it is not being converted to a decimal representation.
> I traced the problem to the following methods in class XmlRpcStructFactory [2]:
> {code}
> public static Hashtable<String, Object> getXmlRpcFileTransferStatus(FileTransferStatus status)
> {
> Hashtable<String, Object> statusHash = new Hashtable<String, Object>();
> statusHash.put("bytesTransferred",Long.toBinaryString(status.getBytesTransferred()));
> statusHash.put("parentProduct", getXmlRpcProduct(status.getParentProduct()));
> statusHash.put("fileRef", getXmlRpcReference(status.getFileRef()));
> return statusHash;
> }
> public static FileTransferStatus getFileTransferStatusFromXmlRpc(Hashtable<String, Object> statusHash)
> {
> FileTransferStatus status = new FileTransferStatus();
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> status.setParentProduct(getProductFromXmlRpc((Hashtable<String, Object>) statusHash.get("parentProduct")));
> status.setFileRef(getReferenceFromXmlRpc((Hashtable<String, Object>)
> statusHash.get("fileRef")));
> return status;
> }
> {code}
> In the getXmlRpcFileTransferStatus method shown above, the following line converts the value returned by 'status.getBytesTransferred()' to a binary representation and stores it as a String in the 'statusHash' Hashtable:
> {code}
> statusHash.put("bytesTransferred", Long.toBinaryString(status.getBytesTransferred()));
> {code}
> But when the value is retrieved from statusHash in method getFileTransferStatusFromXmlRpc, it isn't assumed to be a binary number:
> {code}
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> {code}
> For large files, the binary representation exceeds the capacity of a long. In the example above, the input string "11100111001101000000000000" converted directly to a long (i.e. treated as a decimal) will exceed the maximum size of a long (9,223,372,036,854,775,807).
> A simple fix for this would be to add a radix argument to Long.parseLong in method getFileTransferStatusFromXmlRpc, as follows:
> {code}
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString(), 2));{code}
> I tested this out on a large file (over 100MB) and it seemed to solve the problem. Would this be an acceptable fix? I'll attach a patch to this issue for reference.
> An alternative solution would be not to store a binary representation in the statusHash Hashtable. But I'm assuming there's a reason why it's converted to binary for the hash.
> [1] http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductTransferServlet.java
> [2] http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OODT-483) NumberFormatException when using RSS
service to view transfers for large files
Posted by "Ross Laidlaw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ross Laidlaw resolved OODT-483.
-------------------------------
Resolution: Fixed
Committed change as described in patch #1 to r1370575.
> NumberFormatException when using RSS service to view transfers for large files
> ------------------------------------------------------------------------------
>
> Key: OODT-483
> URL: https://issues.apache.org/jira/browse/OODT-483
> Project: OODT
> Issue Type: Bug
> Components: file manager
> Reporter: Ross Laidlaw
> Assignee: Ross Laidlaw
> Priority: Minor
> Labels: gsoc2012
> Fix For: 0.5
>
> Attachments: OODT-483.rlaidlaw.2012-08-06.patch.1.txt, OODT-483.rlaidlaw.2012-08-06.patch.2.txt
>
>
> I used File Manager to ingest a large file and then attempted to view the progress of the transfer using the 'cas-product' RSS web application (RSSProductTransferServlet [1]). This caused a NumberFormatException as follows:
> {code}
> java.lang.NumberFormatException: For input string: "11100111001101000000000000"
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> java.lang.Long.parseLong(Long.java:422)
> java.lang.Long.parseLong(Long.java:468)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusFromXmlRpc(XmlRpcStructFactory.java:86)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusesFromXmlRpc(XmlRpcStructFactory.java:112)
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.getCurrentFileTransfers(XmlRpcFileManagerClient.java:398)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doIt(RSSProductTransferServlet.java:151)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doGet(RSSProductTransferServlet.java:138)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
> {code}
> From further testing I found that this exception occurs for files approximately 10MB in size or larger.
> This exception occurs because the XmlRpcStructFactory class is attempting to set the value of a long variable using a binary representation (i.e. treating the binary number as a decimal number). The size of the binary representation exceeds the maximum capacity of the long because it is not being converted to a decimal representation.
> I traced the problem to the following methods in class XmlRpcStructFactory [2]:
> {code}
> public static Hashtable<String, Object> getXmlRpcFileTransferStatus(FileTransferStatus status)
> {
> Hashtable<String, Object> statusHash = new Hashtable<String, Object>();
> statusHash.put("bytesTransferred",Long.toBinaryString(status.getBytesTransferred()));
> statusHash.put("parentProduct", getXmlRpcProduct(status.getParentProduct()));
> statusHash.put("fileRef", getXmlRpcReference(status.getFileRef()));
> return statusHash;
> }
> public static FileTransferStatus getFileTransferStatusFromXmlRpc(Hashtable<String, Object> statusHash)
> {
> FileTransferStatus status = new FileTransferStatus();
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> status.setParentProduct(getProductFromXmlRpc((Hashtable<String, Object>) statusHash.get("parentProduct")));
> status.setFileRef(getReferenceFromXmlRpc((Hashtable<String, Object>)
> statusHash.get("fileRef")));
> return status;
> }
> {code}
> In the getXmlRpcFileTransferStatus method shown above, the following line converts the value returned by 'status.getBytesTransferred()' to a binary representation and stores it as a String in the 'statusHash' Hashtable:
> {code}
> statusHash.put("bytesTransferred", Long.toBinaryString(status.getBytesTransferred()));
> {code}
> But when the value is retrieved from statusHash in method getFileTransferStatusFromXmlRpc, it isn't assumed to be a binary number:
> {code}
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> {code}
> For large files, the binary representation exceeds the capacity of a long. In the example above, the input string "11100111001101000000000000" converted directly to a long (i.e. treated as a decimal) will exceed the maximum size of a long (9,223,372,036,854,775,807).
> A simple fix for this would be to add a radix argument to Long.parseLong in method getFileTransferStatusFromXmlRpc, as follows:
> {code}
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString(), 2));{code}
> I tested this out on a large file (over 100MB) and it seemed to solve the problem. Would this be an acceptable fix? I'll attach a patch to this issue for reference.
> An alternative solution would be not to store a binary representation in the statusHash Hashtable. But I'm assuming there's a reason why it's converted to binary for the hash.
> [1] http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductTransferServlet.java
> [2] http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira