You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Ashutosh Chauhan (JIRA)" <ji...@apache.org> on 2012/10/07 21:30:02 UTC
[jira] [Created] (PIG-2951) Overflow, Underflow errors
Ashutosh Chauhan created PIG-2951:
-------------------------------------
Summary: Overflow, Underflow errors
Key: PIG-2951
URL: https://issues.apache.org/jira/browse/PIG-2951
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.10.0, 0.9.2, 0.9.1, 0.9.0, 0.8.1, 0.8.0, 0.7.0, 0.6.0, 0.5.0, 0.4.0, 0.3.0, 0.2.0, 0.1.0, 0.0.0
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
With very large (or very small) integer values there is a possibility of overflow (or underflow) errors. Worse thing is instead of failing, this currently results in incorrect results being returned, thereby leaving user with no clue that some of the tuples may have wrong value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2951) Overflow, Underflow errors
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13471300#comment-13471300 ]
Ashutosh Chauhan commented on PIG-2951:
---------------------------------------
I am not sure what sql standard says for such scenarios, but atleast mysql instead of throwing exceptions does correct math and returns valid results to users.
> Overflow, Underflow errors
> --------------------------
>
> Key: PIG-2951
> URL: https://issues.apache.org/jira/browse/PIG-2951
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.0.0, 0.1.0, 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.7.0, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.9.2, 0.10.0
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: pig-2951.patch
>
>
> With very large (or very small) integer values there is a possibility of overflow (or underflow) errors. Worse thing is instead of failing, this currently results in incorrect results being returned, thereby leaving user with no clue that some of the tuples may have wrong value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2951) Overflow, Underflow errors
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472819#comment-13472819 ]
Dmitriy V. Ryaboy commented on PIG-2951:
----------------------------------------
I don't think killing whole jobs by throwing an exception on a single case of overflow is good.
How about we increment a counter instead, and use the appropriate max or min value as a result instead?
(eg, mysql in non-strict mode uses max_int if you insert 2*max+int)
> Overflow, Underflow errors
> --------------------------
>
> Key: PIG-2951
> URL: https://issues.apache.org/jira/browse/PIG-2951
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.0.0, 0.1.0, 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.7.0, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.9.2, 0.10.0
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: pig-2951.patch
>
>
> With very large (or very small) integer values there is a possibility of overflow (or underflow) errors. Worse thing is instead of failing, this currently results in incorrect results being returned, thereby leaving user with no clue that some of the tuples may have wrong value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2951) Overflow, Underflow errors
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473645#comment-13473645 ]
Jonathan Coveney commented on PIG-2951:
---------------------------------------
+1 to the idea of incrementing a counter
And if you need to work with huge numbers where precision matters, +1 the BigInteger/BigDecimal patch :)
> Overflow, Underflow errors
> --------------------------
>
> Key: PIG-2951
> URL: https://issues.apache.org/jira/browse/PIG-2951
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.0.0, 0.1.0, 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.7.0, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.9.2, 0.10.0
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: pig-2951.patch
>
>
> With very large (or very small) integer values there is a possibility of overflow (or underflow) errors. Worse thing is instead of failing, this currently results in incorrect results being returned, thereby leaving user with no clue that some of the tuples may have wrong value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2951) Overflow, Underflow errors
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated PIG-2951:
----------------------------------
Attachment: pig-2951.patch
Here is the patch which does runtime checking for overflows/underflows and throws exception in case it detects overflows/underflows.
I followed CMU's CERT Oracle Secure Coding Standard for Java https://www.securecoding.cert.org/confluence/display/java/NUM00-J.+Detect+or+prevent+integer+overflow for this. Hopefully, extra CPU cycles it will incur aren't too many.
> Overflow, Underflow errors
> --------------------------
>
> Key: PIG-2951
> URL: https://issues.apache.org/jira/browse/PIG-2951
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.0.0, 0.1.0, 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.7.0, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.9.2, 0.10.0
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: pig-2951.patch
>
>
> With very large (or very small) integer values there is a possibility of overflow (or underflow) errors. Worse thing is instead of failing, this currently results in incorrect results being returned, thereby leaving user with no clue that some of the tuples may have wrong value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira