You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Travis Crawford (JIRA)" <ji...@apache.org> on 2012/05/30 19:36:24 UTC
[jira] [Created] (PIG-2730) TFileStorage getStatistics incorrectly
throws an exception instead of returning null
Travis Crawford created PIG-2730:
------------------------------------
Summary: TFileStorage getStatistics incorrectly throws an exception instead of returning null
Key: PIG-2730
URL: https://issues.apache.org/jira/browse/PIG-2730
Project: Pig
Issue Type: Bug
Reporter: Travis Crawford
Assignee: Travis Crawford
[TFileStorage.java|http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/io/TFileStorage.java] throws an exception in getStatistics:
{code}
@Override
public ResourceStatistics getStatistics(String location, Job job) throws IOException {
throw new UnsupportedOperationException();
}
{code}
However the interface for [LoadMetadata.java|http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/LoadMetadata.java] states null should be returned if statistics are not available:
{code}
/**
* Get statistics about the data to be loaded. If no statistics are
* available, then null should be returned. If the implementing class also extends
* {@link LoadFunc}, then {@link LoadFunc#setLocation(String, org.apache.hadoop.mapreduce.Job)}
* is guaranteed to be called before this method.
* @param location Location as returned by
* {@link LoadFunc#relativeToAbsolutePath(String, org.apache.hadoop.fs.Path)}
* @param job The {@link Job} object - this should be used only to obtain
* cluster properties through {@link Job#getConfiguration()} and not to set/query
* any runtime job information.
* @return statistics about the data to be loaded. If no statistics are
* available, then null should be returned.
* @throws IOException if an exception occurs while retrieving statistics
*/
ResourceStatistics getStatistics(String location, Job job)
throws IOException;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2730) TFileStorage getStatistics
incorrectly throws an exception instead of returning null
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286057#comment-13286057 ]
Daniel Dai commented on PIG-2730:
---------------------------------
Yes, similar to BinStorage, should return null instead.
> TFileStorage getStatistics incorrectly throws an exception instead of returning null
> ------------------------------------------------------------------------------------
>
> Key: PIG-2730
> URL: https://issues.apache.org/jira/browse/PIG-2730
> Project: Pig
> Issue Type: Bug
> Reporter: Travis Crawford
> Assignee: Travis Crawford
>
> [TFileStorage.java|http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/io/TFileStorage.java] throws an exception in getStatistics:
> {code}
> @Override
> public ResourceStatistics getStatistics(String location, Job job) throws IOException {
> throw new UnsupportedOperationException();
> }
> {code}
> However the interface for [LoadMetadata.java|http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/LoadMetadata.java] states null should be returned if statistics are not available:
> {code}
> /**
> * Get statistics about the data to be loaded. If no statistics are
> * available, then null should be returned. If the implementing class also extends
> * {@link LoadFunc}, then {@link LoadFunc#setLocation(String, org.apache.hadoop.mapreduce.Job)}
> * is guaranteed to be called before this method.
> * @param location Location as returned by
> * {@link LoadFunc#relativeToAbsolutePath(String, org.apache.hadoop.fs.Path)}
> * @param job The {@link Job} object - this should be used only to obtain
> * cluster properties through {@link Job#getConfiguration()} and not to set/query
> * any runtime job information.
> * @return statistics about the data to be loaded. If no statistics are
> * available, then null should be returned.
> * @throws IOException if an exception occurs while retrieving statistics
> */
> ResourceStatistics getStatistics(String location, Job job)
> throws IOException;
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2730) TFileStorage getStatistics incorrectly
throws an exception instead of returning null
Posted by "Travis Crawford (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Travis Crawford updated PIG-2730:
---------------------------------
Attachment: PIG-2730.1.patch
> TFileStorage getStatistics incorrectly throws an exception instead of returning null
> ------------------------------------------------------------------------------------
>
> Key: PIG-2730
> URL: https://issues.apache.org/jira/browse/PIG-2730
> Project: Pig
> Issue Type: Bug
> Reporter: Travis Crawford
> Assignee: Travis Crawford
> Attachments: PIG-2730.1.patch
>
>
> [TFileStorage.java|http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/io/TFileStorage.java] throws an exception in getStatistics:
> {code}
> @Override
> public ResourceStatistics getStatistics(String location, Job job) throws IOException {
> throw new UnsupportedOperationException();
> }
> {code}
> However the interface for [LoadMetadata.java|http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/LoadMetadata.java] states null should be returned if statistics are not available:
> {code}
> /**
> * Get statistics about the data to be loaded. If no statistics are
> * available, then null should be returned. If the implementing class also extends
> * {@link LoadFunc}, then {@link LoadFunc#setLocation(String, org.apache.hadoop.mapreduce.Job)}
> * is guaranteed to be called before this method.
> * @param location Location as returned by
> * {@link LoadFunc#relativeToAbsolutePath(String, org.apache.hadoop.fs.Path)}
> * @param job The {@link Job} object - this should be used only to obtain
> * cluster properties through {@link Job#getConfiguration()} and not to set/query
> * any runtime job information.
> * @return statistics about the data to be loaded. If no statistics are
> * available, then null should be returned.
> * @throws IOException if an exception occurs while retrieving statistics
> */
> ResourceStatistics getStatistics(String location, Job job)
> throws IOException;
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2730) TFileStorage getStatistics
incorrectly throws an exception instead of returning null
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285978#comment-13285978 ]
Dmitriy V. Ryaboy commented on PIG-2730:
----------------------------------------
Yep, implementation is wrong. Travis, can you submit a patch for this?
> TFileStorage getStatistics incorrectly throws an exception instead of returning null
> ------------------------------------------------------------------------------------
>
> Key: PIG-2730
> URL: https://issues.apache.org/jira/browse/PIG-2730
> Project: Pig
> Issue Type: Bug
> Reporter: Travis Crawford
> Assignee: Travis Crawford
>
> [TFileStorage.java|http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/io/TFileStorage.java] throws an exception in getStatistics:
> {code}
> @Override
> public ResourceStatistics getStatistics(String location, Job job) throws IOException {
> throw new UnsupportedOperationException();
> }
> {code}
> However the interface for [LoadMetadata.java|http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/LoadMetadata.java] states null should be returned if statistics are not available:
> {code}
> /**
> * Get statistics about the data to be loaded. If no statistics are
> * available, then null should be returned. If the implementing class also extends
> * {@link LoadFunc}, then {@link LoadFunc#setLocation(String, org.apache.hadoop.mapreduce.Job)}
> * is guaranteed to be called before this method.
> * @param location Location as returned by
> * {@link LoadFunc#relativeToAbsolutePath(String, org.apache.hadoop.fs.Path)}
> * @param job The {@link Job} object - this should be used only to obtain
> * cluster properties through {@link Job#getConfiguration()} and not to set/query
> * any runtime job information.
> * @return statistics about the data to be loaded. If no statistics are
> * available, then null should be returned.
> * @throws IOException if an exception occurs while retrieving statistics
> */
> ResourceStatistics getStatistics(String location, Job job)
> throws IOException;
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2730) TFileStorage getStatistics incorrectly
throws an exception instead of returning null
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2730:
----------------------------
Resolution: Fixed
Fix Version/s: 0.10.1
0.11
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Patch committed to 0.10/trunk. Thanks Travis!
> TFileStorage getStatistics incorrectly throws an exception instead of returning null
> ------------------------------------------------------------------------------------
>
> Key: PIG-2730
> URL: https://issues.apache.org/jira/browse/PIG-2730
> Project: Pig
> Issue Type: Bug
> Reporter: Travis Crawford
> Assignee: Travis Crawford
> Fix For: 0.11, 0.10.1
>
> Attachments: PIG-2730.1.patch
>
>
> [TFileStorage.java|http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/io/TFileStorage.java] throws an exception in getStatistics:
> {code}
> @Override
> public ResourceStatistics getStatistics(String location, Job job) throws IOException {
> throw new UnsupportedOperationException();
> }
> {code}
> However the interface for [LoadMetadata.java|http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/LoadMetadata.java] states null should be returned if statistics are not available:
> {code}
> /**
> * Get statistics about the data to be loaded. If no statistics are
> * available, then null should be returned. If the implementing class also extends
> * {@link LoadFunc}, then {@link LoadFunc#setLocation(String, org.apache.hadoop.mapreduce.Job)}
> * is guaranteed to be called before this method.
> * @param location Location as returned by
> * {@link LoadFunc#relativeToAbsolutePath(String, org.apache.hadoop.fs.Path)}
> * @param job The {@link Job} object - this should be used only to obtain
> * cluster properties through {@link Job#getConfiguration()} and not to set/query
> * any runtime job information.
> * @return statistics about the data to be loaded. If no statistics are
> * available, then null should be returned.
> * @throws IOException if an exception occurs while retrieving statistics
> */
> ResourceStatistics getStatistics(String location, Job job)
> throws IOException;
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2730) TFileStorage getStatistics incorrectly
throws an exception instead of returning null
Posted by "Travis Crawford (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Travis Crawford updated PIG-2730:
---------------------------------
Status: Patch Available (was: Open)
> TFileStorage getStatistics incorrectly throws an exception instead of returning null
> ------------------------------------------------------------------------------------
>
> Key: PIG-2730
> URL: https://issues.apache.org/jira/browse/PIG-2730
> Project: Pig
> Issue Type: Bug
> Reporter: Travis Crawford
> Assignee: Travis Crawford
> Attachments: PIG-2730.1.patch
>
>
> [TFileStorage.java|http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/io/TFileStorage.java] throws an exception in getStatistics:
> {code}
> @Override
> public ResourceStatistics getStatistics(String location, Job job) throws IOException {
> throw new UnsupportedOperationException();
> }
> {code}
> However the interface for [LoadMetadata.java|http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/LoadMetadata.java] states null should be returned if statistics are not available:
> {code}
> /**
> * Get statistics about the data to be loaded. If no statistics are
> * available, then null should be returned. If the implementing class also extends
> * {@link LoadFunc}, then {@link LoadFunc#setLocation(String, org.apache.hadoop.mapreduce.Job)}
> * is guaranteed to be called before this method.
> * @param location Location as returned by
> * {@link LoadFunc#relativeToAbsolutePath(String, org.apache.hadoop.fs.Path)}
> * @param job The {@link Job} object - this should be used only to obtain
> * cluster properties through {@link Job#getConfiguration()} and not to set/query
> * any runtime job information.
> * @return statistics about the data to be loaded. If no statistics are
> * available, then null should be returned.
> * @throws IOException if an exception occurs while retrieving statistics
> */
> ResourceStatistics getStatistics(String location, Job job)
> throws IOException;
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira