You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Namit Jain (JIRA)" <ji...@apache.org> on 2009/07/23 23:15:14 UTC
[jira] Created: (HIVE-680) add user constraints
add user constraints
---------------------
Key: HIVE-680
URL: https://issues.apache.org/jira/browse/HIVE-680
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Reporter: Namit Jain
Many times, because of a few bad input rows, the whole job fails and it takes a long time to debug those.
It might be very useful to add some constraints, which can be checked while reading the data.
An option can be added to ignore configurable number of bad rows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-680) add user constraints
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12734794#action_12734794 ]
Namit Jain commented on HIVE-680:
---------------------------------
There was a internal problem recently:
One of internal custom mapper scripts was trying to split the string: probably it expected to be in some fixed format, say a:b:c
It was not, and the process died.
The user was perfectly happy to ignore such records.
> add user constraints
> ---------------------
>
> Key: HIVE-680
> URL: https://issues.apache.org/jira/browse/HIVE-680
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Namit Jain
>
> Many times, because of a few bad input rows, the whole job fails and it takes a long time to debug those.
> It might be very useful to add some constraints, which can be checked while reading the data.
> An option can be added to ignore configurable number of bad rows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-680) add user constraints
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12734798#action_12734798 ]
Ashish Thusoo commented on HIVE-680:
------------------------------------
Should that not be handled within the user scripts and if not can the user not just have a where clause which can ignore strings that do not fit this format?
> add user constraints
> ---------------------
>
> Key: HIVE-680
> URL: https://issues.apache.org/jira/browse/HIVE-680
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Namit Jain
>
> Many times, because of a few bad input rows, the whole job fails and it takes a long time to debug those.
> It might be very useful to add some constraints, which can be checked while reading the data.
> An option can be added to ignore configurable number of bad rows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-680) add user constraints
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12734789#action_12734789 ]
Ashish Thusoo commented on HIVE-680:
------------------------------------
Actually constraint checking is usually done on inserting and loading rows.
Constraints on reading data are just views, no?
Any concrete example you have in mind.
> add user constraints
> ---------------------
>
> Key: HIVE-680
> URL: https://issues.apache.org/jira/browse/HIVE-680
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Namit Jain
>
> Many times, because of a few bad input rows, the whole job fails and it takes a long time to debug those.
> It might be very useful to add some constraints, which can be checked while reading the data.
> An option can be added to ignore configurable number of bad rows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.