You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@pig.apache.org by Apache Wiki <wi...@apache.org> on 2011/01/21 22:15:16 UTC

[Pig Wiki] Update of "PigErrorHandlingInScripts" by OlgaN

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.

The "PigErrorHandlingInScripts" page has been changed by OlgaN.
http://wiki.apache.org/pig/PigErrorHandlingInScripts?action=diff&rev1=6&rev2=7

--------------------------------------------------

  
   10. OK
  
+ Olga, 1/21/11
+ 
+ Glad we are discussing this! A couple of questions/comments and follow ups on discussion with Alan:
+ 
+  1. Is error handler a property of a UDF, an operator or both. If it is a property of the UDF then perhaps it can be attached to the UDF via define statement rather than to the Pig statement itself
+  2. How does FailOnThreshhold works? You will still have to run the job to completion since there is no way to get continuous global count, right?
+  3. How does this interact with error handling in Pig? In Pig we also have a way to generate and aggregate warnings. Should UDFs be allowed to do the same and rather than FailOnError just being able to report aggregated warning count so that users can decide what to do.
+  4. Related to 3, UDFs can also choose to patch invalid records with NULL values. Currently, they have no way to communicate that back other than print warnings into stderr. Perhaps this case should also be addressed in this proposal.
+