You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Vivek Padmanabhan (Created) (JIRA)" <ji...@apache.org> on 2012/02/08 17:05:59 UTC
[jira] [Created] (PIG-2519) Introduce a mechanism to notify if a
larger relation is given for replicated join
Introduce a mechanism to notify if a larger relation is given for replicated join
----------------------------------------------------------------------------------
Key: PIG-2519
URL: https://issues.apache.org/jira/browse/PIG-2519
Project: Pig
Issue Type: Improvement
Affects Versions: 0.9.1
Reporter: Vivek Padmanabhan
In a replicated join if a huge relation is mentioned wrongly on the right side, the job may start filling up the disks and eventually impact the overall cluster.
Furthermore, the document for replicated join says,
"The small relations must be small enough to fit into main memory; if they don't, the process fails and an error is generated."
http://pig.apache.org/docs/r0.8.1/piglatin_ref1.html#Replicated+Joins
It would be nice to have a mechanism to fail fast this sort of scenarios or may be have some warning messages to notify this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira