You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by Swati Jain <sw...@aggiemail.usu.edu> on 2010/07/13 08:19:37 UTC
Add "deepCopy" in LogicalExpression
Hi,
I am working on ticket PIG -1494 (
https://issues.apache.org/jira/browse/PIG-1494 ).
While implementing this functionality (conversion of logical expression into
CNF), I need to construct the OperatorPlan for the base expressions of the
CNF. For example, given an expression "(c1 < 10) AND (c3+b3 > 10)", the CNF
form will result in expressions "(c1 < 10)" and "(c3+b3 > 10)". However,
each of these expressions would be referencing the original OperatorPlan
(that of expression "(c1 < 10) AND (c3+b3 > 10)" ) whereas they should
really be referencing their local OperatorPlan post CNF conversion.
To ensure correctness of the above approach, I am planning to add a
"deepCopy" method to LogicalExpression to create a copy of expressions. In
my opinion, "deepCopy" will be a useful construct to have in general. It
would be used as follows:
LogicalExpressionPlan logPlan = new LogicalExpressionPlan();
LogicalExpression copyExpression = origExpression->deepcopy( logPlan );
Please provide feedback if any on the above approach.
Note that I considered writing a deepCopy visitor but found that approach
flawed because a valid plan is required for a visitor to work correctly, and
in this case we need to construct that plan as we copy the expression.
Thanks
Swati
RE: Add "deepCopy" in LogicalExpression
Posted by Yan Zhou <ya...@yahoo-inc.com>.
What I used in logic expression simplification, where a DNF plan is
used, is a "logical expression proxy" that wraps up the original
expression in the original plan. But it is not translated to physical
plans, where I believe
extra care would be needed.
Yan
-----Original Message-----
From: Swati Jain [mailto:swati.j@aggiemail.usu.edu]
Sent: Monday, July 12, 2010 11:20 PM
To: pig-dev@hadoop.apache.org
Subject: Add "deepCopy" in LogicalExpression
Hi,
I am working on ticket PIG -1494 (
https://issues.apache.org/jira/browse/PIG-1494 ).
While implementing this functionality (conversion of logical expression
into
CNF), I need to construct the OperatorPlan for the base expressions of
the
CNF. For example, given an expression "(c1 < 10) AND (c3+b3 > 10)", the
CNF
form will result in expressions "(c1 < 10)" and "(c3+b3 > 10)". However,
each of these expressions would be referencing the original OperatorPlan
(that of expression "(c1 < 10) AND (c3+b3 > 10)" ) whereas they should
really be referencing their local OperatorPlan post CNF conversion.
To ensure correctness of the above approach, I am planning to add a
"deepCopy" method to LogicalExpression to create a copy of expressions.
In
my opinion, "deepCopy" will be a useful construct to have in general. It
would be used as follows:
LogicalExpressionPlan logPlan = new LogicalExpressionPlan();
LogicalExpression copyExpression = origExpression->deepcopy( logPlan );
Please provide feedback if any on the above approach.
Note that I considered writing a deepCopy visitor but found that
approach
flawed because a valid plan is required for a visitor to work correctly,
and
in this case we need to construct that plan as we copy the expression.
Thanks
Swati
Re: Add "deepCopy" in LogicalExpression
Posted by Ashutosh Chauhan <as...@gmail.com>.
You are now charting into danger zone :) I tried doing something
similar when we were trying to add similar kind of logical
optimizations to Pig but was not able to get my head around it that
time. My attempt to it is at:
https://issues.apache.org/jira/browse/PIG-1073
But then I might be too naive at that time.
Ashutosh
On Tue, Jul 13, 2010 at 10:48, Swati Jain <sw...@aggiemail.usu.edu> wrote:
> Hi Alan,
>
> By default clone creates a shallow copy of the object in the sense that it
> will create a new instance of the object but reference will be the same. Any
> change applied to anyone of the object will reflect in both.
>
> The way I am proposing deep copy will create a completely new object in the
> sense that changes made to anyone of the object will not reflect in another
> one.
> We can also override to do the same as above, however it may be better to
> use "deepCopy" since the copy semantics are explicit (since deepCopy may be
> expensive).
>
> A second important reason for the way I defined deepCopy is that I can pass
> a plan as an argument which will be updated as the expression is copied
> (through plan.add() and plan.connect() ).
>
> Please let me know what you think.
>
> Thanks,
> Swati
>
> On Tue, Jul 13, 2010 at 8:46 AM, Alan Gates <ga...@yahoo-inc.com> wrote:
>
>> How does deepCopy differ from clone?
>>
>> Alan.
>>
>>
>> On Jul 12, 2010, at 11:19 PM, Swati Jain wrote:
>>
>> Hi,
>>>
>>> I am working on ticket PIG -1494 (
>>> https://issues.apache.org/jira/browse/PIG-1494 ).
>>>
>>> While implementing this functionality (conversion of logical expression
>>> into
>>> CNF), I need to construct the OperatorPlan for the base expressions of the
>>> CNF. For example, given an expression "(c1 < 10) AND (c3+b3 > 10)", the
>>> CNF
>>> form will result in expressions "(c1 < 10)" and "(c3+b3 > 10)". However,
>>> each of these expressions would be referencing the original OperatorPlan
>>> (that of expression "(c1 < 10) AND (c3+b3 > 10)" ) whereas they should
>>> really be referencing their local OperatorPlan post CNF conversion.
>>>
>>> To ensure correctness of the above approach, I am planning to add a
>>> "deepCopy" method to LogicalExpression to create a copy of expressions. In
>>> my opinion, "deepCopy" will be a useful construct to have in general. It
>>> would be used as follows:
>>>
>>> LogicalExpressionPlan logPlan = new LogicalExpressionPlan();
>>> LogicalExpression copyExpression = origExpression->deepcopy( logPlan );
>>>
>>> Please provide feedback if any on the above approach.
>>>
>>> Note that I considered writing a deepCopy visitor but found that approach
>>> flawed because a valid plan is required for a visitor to work correctly,
>>> and
>>> in this case we need to construct that plan as we copy the expression.
>>>
>>> Thanks
>>> Swati
>>>
>>
>>
>
Re: Add "deepCopy" in LogicalExpression
Posted by Swati Jain <sw...@aggiemail.usu.edu>.
Hi Alan,
By default clone creates a shallow copy of the object in the sense that it
will create a new instance of the object but reference will be the same. Any
change applied to anyone of the object will reflect in both.
The way I am proposing deep copy will create a completely new object in the
sense that changes made to anyone of the object will not reflect in another
one.
We can also override to do the same as above, however it may be better to
use "deepCopy" since the copy semantics are explicit (since deepCopy may be
expensive).
A second important reason for the way I defined deepCopy is that I can pass
a plan as an argument which will be updated as the expression is copied
(through plan.add() and plan.connect() ).
Please let me know what you think.
Thanks,
Swati
On Tue, Jul 13, 2010 at 8:46 AM, Alan Gates <ga...@yahoo-inc.com> wrote:
> How does deepCopy differ from clone?
>
> Alan.
>
>
> On Jul 12, 2010, at 11:19 PM, Swati Jain wrote:
>
> Hi,
>>
>> I am working on ticket PIG -1494 (
>> https://issues.apache.org/jira/browse/PIG-1494 ).
>>
>> While implementing this functionality (conversion of logical expression
>> into
>> CNF), I need to construct the OperatorPlan for the base expressions of the
>> CNF. For example, given an expression "(c1 < 10) AND (c3+b3 > 10)", the
>> CNF
>> form will result in expressions "(c1 < 10)" and "(c3+b3 > 10)". However,
>> each of these expressions would be referencing the original OperatorPlan
>> (that of expression "(c1 < 10) AND (c3+b3 > 10)" ) whereas they should
>> really be referencing their local OperatorPlan post CNF conversion.
>>
>> To ensure correctness of the above approach, I am planning to add a
>> "deepCopy" method to LogicalExpression to create a copy of expressions. In
>> my opinion, "deepCopy" will be a useful construct to have in general. It
>> would be used as follows:
>>
>> LogicalExpressionPlan logPlan = new LogicalExpressionPlan();
>> LogicalExpression copyExpression = origExpression->deepcopy( logPlan );
>>
>> Please provide feedback if any on the above approach.
>>
>> Note that I considered writing a deepCopy visitor but found that approach
>> flawed because a valid plan is required for a visitor to work correctly,
>> and
>> in this case we need to construct that plan as we copy the expression.
>>
>> Thanks
>> Swati
>>
>
>
Re: Add "deepCopy" in LogicalExpression
Posted by Alan Gates <ga...@yahoo-inc.com>.
How does deepCopy differ from clone?
Alan.
On Jul 12, 2010, at 11:19 PM, Swati Jain wrote:
> Hi,
>
> I am working on ticket PIG -1494 (
> https://issues.apache.org/jira/browse/PIG-1494 ).
>
> While implementing this functionality (conversion of logical
> expression into
> CNF), I need to construct the OperatorPlan for the base expressions
> of the
> CNF. For example, given an expression "(c1 < 10) AND (c3+b3 > 10)",
> the CNF
> form will result in expressions "(c1 < 10)" and "(c3+b3 > 10)".
> However,
> each of these expressions would be referencing the original
> OperatorPlan
> (that of expression "(c1 < 10) AND (c3+b3 > 10)" ) whereas they should
> really be referencing their local OperatorPlan post CNF conversion.
>
> To ensure correctness of the above approach, I am planning to add a
> "deepCopy" method to LogicalExpression to create a copy of
> expressions. In
> my opinion, "deepCopy" will be a useful construct to have in
> general. It
> would be used as follows:
>
> LogicalExpressionPlan logPlan = new LogicalExpressionPlan();
> LogicalExpression copyExpression = origExpression-
> >deepcopy( logPlan );
>
> Please provide feedback if any on the above approach.
>
> Note that I considered writing a deepCopy visitor but found that
> approach
> flawed because a valid plan is required for a visitor to work
> correctly, and
> in this case we need to construct that plan as we copy the expression.
>
> Thanks
> Swati