You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Thejas M Nair (JIRA)" <ji...@apache.org> on 2011/02/09 18:13:57 UTC

[jira] Created: (PIG-1847) implement typechecking and lineage tracking on new logical plan

implement typechecking and lineage tracking on new logical plan
---------------------------------------------------------------

                 Key: PIG-1847
                 URL: https://issues.apache.org/jira/browse/PIG-1847
             Project: Pig
          Issue Type: Task
    Affects Versions: 0.8.0
            Reporter: Thejas M Nair
            Assignee: Thejas M Nair
             Fix For: 0.9.0


In 0.8, TypeCheckingVisitor works on the old logical plan, before it is translated to new logical plan. The TypeCheckingVisitor checks the types in expressions, adds casts and tracks lineage for identifying which load function's LoadCaster interface is to be used to convert bytearrays into other types. 
As the old logical plan is going away in 0.9,  this functionality needs to be added for new plan.



-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] Updated: (PIG-1847) implement typechecking and lineage tracking on new logical plan

Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thejas M Nair updated PIG-1847:
-------------------------------

    Attachment: PIG-1847.1.patch

Some test cases have not been migrated in this patch, that will be part of other patches in this jira.
There describe output has changed for the test case TestPigServer.testDescribeForeachNoSchema, that will be fixed as part of another patch.

This patch also brings some changes to pig's behavior to make it more consistent -
If the load function associated with a bytearray column that needs to be cast is not found, now it will result in a warning and not an error. Previously, in some cases this would result in an error at compile time, while in other cases it would result in a warning.


> implement typechecking and lineage tracking on new logical plan
> ---------------------------------------------------------------
>
>                 Key: PIG-1847
>                 URL: https://issues.apache.org/jira/browse/PIG-1847
>             Project: Pig
>          Issue Type: Task
>    Affects Versions: 0.8.0
>            Reporter: Thejas M Nair
>            Assignee: Thejas M Nair
>             Fix For: 0.9.0
>
>         Attachments: PIG-1847.1.patch
>
>
> In 0.8, TypeCheckingVisitor works on the old logical plan, before it is translated to new logical plan. The TypeCheckingVisitor checks the types in expressions, adds casts and tracks lineage for identifying which load function's LoadCaster interface is to be used to convert bytearrays into other types. 
> As the old logical plan is going away in 0.9,  this functionality needs to be added for new plan.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] Updated: (PIG-1847) implement typechecking and lineage tracking on new logical plan

Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thejas M Nair updated PIG-1847:
-------------------------------

    Release Note:   (was: The typechecker and lineage tracking on new plan is part of NewParser-18.patch in PIG-1618 .)

The typechecker and lineage tracking on new plan is part of NewParser-18.patch in PIG-1618 . Closing this jira.

> implement typechecking and lineage tracking on new logical plan
> ---------------------------------------------------------------
>
>                 Key: PIG-1847
>                 URL: https://issues.apache.org/jira/browse/PIG-1847
>             Project: Pig
>          Issue Type: Task
>    Affects Versions: 0.8.0
>            Reporter: Thejas M Nair
>            Assignee: Thejas M Nair
>             Fix For: 0.9.0
>
>         Attachments: PIG-1847.1.patch
>
>
> In 0.8, TypeCheckingVisitor works on the old logical plan, before it is translated to new logical plan. The TypeCheckingVisitor checks the types in expressions, adds casts and tracks lineage for identifying which load function's LoadCaster interface is to be used to convert bytearrays into other types. 
> As the old logical plan is going away in 0.9,  this functionality needs to be added for new plan.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] Resolved: (PIG-1847) implement typechecking and lineage tracking on new logical plan

Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thejas M Nair resolved PIG-1847.
--------------------------------

      Resolution: Duplicate
    Release Note: The typechecker and lineage tracking on new plan is part of NewParser-18.patch in PIG-1618 .

> implement typechecking and lineage tracking on new logical plan
> ---------------------------------------------------------------
>
>                 Key: PIG-1847
>                 URL: https://issues.apache.org/jira/browse/PIG-1847
>             Project: Pig
>          Issue Type: Task
>    Affects Versions: 0.8.0
>            Reporter: Thejas M Nair
>            Assignee: Thejas M Nair
>             Fix For: 0.9.0
>
>         Attachments: PIG-1847.1.patch
>
>
> In 0.8, TypeCheckingVisitor works on the old logical plan, before it is translated to new logical plan. The TypeCheckingVisitor checks the types in expressions, adds casts and tracks lineage for identifying which load function's LoadCaster interface is to be used to convert bytearrays into other types. 
> As the old logical plan is going away in 0.9,  this functionality needs to be added for new plan.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] Commented: (PIG-1847) implement typechecking and lineage tracking on new logical plan

Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/PIG-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992638#comment-12992638 ] 

Thejas M Nair commented on PIG-1847:
------------------------------------

More comments on PIG-1847.1.patch - 
MultiMap has changed to return List instead of Collection, because many places already assume it to be List. I had to change the cast to List in several places to get rid of javac warnings.

Unit tests and test-patch pass.

     [exec] -1 overall.
     [exec]
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec]
     [exec]     +1 tests included.  The patch appears to include 17 new or modified tests.
     [exec]
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec]
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
     [exec]
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec]
     [exec]     -1 release audit.  The applied patch generated 521 release audit warnings (more than the trunk's current 516 warnings).



> implement typechecking and lineage tracking on new logical plan
> ---------------------------------------------------------------
>
>                 Key: PIG-1847
>                 URL: https://issues.apache.org/jira/browse/PIG-1847
>             Project: Pig
>          Issue Type: Task
>    Affects Versions: 0.8.0
>            Reporter: Thejas M Nair
>            Assignee: Thejas M Nair
>             Fix For: 0.9.0
>
>         Attachments: PIG-1847.1.patch
>
>
> In 0.8, TypeCheckingVisitor works on the old logical plan, before it is translated to new logical plan. The TypeCheckingVisitor checks the types in expressions, adds casts and tracks lineage for identifying which load function's LoadCaster interface is to be used to convert bytearrays into other types. 
> As the old logical plan is going away in 0.9,  this functionality needs to be added for new plan.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] Commented: (PIG-1847) implement typechecking and lineage tracking on new logical plan

Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/PIG-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992650#comment-12992650 ] 

Thejas M Nair commented on PIG-1847:
------------------------------------

The release audit warnings are false alarms, they are pointing to javadoc api changes.


> implement typechecking and lineage tracking on new logical plan
> ---------------------------------------------------------------
>
>                 Key: PIG-1847
>                 URL: https://issues.apache.org/jira/browse/PIG-1847
>             Project: Pig
>          Issue Type: Task
>    Affects Versions: 0.8.0
>            Reporter: Thejas M Nair
>            Assignee: Thejas M Nair
>             Fix For: 0.9.0
>
>         Attachments: PIG-1847.1.patch
>
>
> In 0.8, TypeCheckingVisitor works on the old logical plan, before it is translated to new logical plan. The TypeCheckingVisitor checks the types in expressions, adds casts and tracks lineage for identifying which load function's LoadCaster interface is to be used to convert bytearrays into other types. 
> As the old logical plan is going away in 0.9,  this functionality needs to be added for new plan.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira