[jira] [Commented] (SYSTEMML-1662) Extended HOP DAG validator

["Dylan Hutchison (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SYSTEMML-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16042245#comment-16042245 ] 

Dylan Hutchison commented on SYSTEMML-1662:
-------------------------------------------

Hi folks, I started working on this issue as a way to get familiar with SystemML. For the extension of the HOP DAG Validator to check the *arity* (number of input children hops) of each hop, I have a question.  Some of the operators I am uncertain as to their arity. Do the following operators have a fixed arity or a variable arity?

* DataOp -  I think this should be a zeroary operator (no inputs) but I see on line 122 and 173 that DataOp adds operators to its input list based on the input parameters passed to its constructor.
* DataGenOp, ParameterizedBuiltinOp - similar to DataOp. These operators take parameters. Are parameters the same as normal inputs to a hop?
* ReorgOp - marked as a unary operator, which makes sense as it re-organizes the entry of a single input. However line 85 adds a variable number of input hops. Is this intended?
* TernaryOp - ought to have 3 inputs but it has a constructor that takes 5 inputs. It seems the last two inputs have been used to hold output dimensions.
* SpoofFusedOp - I guess this is variable arity?

> Extended HOP DAG validator
> --------------------------
>
>                 Key: SYSTEMML-1662
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1662
>             Project: SystemML
>          Issue Type: Sub-task
>          Components: Compiler
>            Reporter: Matthias Boehm
>              Labels: beginner
>             Fix For: SystemML 1.0
>
>
> This task aims to extend the existing HOP DAG validator (see {{org.apache.sysml.hops.rewrite.HopDagValidator}}, which can be enabled via {{org.apache.sysml.hops.rewriteProgramRewriter.CHECK}}) in various ways in order to provide better developer tooling for checking the correctness of new and existing rewrites.
> So far, this validator, checks only for:
> * Correct parent node linking
> * Correct child node linking
> * Non-empty children (for all hops other than {{DataOp}} and {{LiteralOp}})
> Possible extensions include (but are not limited to):
> * Correct HOP output data types
> * Correct HOP output value types
> * Correct number of expected child nodes
> * Correct output size information wrt input sizes
> * Correct visit status 
> These extensions would be very useful for multiple reasons. First, they would detect rewrite issues early on in the development process. This is important because rewrite issues usually lead to strange and non-obvious behavior of real application scripts. Second, the HOP DAG validator provides a systematic way of debugging optimizer issues. The intended future workflow is as follows:
> * 1. Disable rewrites via optimization level 1 to determine if rewrites are the issue.
> * 2. Use the extended {{HopDagValidator}} validator to find the source of corruption.
> * 3. If (2) did not find the issue, resort to low-level debugging, and extend the {{HopDagValidator}} to capture the root cause of the issue.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)