[jira] [Commented] (SYSTEMML-1467) optimizer changes the output of table funciton

["Imran Younus (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SYSTEMML-1467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15958109#comment-15958109 ] 

Imran Younus commented on SYSTEMML-1467:
----------------------------------------

[~mboehm7] [~niketanpansare] [~nakul02]

> optimizer changes the output of table funciton
> ----------------------------------------------
>
>                 Key: SYSTEMML-1467
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1467
>             Project: SystemML
>          Issue Type: Bug
>          Components: Runtime
>            Reporter: Imran Younus
>
> Following is small code snippet which is part of an r4ml dml script.
> {code}
> reorder_matrix = function(
>   matrix[double] X, # the input matrix, we have to account for extra beta0 for the intercept
>   matrix[double] B, # beta
>   matrix[double] S  # Selected
> ) return (matrix[double] Y) {
>   S = t(S);
>   whatever = ncol(X)
>   num_empty_B = ncol(X) - nrow(B);
>   if (num_empty_B < 0) {
>     stop("Error: unable to re-order the matrix. Reason: B more than matrix X");
>   }
>   P = table(seq(1, nrow(S)), S, ncol(X), ncol(X));
>   #P = table(seq(1, nrow(S)), S, whatever, whatever);
>   Y = t(P) %*% B;
> }
> X = rand(rows=5000, cols=11, min=0, max=10, pdf="uniform")
> Beta = matrix("-0.048 -1.918 0.000 0.024 0.000 0.005 -0.001 1.842 0.216 0.483 0.361", rows=11, cols=1)
> Selected = matrix("11.000 8.000 2.000 4.000 1.000 6.000 10.000 7.000 9.000", rows=1, cols=9)
> B = reorder_matrix(X, Beta, Selected)
> print(toString(B))
> {code}
> So this works without any problem. Matrix {{P}} is {{11x11}}, with last two rows filled with zeros. BTW, I'm running it in standalone mode.
> Now, if I change line 14 to this:
> {code}
> P = table(seq(1, nrow(S)), S, whatever, whatever);
> {code}
> then nothing should change because {{whatever = ncol(X)}}. But, this breaks because now the shape of {{P}} changes. The last two rows are removed because these are all zeros.
> Here is the explain for this case:
> {code}
> 17/04/05 17:38:40 INFO api.DMLScript: EXPLAIN (RUNTIME):
> # Memory Budget local/remote = 5496MB/140MB/140MB
> # Degree of Parallelism (vcores) local/remote = 8/1/1
> PROGRAM ( size CP/MR = 0/0 )
> --MAIN PROGRAM
> ----GENERIC (lines 20-24) [recompile=false]
> ------CP createvar _mVar0 scratch_space//_p26900_127.0.1.1//_t0/temp0 true MATRIX binaryblock 11 1 1000 1000 -1 copy
> ------CP sinit 11 1 1000 1000 -0.048 -1.918 0.000 0.024 0.000 0.005 -0.001 1.842 0.216 0.483 0.361 _mVar0.MATRIX.DOUBLE
> ------CP createvar _mVar1 scratch_space//_p26900_127.0.1.1//_t0/temp1 true MATRIX binaryblock 9 1 1000 1000 -1 copy
> ------CP sinit 9 1 1000 1000 11.000 8.000 2.000 4.000 1.000 6.000 10.000 7.000 9.000 _mVar1.MATRIX.DOUBLE
> ------CP createvar _mVar2 scratch_space//_p26900_127.0.1.1//_t0/temp2 true MATRIX binarycell 1 11 -1 -1 -1 copy
> ------CP r' _mVar0.MATRIX.DOUBLE _mVar2.MATRIX.DOUBLE 8
> ------CP rmvar _mVar0
> ------CP createvar _mVar3 scratch_space//_p26900_127.0.1.1//_t0/temp3 true MATRIX binarycell 9 11 -1 -1 -1 copy
> ------CP rexpand cast=true max=11 ignore=false dir=cols target=_mVar1 _mVar3.MATRIX.DOUBLE
> ------CP rmvar _mVar1
> ------CP createvar _mVar4 scratch_space//_p26900_127.0.1.1//_t0/temp4 true MATRIX binarycell 1 11 -1 -1 -1 copy
> ------CP ba+* _mVar2.MATRIX.DOUBLE _mVar3.MATRIX.DOUBLE _mVar4.MATRIX.DOUBLE 8
> ------CP rmvar _mVar2
> ------CP rmvar _mVar3
> ------CP createvar _mVar5 scratch_space//_p26900_127.0.1.1//_t0/temp5 true MATRIX binarycell 11 1 -1 -1 -1 copy
> ------CP r' _mVar4.MATRIX.DOUBLE _mVar5.MATRIX.DOUBLE 8
> ------CP rmvar _mVar4
> ------CP toString target=_mVar5 _Var6.SCALAR.STRING
> ------CP rmvar _mVar5
> ------CP print _Var6.SCALAR.STRING.false _Var7.SCALAR.STRING
> ------CP rmvar _Var6
> ------CP rmvar _Var7
> {code}
> Now, lets revert back the definition of {{P}} to
> {code}
> P = table(seq(1, nrow(S)), S, ncol(X), ncol(X));
> {code}
> If I comment out the {{if}} statement, then it breaks again because of the same reason.
> Off course, this is the "optimization" at work here. If I set the {{opetlevel}} to 0 or 1 in {{SystemML-config.xml}}, then it doesn't break.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)