[TESTS] What are the best practices for dml script testing. Thanks.

[Janardhan Pulivarthi <ja...@gmail.com>]

Hi prithvirajsen,

If a script contains the gradients, then we could compare them with
numerical gradients, as we have done in the `grad_check.dml`, file.

How can we place a test script with a main script (dml), so that we can be
confident that the script will work. (just like junit test for java files)

Thanks,
Janardhan.

Re: [TESTS] What are the best practices for dml script testing. Thanks.

[du...@gmail.com]

Hi Janardhan,

Prithvi’s answer is great.  To add a bit more about the main script, please see the `run_tests.dml` script in the `nn/test` folder: https://github.com/apache/systemml/tree/master/scripts/nn/test.  In general, `grad_check.dml` and `test.dml` contain test functions for the gradients and other types of correctness, respectively.  By convention, each test prints out the name or phrase that identifies that test, and any failure results in an additional printing of a phrase beginning with `ERROR: ....`.  Thus, when the tests are run, one can clearly see whether or not any tests have failed.  In terms of actually running the tests with a main script, the `run_tests.dml` script simply imports the other files and calls every test function.  Therefore, one only needs to manually run that file in order to run all of the `nn` tests.

At some point in the past, we ended up also creating a junit test that simply runs `run_tests.dml` and checks for the presence of any `ERROR:` messages in the output.  That way, the `nn` tests are also run daily alongside all of the classic java tests.

For the FM algorithm, the layers could be added directly into the `nn/layers` directory, and thus you could add the associated gradient tests and any other correctness tests directly into the existing `grad_check.dml` and `test.dml` files, and call those functions in `run_tests.dml`.

-Mike

--

Mike Dusenberry
GitHub: github.com/dusenberrymw
LinkedIn: linkedin.com/in/mikedusenberry

Sent from my iPhone.


> On Jan 11, 2018, at 9:35 AM, Prithviraj Sen <se...@us.ibm.com> wrote:
> 
> Hi Janardhan
> 
> SystemML allows scripts to import other scripts. So its possible for your test script to import your script and test portions of it (assuming things are appropriately function-ified).
> 
> If your question is specifically to check gradients of a certain objective, then you may want to follow the recipe of 'grad_check.dml'. In general, its a good idea to test out smaller portions (gradients or otherwise) of a large dml script in a piecemeal manner.
> 
> regards
> prithvi
> 
> 
> Janardhan Pulivarthi ---01/11/2018 09:23:39 AM---Hi prithvirajsen, If a script contains the gradients, then we could compare them with
> 
> From: Janardhan Pulivarthi <ja...@gmail.com>
> To: senp@us.ibm.com, dev@systemml.apache.org
> Date: 01/11/2018 09:23 AM
> Subject: [TESTS] What are the best practices for dml script testing. Thanks.
> 
> 
> 
> 
> Hi prithvirajsen,
> 
> If a script contains the gradients, then we could compare them with numerical gradients, as we have done in the `grad_check.dml`, file. 
> 
> How can we place a test script with a main script (dml), so that we can be confident that the script will work. (just like junit test for java files)
> 
> Thanks,
> Janardhan.
> 
> 

Re: [TESTS] What are the best practices for dml script testing. Thanks.

[Prithviraj Sen <se...@us.ibm.com>]

Hi Janardhan

SystemML allows scripts to import other scripts. So its possible for your
test script to import your script and test portions of it (assuming things
are appropriately function-ified).

If your question is specifically to check gradients of a certain objective,
then you may want to follow the recipe of 'grad_check.dml'. In general, its
a good idea to test out smaller portions (gradients or otherwise) of a
large dml script in a piecemeal manner.

regards
prithvi




From:	Janardhan Pulivarthi <ja...@gmail.com>
To:	senp@us.ibm.com, dev@systemml.apache.org
Date:	01/11/2018 09:23 AM
Subject:	[TESTS] What are the best practices for dml script testing.
            Thanks.



Hi prithvirajsen,

If a script contains the gradients, then we could compare them with
numerical gradients, as we have done in the `grad_check.dml`, file.

How can we place a test script with a main script (dml), so that we can be
confident that the script will work. (just like junit test for java files)

Thanks,
Janardhan.