[jira] [Created] (SINGA-45) Set openblas num threads in job configuration

["wangwei (JIRA)" <ji...@apache.org>]

wangwei created SINGA-45:
----------------------------

             Summary: Set openblas num threads in job configuration
                 Key: SINGA-45
                 URL: https://issues.apache.org/jira/browse/SINGA-45
             Project: Singa
          Issue Type: New Feature
            Reporter: wangwei


SINGA uses OpenBlas to do time consuming operations like matrix multiplications. 
According to [OpenBlas's documentation|https://github.com/xianyi/OpenBLAS/wiki/faq#multi-threaded], if multiple worker threads are launched within one process, we should set OpenBlas to use single thread. 

This can be done by adding a job configuration field, e.g., openblas_num_thread, and calling openblas_set_num_threads() in the dirver (main.cc) program. 
If users do not set this field, SINGA should set it to 1 if there are multiple worker threads within one SINGA process. If there is only one worker thread, then let OpenBlas use as many threads (cores) as possible. 

In practice, better performance is achieved by launching n workers (each assigned 1/n images) than using OpenBlas with n threads. Because OpenBlas only parallelize a subset of operations. n workers can run in fully parallel.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)