You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@singa.apache.org by "wangwei (JIRA)" <ji...@apache.org> on 2015/12/03 10:34:10 UTC
[jira] [Updated] (SINGA-107) pre-trained param checkpoint version
reset problem
[ https://issues.apache.org/jira/browse/SINGA-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
wangwei updated SINGA-107:
--------------------------
Description:
When params are loaded from checkpoint files, their version numbers will be reset to 0 for fine-tuning as explained in the comments of SINGA-42.
Then if this param is used again in another model, then the version number will become 0 and this param is not regarded as pre-training param thus will require initialization, which will incur problems.
Present solution is to load this param more than one time, so that the latter loading can override the first loading, and the version number will not be 0 and this param is still regarded as pre-training param.
For example, in rbm3.conf, we write like:
checkpoint_path: "examples/rbm/rbm2/checkpoint/step6000-worker0"
checkpoint_path: "examples/rbm/rbm1/checkpoint/step6000-worker0"
in order to load w1 and b12 twice.
was:
When loading pre-training params, this version of code will reset the param version to 0. Then if this param is used again in another model, then the version number will become 0 and this param is not regarded as pre-training param thus will require initialization, which will incur problems.
Present solution is to load this param more than one time, so that the latter loading can override the first loading, and the version number will not be 0 and this param is still regarded as pre-training param.
For example, in rbm3.conf, we write like:
checkpoint_path: "examples/rbm/rbm2/checkpoint/step6000-worker0"
checkpoint_path: "examples/rbm/rbm1/checkpoint/step6000-worker0"
in order to load w1 and b12 twice.
> pre-trained param checkpoint version reset problem
> ---------------------------------------------------
>
> Key: SINGA-107
> URL: https://issues.apache.org/jira/browse/SINGA-107
> Project: Singa
> Issue Type: Bug
> Reporter: ZHAOJING
>
> When params are loaded from checkpoint files, their version numbers will be reset to 0 for fine-tuning as explained in the comments of SINGA-42.
> Then if this param is used again in another model, then the version number will become 0 and this param is not regarded as pre-training param thus will require initialization, which will incur problems.
> Present solution is to load this param more than one time, so that the latter loading can override the first loading, and the version number will not be 0 and this param is still regarded as pre-training param.
> For example, in rbm3.conf, we write like:
> checkpoint_path: "examples/rbm/rbm2/checkpoint/step6000-worker0"
> checkpoint_path: "examples/rbm/rbm1/checkpoint/step6000-worker0"
> in order to load w1 and b12 twice.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)