[MXNet Forum] Transfer Learning Resnest269

[Duc Tran via MXNet Forum <mx...@discoursemail.com.INVALID>]

OS: Ubuntu 18.04
Mxnet: 1.6-cu101-mkl

Hi, i am beginner. I am trying to fine tune the resnest269 model by following [tutorial](https://gluon-cv.mxnet.io/build/examples_classification/transfer_learning_minc.html).
When i change the model name to `resnest269`. There is an error appear at the training phase:

>      24         l.backward()
>      25 
>      26     ->  trainer.step(batch_size)
>      27         train_loss += sum([l.mean().asscalar() for l in loss]) / len(loss)
>        
>      DeferredInitializationError: Parameter 'resnest_dense1_weight' has not been initialized yet because initialization was deferred. Actual initialization happens during the first forward pass. Please pass one batch of data through the network before accessing Parameters. You can also avoid deferred initialization by specifying in_units, num_features, etc., for network layers.

the full code here:
>     lr = 0.001
>     momentum = 0.9
>     wd = 0.0001
>     classes = 3
> 
>     finetune_net = get_model('resnest269', pretrained=True)
>     with finetune_net.name_scope():
>         finetune_net.output = nn.Dense(classes)
>     finetune_net.output.initialize(init.Xavier(), ctx = ctx)
>     finetune_net.collect_params().reset_ctx(ctx)
>         finetune_net.hybridize()
> 
>     trainer = gluon.Trainer(finetune_net.collect_params(), 'sgd', {
>                         'learning_rate': lr, 'momentum': momentum, 'wd': wd})
> 
>     metric = mx.metric.Accuracy()
>     L = gluon.loss.SoftmaxCrossEntropyLoss()
> 
>     epochs = 10
>     lr_counter = 0
>     num_batch = len(train_data)
>     lr_steps = [10, 20, 30, np.inf]
>     lr_factor = 0.75
> 
>     for epoch in tqdm(range(epochs)):
>         if epoch == lr_steps[lr_counter]:
>             trainer.set_learning_rate(trainer.learning_rate*lr_factor)
>             lr_counter += 1
> 
>     tic = time.time()
>     train_loss = 0
>     metric.reset()
>     finetune_net.hybridize()
> 
>     for i, batch in enumerate(train_data):
>         data = gluon.utils.split_and_load(batch[0], ctx_list=[ctx], batch_axis=0, even_split=False)
>         label = gluon.utils.split_and_load(batch[1], ctx_list=[ctx], batch_axis=0, even_split=False)
>         with ag.record():
>             outputs = [finetune_net(X) for X in data]
>             loss = [L(yhat, y) for yhat, y in zip(outputs, label)]
>         for l in loss:
>             l.backward()
> 
>         trainer.step(batch_size)
>         train_loss += sum([l.mean().asscalar() for l in loss]) / len(loss)
> 
>         metric.update(label, outputs)
> 
>     _, train_acc = metric.get()
>     train_loss /= num_batch
> 
>     _, val_acc = test(finetune_net, val_data, [ctx])
> 
>     print('[Epoch %d] Train-acc: %.3f, loss: %.3f | Val-acc: %.3f | time: %.1f' %
>              (epoch, train_acc, train_loss, val_acc, time.time() - tic))
> 
>     _, test_acc = test(finetune_net, test_data, ctx)
>     print('[Finished] Test-acc: %.3f' % (test_acc))

Thanks for read my topic.





---
[Visit Topic](https://discuss.mxnet.io/t/transfer-learning-resnest269/6535/1) or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.mxnet.io/email/unsubscribe/b02cf263f28c594c07a71f9e842e6f97eed1211e7628a944bde3b6291bfa3620).