[GitHub] [incubator-mxnet] eric-haibin-lin opened a new issue #17612: Gluon block thread safety bug

[GitBox <gi...@apache.org>]

eric-haibin-lin opened a new issue #17612: Gluon block thread safety bug
URL: https://github.com/apache/incubator-mxnet/issues/17612
 
 
   ```
   import mxnet as mx
   import threading
   import queue
   import gluonnlp as nlp
   
   q = queue.Queue()
   count = 2
   net = mx.gluon.nn.Dense((10))
   net.initialize(mx.init.Uniform(), ctx=[mx.gpu(0), mx.gpu(1)])
   mx.nd.waitall()
   
   def fn(net, ctx):
       with mx.autograd.record():
           x = mx.nd.random.uniform(low=0, high=1000, shape=(10, 128), ctx=ctx)
           q.put(net(x))
       mx.nd.waitall()
       return
   
   threads = []
   for i in range(count):
       t = threading.Thread(target=fn, args=(net, mx.gpu()))
       threads.append(t)
   for i in range(count):
       threads[i].start()
   for i in range(count):
       threads[i].join()
   
   ys = []
   for i in range(count):
       ys.append(q.get())
   print(ys)
   ```
   
   Error: 
   ```
   Exception in thread Thread-2:
   Traceback (most recent call last):
     File "/home/ubuntu/src/mxnet/python/mxnet/gluon/block.py", line 1154, in forward
       params = {k: v.data(ctx) for k, v in self._reg_params.items()}
     File "/home/ubuntu/src/mxnet/python/mxnet/gluon/block.py", line 1154, in <dictcomp>
       params = {k: v.data(ctx) for k, v in self._reg_params.items()}
     File "/home/ubuntu/src/mxnet/python/mxnet/gluon/parameter.py", line 565, in data
       return self._check_and_get(self._data, ctx)
     File "/home/ubuntu/src/mxnet/python/mxnet/gluon/parameter.py", line 236, in _check_and_get
       "num_features, etc., for network layers."%(self.name))
   mxnet.gluon.parameter.DeferredInitializationError: Parameter 'dense0_weight' has not been initialized yet because initialization was deferred. Actual initialization happens during the first forward pass. Please pass one batch of data through the network before accessing Parameters. You can also avoid deferred initialization by specifying in_units, num_features, etc., for network layers.
   
   During handling of the above exception, another exception occurred:
   
   Traceback (most recent call last):
     File "/home/ubuntu/src/mxnet/python/mxnet/gluon/block.py", line 976, in _deferred_infer_shape
       self.infer_shape(*args)
     File "/home/ubuntu/src/mxnet/python/mxnet/gluon/block.py", line 1074, in infer_shape
       self._infer_attrs('infer_shape', 'shape', *args)
     File "/home/ubuntu/src/mxnet/python/mxnet/gluon/block.py", line 1058, in _infer_attrs
       inputs, out = self._get_graph(*args)
     File "/home/ubuntu/src/mxnet/python/mxnet/gluon/block.py", line 928, in _get_graph
       out = self.hybrid_forward(symbol, *grouped_inputs, **params)  # pylint: disable=no-value-for-parameter
     File "/home/ubuntu/src/mxnet/python/mxnet/gluon/block.py", line 94, in __exit__
       self._name_scope.__exit__(ptype, value, trace)
   AttributeError: 'NoneType' object has no attribute '__exit__'
   
   During handling of the above exception, another exception occurred:
   
   Traceback (most recent call last):
     File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
       self.run()
     File "/usr/lib/python3.5/threading.py", line 862, in run
       self._target(*self._args, **self._kwargs)
     File "test.py", line 29, in fn
       q.put(net(x))
     File "/home/ubuntu/src/mxnet/python/mxnet/gluon/block.py", line 696, in __call__
       out = self.forward(*args)
     File "/home/ubuntu/src/mxnet/python/mxnet/gluon/block.py", line 1156, in forward
       self._deferred_infer_shape(x, *args)
     File "/home/ubuntu/src/mxnet/python/mxnet/gluon/block.py", line 980, in _deferred_infer_shape
       raise ValueError(error_msg)
   ValueError: Deferred initialization failed because shape cannot be inferred. 'NoneType' object has no attribute '__exit__'
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-mxnet] eric-haibin-lin commented on issue #17612: Gluon block thread safety bug

[GitBox <gi...@apache.org>]

eric-haibin-lin commented on issue #17612: Gluon block thread safety bug
URL: https://github.com/apache/incubator-mxnet/issues/17612#issuecomment-590430378
 
 
   This is because self.namescope is shared and overriden: https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/gluon/block.py#L92 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services