You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/03/03 06:14:33 UTC
[GitHub] [incubator-mxnet] hzfan opened a new pull request #17749: Fix races
in block scope and deferred_init
hzfan opened a new pull request #17749: Fix races in block scope and deferred_init
URL: https://github.com/apache/incubator-mxnet/pull/17749
## Description ##
Fix #17612
Thank @eric-haibin-lin for guidance :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] hzfan commented on a change in pull request
#17749: Fix races in block scope and deferred_init
Posted by GitBox <gi...@apache.org>.
hzfan commented on a change in pull request #17749: Fix races in block scope and deferred_init
URL: https://github.com/apache/incubator-mxnet/pull/17749#discussion_r387592912
##########
File path: python/mxnet/gluon/block.py
##########
@@ -48,8 +48,9 @@ class _BlockScope(object):
def __init__(self, block):
self._block = block
self._counter = {}
- self._old_scope = None
- self._name_scope = None
+ self._local = threading.local()
Review comment:
Thanks for sharing this. Do we have a plan to switch to contextvars in the near future? This pr focuses on the _BlockScope issue. Maybe we can switch all threading.local() to contextvars in another pr.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] hzfan commented on issue #17749: Fix races in
block scope and deferred_init
Posted by GitBox <gi...@apache.org>.
hzfan commented on issue #17749: Fix races in block scope and deferred_init
URL: https://github.com/apache/incubator-mxnet/pull/17749#issuecomment-593838223
@eric-haibin-lin Tests added. Seems that ctx is not relevant to this issue, and I just test it on cpu.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] leezu commented on a change in pull request
#17749: Fix races in block scope and deferred_init
Posted by GitBox <gi...@apache.org>.
leezu commented on a change in pull request #17749: Fix races in block scope and deferred_init
URL: https://github.com/apache/incubator-mxnet/pull/17749#discussion_r387206282
##########
File path: python/mxnet/gluon/block.py
##########
@@ -48,8 +48,9 @@ class _BlockScope(object):
def __init__(self, block):
self._block = block
self._counter = {}
- self._old_scope = None
- self._name_scope = None
+ self._local = threading.local()
Review comment:
How about using contextvars here? This will ensure the code is compatible with python coroutines.
https://docs.python.org/3/library/contextvars.html
You need to include https://pypi.org/project/contextvars/ as dependency on Python 3.5, 3.6
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] hzfan edited a comment on issue #17749: Fix races
in block scope and deferred_init
Posted by GitBox <gi...@apache.org>.
hzfan edited a comment on issue #17749: Fix races in block scope and deferred_init
URL: https://github.com/apache/incubator-mxnet/pull/17749#issuecomment-593838223
@eric-haibin-lin Tests added.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] hzfan commented on a change in pull request
#17749: Fix races in block scope and deferred_init
Posted by GitBox <gi...@apache.org>.
hzfan commented on a change in pull request #17749: Fix races in block scope and deferred_init
URL: https://github.com/apache/incubator-mxnet/pull/17749#discussion_r387592912
##########
File path: python/mxnet/gluon/block.py
##########
@@ -48,8 +48,9 @@ class _BlockScope(object):
def __init__(self, block):
self._block = block
self._counter = {}
- self._old_scope = None
- self._name_scope = None
+ self._local = threading.local()
Review comment:
Thanks for sharing this. Seems that contextvars has not been used in MXNet so far. Do we have a plan to switch to contextvars in the near future?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] leezu commented on a change in pull request
#17749: Fix races in block scope
Posted by GitBox <gi...@apache.org>.
leezu commented on a change in pull request #17749: Fix races in block scope
URL: https://github.com/apache/incubator-mxnet/pull/17749#discussion_r387861538
##########
File path: python/mxnet/gluon/block.py
##########
@@ -48,8 +48,9 @@ class _BlockScope(object):
def __init__(self, block):
self._block = block
self._counter = {}
- self._old_scope = None
- self._name_scope = None
+ self._local = threading.local()
Review comment:
It's a bug. MXNet will currently experience undefined behaviour when coroutines are used. There's no immediate plan to fix it, as it requires some backend work as well. However, new code should ideally be written in a correct way to reduce future efforts.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] hzfan commented on a change in pull request
#17749: Fix races in block scope
Posted by GitBox <gi...@apache.org>.
hzfan commented on a change in pull request #17749: Fix races in block scope
URL: https://github.com/apache/incubator-mxnet/pull/17749#discussion_r388115055
##########
File path: python/mxnet/gluon/block.py
##########
@@ -48,8 +48,9 @@ class _BlockScope(object):
def __init__(self, block):
self._block = block
self._counter = {}
- self._old_scope = None
- self._name_scope = None
+ self._local = threading.local()
Review comment:
Considering that additional dependency on contextvars needs to be introduced for py3.5, a separate pr for contextvars seems better, which adds the dependency, and switch all thread local to contextvars. I may still go with thread local here, but I’ll be glad to switch to contextvars after we add the dependency.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] eric-haibin-lin merged pull request #17749: Fix races in block scope
Posted by GitBox <gi...@apache.org>.
eric-haibin-lin merged pull request #17749:
URL: https://github.com/apache/incubator-mxnet/pull/17749
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] leezu commented on pull request #17749: Fix races in block scope
Posted by GitBox <gi...@apache.org>.
leezu commented on pull request #17749:
URL: https://github.com/apache/incubator-mxnet/pull/17749#issuecomment-632262724
@chinakook gluoncv does not support MXNet 2 development version at this point. You can use MXNet 1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] chinakook commented on pull request #17749: Fix races in block scope
Posted by GitBox <gi...@apache.org>.
chinakook commented on pull request #17749:
URL: https://github.com/apache/incubator-mxnet/pull/17749#issuecomment-663428172
It's also a problem with Tensorflow and Keras, and there are lots of work arounds.
https://github.com/tensorflow/tensorflow/issues/36897
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] chinakook commented on pull request #17749: Fix races in block scope
Posted by GitBox <gi...@apache.org>.
chinakook commented on pull request #17749:
URL: https://github.com/apache/incubator-mxnet/pull/17749#issuecomment-632050382
This commit may cause rcnn/mask-rcnn cannot be trained.
https://github.com/dmlc/gluon-cv/blob/3c4150a964c776e4f7da0eb30b55ab05b7554c8d/gluoncv/data/transforms/presets/rcnn.py#L162
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] hzfan commented on a change in pull request #17749: Fix races in block scope
Posted by GitBox <gi...@apache.org>.
hzfan commented on a change in pull request #17749:
URL: https://github.com/apache/incubator-mxnet/pull/17749#discussion_r425799124
##########
File path: python/mxnet/gluon/block.py
##########
@@ -91,22 +92,22 @@ def create(prefix, params, hint):
def __enter__(self):
if self._block._empty_prefix:
return self
- self._old_scope = getattr(_BlockScope._current, "value", None)
+ self._local._old_scope = getattr(_BlockScope._current, "value", None)
_BlockScope._current.value = self
Review comment:
Seems that it has been thread-local ?
https://github.com/apache/incubator-mxnet/blob/f3e8dc14de881a5b8ccc3ff5985167f8d6fa2fb8/python/mxnet/gluon/block.py#L47
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] eric-haibin-lin commented on a change in pull request #17749: Fix races in block scope
Posted by GitBox <gi...@apache.org>.
eric-haibin-lin commented on a change in pull request #17749:
URL: https://github.com/apache/incubator-mxnet/pull/17749#discussion_r425564767
##########
File path: python/mxnet/gluon/block.py
##########
@@ -91,22 +92,22 @@ def create(prefix, params, hint):
def __enter__(self):
if self._block._empty_prefix:
return self
- self._old_scope = getattr(_BlockScope._current, "value", None)
+ self._local._old_scope = getattr(_BlockScope._current, "value", None)
_BlockScope._current.value = self
Review comment:
Shall we make `_BlockScope._current` also thread-local for consistency?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org