[GitHub] zhreshold opened a new pull request #12025: disable opencv threading for forked process

[GitBox <gi...@apache.org>]

zhreshold opened a new pull request #12025: disable opencv threading for forked process
URL: https://github.com/apache/incubator-mxnet/pull/12025
 
 
   ## Description ##
   After 3.4, opencv seems to be enabling threading automatically. 
   This PR disable opencv threading for forked process, to alleviate thread contention problem.
   
   This PR improved image loading performance from 160 sample/s to 450 sample/s with gluon data loader when `num_workers=4`
   
   example code with real data:
   
   ```python
   
   import mxnet as mx
   import gluoncv as gcv
   from tqdm import tqdm
   
   class NaiveTransform(object):
       def __init__(self, width, height):
           self._width = width
           self._height = height
   
       def __call__(self, src, label):
           src = mx.image.imresize(src, self._width, self._height, interp=1)
           return src, label
   
   dataset = gcv.data.COCODetection(splits=('instances_train2017',), skip_empty=True, use_crowd=False)
   dataloader = mx.gluon.data.DataLoader(dataset.transform(NaiveTransform(512, 512)),
                                         batch_size=1, shuffle=True, num_workers=4)
   with tqdm(total=len(dataset)) as pbar:
       for batch in dataloader:
           mx.nd.waitall()
           pbar.update(1)
   ```
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services