[incubator-mxnet] branch master updated: [FEATURE] MXIndexedRecordIO: avoid re-build index (#20549)

[pt...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

ptrendx pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
     new 72e6b16  [FEATURE] MXIndexedRecordIO: avoid re-build index (#20549)
72e6b16 is described below

commit 72e6b162e0e1bd959dec94f5f4dc049153778558
Author: 胡玮文 <hu...@outlook.com>
AuthorDate: Fri Sep 17 07:48:30 2021 +0800

    [FEATURE] MXIndexedRecordIO: avoid re-build index (#20549)
    
    When resetting, avoid re-build index for read-only instances to save time and
    memory (in case of forked, memory pages can be shared)
---
 python/mxnet/recordio.py | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/python/mxnet/recordio.py b/python/mxnet/recordio.py
index eb86114..b1d972e 100644
--- a/python/mxnet/recordio.py
+++ b/python/mxnet/recordio.py
@@ -245,10 +245,11 @@ class MXIndexedRecordIO(MXRecordIO):
 
     def open(self):
         super(MXIndexedRecordIO, self).open()
-        self.idx = {}
-        self.keys = []
         self.fidx = open(self.idx_path, self.flag)
-        if not self.writable:
+        if self.writable:
+            self.idx = {}
+            self.keys = []
+        elif not self.idx:
             for line in iter(self.fidx.readline, ''):
                 line = line.strip().split('\t')
                 key = self.key_type(line[0])