[GitHub] [tvm] Maximilianxu commented on issue #7758: TVM RPC will fail when allocating large arrays on an Android phone

[GitBox <gi...@apache.org>]

Maximilianxu commented on issue #7758:
URL: https://github.com/apache/tvm/issues/7758#issuecomment-808674041


   ```
   Traceback (most recent call last):
     File "tune_network_x86.py", line 492, in <module>
       tune_network()
     File "tune_network_x86.py", line 423, in tune_network
       tmp = tvm.nd.array(np.random.uniform(size=(30522, 7680)).astype(np.float32), ctx)
     File "/home/zyx/workspaces/python/tvm0.8_v3/python/tvm/runtime/ndarray.py", line 513, in array
       return empty(arr.shape, arr.dtype, device).copyfrom(arr)
     File "/home/zyx/workspaces/python/tvm0.8_v3/python/tvm/runtime/ndarray.py", line 152, in copyfrom
       check_call(_LIB.TVMArrayCopyFromBytes(self.handle, data, nbytes))
     File "/home/zyx/workspaces/python/tvm0.8_v3/python/tvm/_ffi/base.py", line 346, in check_call
       raise get_last_ffi_error()
   tvm._ffi.base.TVMError: Traceback (most recent call last):
     31: 0xffffffffffffffff
     30: _start
     29: __libc_start_main
     28: Py_BytesMain
     27: Py_RunMain
     26: PyRun_SimpleFileExFlags
     25: PyRun_FileExFlags
     24: 0x000000000067d61e
     23: 0x000000000067d5a0
     22: PyEval_EvalCode
     21: _PyEval_EvalCodeWithName
     20: _PyEval_EvalFrameDefault
     19: _PyFunction_Vectorcall
     18: _PyEval_EvalCodeWithName
     17: _PyEval_EvalFrameDefault
     16: _PyFunction_Vectorcall
     15: _PyEval_EvalCodeWithName
     14: _PyEval_EvalFrameDefault
     13: _PyFunction_Vectorcall
     12: _PyEval_EvalFrameDefault
     11: _PyObject_MakeTpCall
     10: 0x00007f19e8e4d7df
     9: _ctypes_callproc
     8: 0x00007f19e9d82409
     7: 0x00007f19e9d82ff4
     6: TVMArrayCopyFromBytes
     5: tvm::runtime::ArrayCopyFromBytes(DLTensor*, void const*, unsigned long)
     4: tvm::runtime::RPCDeviceAPI::CopyDataFromTo(DLTensor*, DLTensor*, void*)
     3: tvm::runtime::RPCEndpoint::CopyToRemote(void*, DLTensor*, unsigned long)
     2: tvm::runtime::RPCEndpoint::HandleUntilReturnEvent(bool, std::function<void (tvm::runtime::TVMArgs)>)
     1: tvm::runtime::SockChannel::Send(void const*, unsigned long)
     0: tvm::support::Socket::Error(char const*)
     File "/home/zyx/workspaces/python/tvm0.8_v3/src/runtime/rpc/../../support/socket.h", line 360
   ```
   The last error message was generated using the latest version and the following code.
   ```
       local_demo = True if TARGET != "android" else False
       if local_demo:
           remote = rpc.LocalSession()
       else:
           tracker_host = os.environ.get("TVM_TRACKER_HOST", "192.168.1.103")
           tracker_port = int(os.environ.get("TVM_TRACKER_PORT", 9196))
           key = "huawei"
           tracker = rpc.connect_tracker(tracker_host, tracker_port)
           # When running a heavy model, we should increase the `session_timeout`
           remote = tracker.request(key, priority=0, session_timeout=1000)
       weights = {}
       ctx = remote.cpu(0) if TARGET != "cuda" else remote.gpu(0)
       MB = 0
       from functools import reduce
       byte = lambda x: reduce(lambda a, b: a * b, x) / 1e6
       for _ in range(100):
           tmp = tvm.nd.array(np.random.uniform(size=(30522, 7680)).astype(np.float32), ctx)
           MB += byte((30522, 7680)) * 4
           print(tmp.shape)
           print("MB:", MB)
       exit()
   ```
   I cannot allocate a single array with the above code.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org