[jira] [Closed] (ARROW-2879) [Python] Arrow plasma can only use a small part of specified shared memory

["Wes McKinney (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wes McKinney closed ARROW-2879.
-------------------------------
    Resolution: Won't Fix

> [Python] Arrow plasma can only use a small part of specified shared memory
> --------------------------------------------------------------------------
>
>                 Key: ARROW-2879
>                 URL: https://issues.apache.org/jira/browse/ARROW-2879
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>            Reporter: chineking
>            Priority: Major
>
> Hi, thanks for the great job of arrow, it helps us a lot.
> However, we encounter a problem when we were using plasma.
> The sample code:
> {code:python}
> import numpy as np
> import pyarrow as pa
> import pyarrow.plasma as plasma
> client = plasma.connect("/tmp/plasma", "", 0)
> puts = []
> nbytes = 0
> while True:
>     a = np.ones((1000, 1000))
>     try:
>         oid = client.put(a)
>         puts.append(client.get(oid))
>         nbytes += a.nbytes
>     except pa.lib.PlasmaStoreFull:
>         print('use nbytes', nbytes)
>         break
> {code}
> We start a plasma store with 1G memory, but the nbytes output above is only 496000000, which cannot even reach half of the memory we specified.
> I cannot figure out why plasma can only use such a small part of shared memory. Could anybody help me? Thanks a lot.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)