You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Jeff Hammerbacher (JIRA)" <ji...@apache.org> on 2010/02/23 15:42:28 UTC
[jira] Reopened: (AVRO-429) investigate large heap size when
writing many objects to an avro data file (Python)
[ https://issues.apache.org/jira/browse/AVRO-429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeff Hammerbacher reopened AVRO-429:
------------------------------------
Assignee: Jeff Hammerbacher
> investigate large heap size when writing many objects to an avro data file (Python)
> -----------------------------------------------------------------------------------
>
> Key: AVRO-429
> URL: https://issues.apache.org/jira/browse/AVRO-429
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: R. Tyler Ballance
> Assignee: Jeff Hammerbacher
>
> Logging ~13k entries via the Python client, seeing abnormally large memory usage
> {{
> >>> from guppy import hpy; hp=hpy(); hp.heap()
> Partition of a set of 127501 objects. Total size = 18102576 bytes.
> Index Count % Size % Cumulative % Kind (class / dict of class)
> 0 58443 46 5227392 29 5227392 29 str
> 1 31964 25 2723312 15 7950704 44 tuple
> 2 422 0 1118096 6 9068800 50 dict of module
> 3 1209 1 1007064 6 10075864 56 dict (no owner)
> 4 1111 1 1002608 6 11078472 61 type
> 5 8253 6 990360 5 12068832 67 function
> 6 8225 6 987000 5 13055832 72 types.CodeType
> 7 1111 1 797992 4 13853824 77 dict of type
> 8 377 0 358232 2 14212056 79 dict of class
> 9 103 0 336040 2 14548096 80 dict of django.db.models.fields.CharField
> <448 more rows. Type e.g. '_.more' to view.>
> >>>
> }}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.