You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Vivek Nadkarni (JIRA)" <ji...@apache.org> on 2012/09/14 19:40:07 UTC
[jira] [Created] (AVRO-1165) Avro-C: Memory leak in value iface
containing AVRO_LINK.
Vivek Nadkarni created AVRO-1165:
------------------------------------
Summary: Avro-C: Memory leak in value iface containing AVRO_LINK.
Key: AVRO-1165
URL: https://issues.apache.org/jira/browse/AVRO-1165
Project: Avro
Issue Type: Bug
Components: c
Affects Versions: 1.7.1
Environment: Ubuntu Linux 11.10
Reporter: Vivek Nadkarni
Fix For: 1.7.2
A memory leak can be seen when the following matched pair of commands is called, using a schema containing an AVRO_LINK. This pair of commands constructs a class (value iface) from a schema and then destroys the constructed class.
record_class = avro_generic_class_from_schema( schema );
avro_value_iface_decref( record_class );
If schema contains an AVRO_LINK, then avro_generic_class_from_schema() calls avro_generic_link_class(), which calls avro_schema_incref() on the AVRO_LINK target schema and assigns the target schema pointer to the iface->schema.
When we subsequently call avro_value_iface_decref() to deallocate the class, this function calls avro_generic_link_decref_iface(), which frees the memory for the link interface without calling avro_schema_decref() on the target schema pointed to by iface->schema.
Thus the memory of the target schema is leaked when we create and destroy a value interface for an AVRO_LINK.
Calling avro_schema_decref() on the the target schema (iface->schema) before calling avro_freet() on the iface fixes this memory leak.
Note: The pair of commands shown above results in a memory leak, when we create and destroy a value interface from *any* schema containing an AVRO_LINK, regardless of whether it is recursive or not. There is a separate issue regarding memory leaks with recursive schemas described in AVRO-766. The fix for this issue can only be tested with non-recursive schemas containing AVRO_LINKs until AVRO-766 is fixed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1165) Avro-C: Memory leak in value iface
containing AVRO_LINK.
Posted by "Douglas Creager (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Douglas Creager updated AVRO-1165:
----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Merged into SVN
> Avro-C: Memory leak in value iface containing AVRO_LINK.
> --------------------------------------------------------
>
> Key: AVRO-1165
> URL: https://issues.apache.org/jira/browse/AVRO-1165
> Project: Avro
> Issue Type: Bug
> Components: c
> Affects Versions: 1.7.1
> Environment: Ubuntu Linux 11.10
> Reporter: Vivek Nadkarni
> Fix For: 1.7.2
>
> Attachments: AVRO-1165-FIX.patch, AVRO-1165-TEST.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> A memory leak can be seen when the following matched pair of commands is called, using a schema containing an AVRO_LINK. This pair of commands constructs a class (value iface) from a schema and then destroys the constructed class.
> record_class = avro_generic_class_from_schema( schema );
> avro_value_iface_decref( record_class );
> If schema contains an AVRO_LINK, then avro_generic_class_from_schema() calls avro_generic_link_class(), which calls avro_schema_incref() on the AVRO_LINK target schema and assigns the target schema pointer to the iface->schema.
> When we subsequently call avro_value_iface_decref() to deallocate the class, this function calls avro_generic_link_decref_iface(), which frees the memory for the link interface without calling avro_schema_decref() on the target schema pointed to by iface->schema.
> Thus the memory of the target schema is leaked when we create and destroy a value interface for an AVRO_LINK.
> Calling avro_schema_decref() on the the target schema (iface->schema) before calling avro_freet() on the iface fixes this memory leak.
> Note: The pair of commands shown above results in a memory leak, when we create and destroy a value interface from *any* schema containing an AVRO_LINK, regardless of whether it is recursive or not. There is a separate issue regarding memory leaks with recursive schemas described in AVRO-766. The fix for this issue can only be tested with non-recursive schemas containing AVRO_LINKs until AVRO-766 is fixed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1165) Avro-C: Memory leak in value iface
containing AVRO_LINK.
Posted by "Vivek Nadkarni (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vivek Nadkarni updated AVRO-1165:
---------------------------------
Attachment: AVRO-1165-FIX.patch
This fix (AVRO-1165-FIX.patch) decrements the reference count of the target schema of an AVRO_LINK before deallocating the AVRO_LINK value interface.
Before the fix is applied, valgrind shows the following output:
==6697== 1,410 (24 direct, 1,386 indirect) bytes in 1 blocks are definitely lost in loss record 12 of 12
==6697== at 0x4C28F9F: malloc (vg_replace_malloc.c:236)
==6697== by 0x4C29019: realloc (vg_replace_malloc.c:525)
==6697== by 0x413494: avro_default_allocator (allocation.c:36)
==6697== by 0x40AE84: avro_schema_link (schema.c:663)
==6697== by 0x40B467: avro_schema_from_json_t (schema.c:786)
==6697== by 0x40B80A: avro_schema_from_json_t (schema.c:888)
==6697== by 0x40BE97: avro_schema_from_json_root (schema.c:1083)
==6697== by 0x40BFA2: avro_schema_from_json (schema.c:1108)
==6697== by 0x404A1F: main (test_avro_1165.c:56)
==6697==
==6697== LEAK SUMMARY:
==6697== definitely lost: 48 bytes in 2 blocks
==6697== indirectly lost: 1,386 bytes in 14 blocks
==6697== possibly lost: 0 bytes in 0 blocks
==6697== still reachable: 0 bytes in 0 blocks
==6697== suppressed: 0 bytes in 0 blocks
After the fix is applied, valgrind shows the following output:
==7247== HEAP SUMMARY:
==7247== in use at exit: 0 bytes in 0 blocks
==7247== total heap usage: 180 allocs, 180 frees, 9,563 bytes allocated
==7247==
==7247== All heap blocks were freed -- no leaks are possible
==7247==
==7247== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4)
> Avro-C: Memory leak in value iface containing AVRO_LINK.
> --------------------------------------------------------
>
> Key: AVRO-1165
> URL: https://issues.apache.org/jira/browse/AVRO-1165
> Project: Avro
> Issue Type: Bug
> Components: c
> Affects Versions: 1.7.1
> Environment: Ubuntu Linux 11.10
> Reporter: Vivek Nadkarni
> Fix For: 1.7.2
>
> Attachments: AVRO-1165-FIX.patch, AVRO-1165-TEST.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> A memory leak can be seen when the following matched pair of commands is called, using a schema containing an AVRO_LINK. This pair of commands constructs a class (value iface) from a schema and then destroys the constructed class.
> record_class = avro_generic_class_from_schema( schema );
> avro_value_iface_decref( record_class );
> If schema contains an AVRO_LINK, then avro_generic_class_from_schema() calls avro_generic_link_class(), which calls avro_schema_incref() on the AVRO_LINK target schema and assigns the target schema pointer to the iface->schema.
> When we subsequently call avro_value_iface_decref() to deallocate the class, this function calls avro_generic_link_decref_iface(), which frees the memory for the link interface without calling avro_schema_decref() on the target schema pointed to by iface->schema.
> Thus the memory of the target schema is leaked when we create and destroy a value interface for an AVRO_LINK.
> Calling avro_schema_decref() on the the target schema (iface->schema) before calling avro_freet() on the iface fixes this memory leak.
> Note: The pair of commands shown above results in a memory leak, when we create and destroy a value interface from *any* schema containing an AVRO_LINK, regardless of whether it is recursive or not. There is a separate issue regarding memory leaks with recursive schemas described in AVRO-766. The fix for this issue can only be tested with non-recursive schemas containing AVRO_LINKs until AVRO-766 is fixed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1165) Avro-C: Memory leak in value iface
containing AVRO_LINK.
Posted by "Vivek Nadkarni (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vivek Nadkarni updated AVRO-1165:
---------------------------------
Attachment: AVRO-1165-TEST.patch
This patch (AVRO-1165-TEST.patch) provides a test that shows the memory leak, when the test program in the patch is run through valgrind.
To validate AVRO-1165, run this test program through valgrind before and after applying the AVRO-1165-FIX.patch. Before the applying the fix valgrind will show memory leaks, and after the fix it will not.
The specific valgrind commandline to use from the avro-trunk/lang/c/tests directory is: valgrind -v --track-origins=yes --leak-check=full --show-reachable = yes ../build/tests/test_avro_1165
> Avro-C: Memory leak in value iface containing AVRO_LINK.
> --------------------------------------------------------
>
> Key: AVRO-1165
> URL: https://issues.apache.org/jira/browse/AVRO-1165
> Project: Avro
> Issue Type: Bug
> Components: c
> Affects Versions: 1.7.1
> Environment: Ubuntu Linux 11.10
> Reporter: Vivek Nadkarni
> Fix For: 1.7.2
>
> Attachments: AVRO-1165-TEST.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> A memory leak can be seen when the following matched pair of commands is called, using a schema containing an AVRO_LINK. This pair of commands constructs a class (value iface) from a schema and then destroys the constructed class.
> record_class = avro_generic_class_from_schema( schema );
> avro_value_iface_decref( record_class );
> If schema contains an AVRO_LINK, then avro_generic_class_from_schema() calls avro_generic_link_class(), which calls avro_schema_incref() on the AVRO_LINK target schema and assigns the target schema pointer to the iface->schema.
> When we subsequently call avro_value_iface_decref() to deallocate the class, this function calls avro_generic_link_decref_iface(), which frees the memory for the link interface without calling avro_schema_decref() on the target schema pointed to by iface->schema.
> Thus the memory of the target schema is leaked when we create and destroy a value interface for an AVRO_LINK.
> Calling avro_schema_decref() on the the target schema (iface->schema) before calling avro_freet() on the iface fixes this memory leak.
> Note: The pair of commands shown above results in a memory leak, when we create and destroy a value interface from *any* schema containing an AVRO_LINK, regardless of whether it is recursive or not. There is a separate issue regarding memory leaks with recursive schemas described in AVRO-766. The fix for this issue can only be tested with non-recursive schemas containing AVRO_LINKs until AVRO-766 is fixed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1165) Avro-C: Memory leak in value iface
containing AVRO_LINK.
Posted by "Vivek Nadkarni (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vivek Nadkarni updated AVRO-1165:
---------------------------------
Status: Patch Available (was: Open)
The patch for AVRO-1165 is now available for review and for merging with the trunk.
> Avro-C: Memory leak in value iface containing AVRO_LINK.
> --------------------------------------------------------
>
> Key: AVRO-1165
> URL: https://issues.apache.org/jira/browse/AVRO-1165
> Project: Avro
> Issue Type: Bug
> Components: c
> Affects Versions: 1.7.1
> Environment: Ubuntu Linux 11.10
> Reporter: Vivek Nadkarni
> Fix For: 1.7.2
>
> Attachments: AVRO-1165-FIX.patch, AVRO-1165-TEST.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> A memory leak can be seen when the following matched pair of commands is called, using a schema containing an AVRO_LINK. This pair of commands constructs a class (value iface) from a schema and then destroys the constructed class.
> record_class = avro_generic_class_from_schema( schema );
> avro_value_iface_decref( record_class );
> If schema contains an AVRO_LINK, then avro_generic_class_from_schema() calls avro_generic_link_class(), which calls avro_schema_incref() on the AVRO_LINK target schema and assigns the target schema pointer to the iface->schema.
> When we subsequently call avro_value_iface_decref() to deallocate the class, this function calls avro_generic_link_decref_iface(), which frees the memory for the link interface without calling avro_schema_decref() on the target schema pointed to by iface->schema.
> Thus the memory of the target schema is leaked when we create and destroy a value interface for an AVRO_LINK.
> Calling avro_schema_decref() on the the target schema (iface->schema) before calling avro_freet() on the iface fixes this memory leak.
> Note: The pair of commands shown above results in a memory leak, when we create and destroy a value interface from *any* schema containing an AVRO_LINK, regardless of whether it is recursive or not. There is a separate issue regarding memory leaks with recursive schemas described in AVRO-766. The fix for this issue can only be tested with non-recursive schemas containing AVRO_LINKs until AVRO-766 is fixed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira