You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Mark Gius (JIRA)" <ji...@apache.org> on 2012/07/25 19:36:34 UTC
[jira] [Created] (ZOOKEEPER-1519) Zookeeper Async calls can
reference free()'d memory
Mark Gius created ZOOKEEPER-1519:
------------------------------------
Summary: Zookeeper Async calls can reference free()'d memory
Key: ZOOKEEPER-1519
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519
Project: ZooKeeper
Issue Type: Bug
Components: c client
Affects Versions: 3.3.3
Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes.
Reporter: Mark Gius
zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result.
However, ZKPython is a whole different story. Consider this snippet:
ok = zookeeper.acreate(handle, path, json.dumps(value),
acl, flags, callback)
assert ok == zookeeper.OK
In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory.
I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed.
I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can
reference free()'d memory
Posted by "Stephen Tyree (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422447#comment-13422447 ]
Stephen Tyree commented on ZOOKEEPER-1519:
------------------------------------------
Duplicating the data wouldn't help in circumstances where the data is a structure contain malloc'd memory, i.e.:
struct Foo {
const char* bar;
};
> Zookeeper Async calls can reference free()'d memory
> ---------------------------------------------------
>
> Key: ZOOKEEPER-1519
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519
> Project: ZooKeeper
> Issue Type: Bug
> Components: c client
> Affects Versions: 3.3.3
> Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes.
> Reporter: Mark Gius
>
> zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result.
> However, ZKPython is a whole different story. Consider this snippet:
> ok = zookeeper.acreate(handle, path, json.dumps(value),
> acl, flags, callback)
> assert ok == zookeeper.OK
> In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory.
> I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed.
> I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can
reference free()'d memory
Posted by "Mark Gius (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422461#comment-13422461 ]
Mark Gius commented on ZOOKEEPER-1519:
--------------------------------------
Then perhaps the solution is to document a contract between aysnc calls and library consumers that the data pointer needs to be alive until after the async call has complete (or been submitted, or whatever).
Or maybe async calls need another argument for a cleanup function that frees the data pointer.
> Zookeeper Async calls can reference free()'d memory
> ---------------------------------------------------
>
> Key: ZOOKEEPER-1519
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519
> Project: ZooKeeper
> Issue Type: Bug
> Components: c client
> Affects Versions: 3.3.3
> Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes.
> Reporter: Mark Gius
>
> zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result.
> However, ZKPython is a whole different story. Consider this snippet:
> ok = zookeeper.acreate(handle, path, json.dumps(value),
> acl, flags, callback)
> assert ok == zookeeper.OK
> In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory.
> I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed.
> I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can
reference free()'d memory
Posted by "Mark Gius (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422538#comment-13422538 ]
Mark Gius commented on ZOOKEEPER-1519:
--------------------------------------
well, I don't think zookeeper would handle such a struct correctly anyway, because zookeeper treats whatever you throw into that argument like a byte array. So zookeeper will persist the memory location of the inner-malloc()'d memory, but not the memory itself.
> Zookeeper Async calls can reference free()'d memory
> ---------------------------------------------------
>
> Key: ZOOKEEPER-1519
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519
> Project: ZooKeeper
> Issue Type: Bug
> Components: c client
> Affects Versions: 3.3.3
> Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes.
> Reporter: Mark Gius
>
> zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result.
> However, ZKPython is a whole different story. Consider this snippet:
> ok = zookeeper.acreate(handle, path, json.dumps(value),
> acl, flags, callback)
> assert ok == zookeeper.OK
> In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory.
> I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed.
> I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira