You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hawq.apache.org by kavinderd <gi...@git.apache.org> on 2016/04/14 00:33:10 UTC
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
GitHub user kavinderd opened a pull request:
https://github.com/apache/incubator-hawq/pull/604
HAWQ-644. Account for '\0' when dispatching namespace length
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kavinderd/incubator-hawq HAWQ-644
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-hawq/pull/604.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #604
----
commit 6538dce957cd59079db524b2466d0f27d4b5c287
Author: Kavinder Dhaliwal <ka...@gmail.com>
Date: 2016-04-13T22:32:19Z
HAWQ-644. Account for '\0' when dispatching namespace length
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #604: HAWQ-644. Account for '\0' when dispatching names...
Posted by changleicn <gi...@git.apache.org>.
Github user changleicn commented on the issue:
https://github.com/apache/incubator-hawq/pull/604
@kavinderd the code is in, can you close this PR? thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Posted by ztao1987 <gi...@git.apache.org>.
Github user ztao1987 commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/604#discussion_r59819532
--- Diff: src/backend/cdb/cdbquerycontextdispatching.c ---
@@ -779,21 +779,24 @@ static void
RebuildNamespace(QueryContextInfo *cxt)
{
+ MemoryContext oldContext;
int len;
char buffer[4], *binary;
+ oldContext = MemoryContextSwitchTo(MessageContext);
ReadData(cxt, buffer, sizeof(buffer), TRUE);
len = (int) ntohl(*(uint32 *) buffer);
binary = palloc(len);
if(ReadData(cxt, binary, len, TRUE))
{
- StringInfoData buffer;
- initStringInfoOfString(&buffer, binary, len);
- dfs_address = strdup(buffer.data);
+ dfs_address = pstrdup(binary);
--- End diff --
pstrdup is a kind of malloc, not covered by memorycontext. So there is memory leak in your code. Why not just use dfs_address directly instead of binary? Also even memorycontext used, the best practice is to pfree the memory explictly. Memorycontext is the last barrier which protects the memory.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Posted by shivzone <gi...@git.apache.org>.
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/604#discussion_r59805600
--- Diff: src/backend/cdb/cdbquerycontextdispatching.c ---
@@ -779,21 +779,24 @@ static void
RebuildNamespace(QueryContextInfo *cxt)
{
+ MemoryContext oldContext;
int len;
char buffer[4], *binary;
+ oldContext = MemoryContextSwitchTo(MessageContext);
ReadData(cxt, buffer, sizeof(buffer), TRUE);
len = (int) ntohl(*(uint32 *) buffer);
binary = palloc(len);
if(ReadData(cxt, binary, len, TRUE))
{
- StringInfoData buffer;
- initStringInfoOfString(&buffer, binary, len);
- dfs_address = strdup(buffer.data);
+ dfs_address = pstrdup(binary);
} else {
--- End diff --
convention in C is to have
else
{
Multi line stmts
}
or
else
Single line sttm
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Posted by kavinderd <gi...@git.apache.org>.
Github user kavinderd commented on the pull request:
https://github.com/apache/incubator-hawq/pull/604#issuecomment-210194710
@ictmalili @ztao1987 Please review use of MessageContext for `dfs_address` in `RebuildNamespace`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Posted by ztao1987 <gi...@git.apache.org>.
Github user ztao1987 commented on the pull request:
https://github.com/apache/incubator-hawq/pull/604#issuecomment-209733741
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Posted by shivzone <gi...@git.apache.org>.
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/604#discussion_r59638357
--- Diff: src/backend/cdb/cdbquerycontextdispatching.c ---
@@ -3031,7 +3031,7 @@ prepareDfsAddressForDispatch(QueryContextInfo* cxt)
if (!enable_secure_filesystem)
return;
const char *namespace = cxt->sharedPath;
- int size = strlen(namespace);
+ int size = strlen(namespace) + 1;
--- End diff --
please add a comment mentioning about null terminator character
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #604: HAWQ-644. Account for '\0' when dispatching names...
Posted by kavinderd <gi...@git.apache.org>.
Github user kavinderd commented on the issue:
https://github.com/apache/incubator-hawq/pull/604
Merged
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Posted by ztao1987 <gi...@git.apache.org>.
Github user ztao1987 commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/604#discussion_r59820817
--- Diff: src/backend/cdb/cdbquerycontextdispatching.c ---
@@ -779,21 +779,24 @@ static void
RebuildNamespace(QueryContextInfo *cxt)
{
+ MemoryContext oldContext;
int len;
char buffer[4], *binary;
+ oldContext = MemoryContextSwitchTo(MessageContext);
ReadData(cxt, buffer, sizeof(buffer), TRUE);
len = (int) ntohl(*(uint32 *) buffer);
binary = palloc(len);
if(ReadData(cxt, binary, len, TRUE))
{
- StringInfoData buffer;
- initStringInfoOfString(&buffer, binary, len);
- dfs_address = strdup(buffer.data);
+ dfs_address = pstrdup(binary);
--- End diff --
ok, I thought you used strdup, missed you have changed to pstrdup, this is covered by memorycontext
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #604: HAWQ-644. Account for '\0' when dispatchin...
Posted by kavinderd <gi...@git.apache.org>.
Github user kavinderd closed the pull request at:
https://github.com/apache/incubator-hawq/pull/604
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Posted by shivzone <gi...@git.apache.org>.
Github user shivzone commented on the pull request:
https://github.com/apache/incubator-hawq/pull/604#issuecomment-210194334
+1 on the switch to use messagecontext instead of global variable to store dfs_address
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Posted by shivzone <gi...@git.apache.org>.
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/604#discussion_r59805511
--- Diff: src/backend/cdb/cdbquerycontextdispatching.c ---
@@ -779,21 +779,24 @@ static void
RebuildNamespace(QueryContextInfo *cxt)
{
+ MemoryContext oldContext;
int len;
char buffer[4], *binary;
+ oldContext = MemoryContextSwitchTo(MessageContext);
ReadData(cxt, buffer, sizeof(buffer), TRUE);
len = (int) ntohl(*(uint32 *) buffer);
binary = palloc(len);
if(ReadData(cxt, binary, len, TRUE))
{
- StringInfoData buffer;
- initStringInfoOfString(&buffer, binary, len);
- dfs_address = strdup(buffer.data);
+ dfs_address = pstrdup(binary);
--- End diff --
identation needs to be 4 spaces
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Posted by sansanichfb <gi...@git.apache.org>.
Github user sansanichfb commented on the pull request:
https://github.com/apache/incubator-hawq/pull/604#issuecomment-209679145
Any regression tests needed to cover edge case?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Posted by shivzone <gi...@git.apache.org>.
Github user shivzone commented on the pull request:
https://github.com/apache/incubator-hawq/pull/604#issuecomment-209678894
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Posted by kavinderd <gi...@git.apache.org>.
Github user kavinderd commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/604#discussion_r59819870
--- Diff: src/backend/cdb/cdbquerycontextdispatching.c ---
@@ -779,21 +779,24 @@ static void
RebuildNamespace(QueryContextInfo *cxt)
{
+ MemoryContext oldContext;
int len;
char buffer[4], *binary;
+ oldContext = MemoryContextSwitchTo(MessageContext);
ReadData(cxt, buffer, sizeof(buffer), TRUE);
len = (int) ntohl(*(uint32 *) buffer);
binary = palloc(len);
if(ReadData(cxt, binary, len, TRUE))
{
- StringInfoData buffer;
- initStringInfoOfString(&buffer, binary, len);
- dfs_address = strdup(buffer.data);
+ dfs_address = pstrdup(binary);
--- End diff --
In the definition of `pstrdup` it states
```
/*
* MemoryContextStrdup
* Like strdup(), but allocate from the specified context
*/
```
I chose to use the intermediary `binary` so that if there is an error `dfs_address` is not set with any data, but your point for not using `binary` at all is valid.
We are aware that we should `pfree` even with MemoryContext, but we need the `dfs_address` for the lifetime of each query so this is the simplest & safest means of doing so with the time given
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Posted by kavinderd <gi...@git.apache.org>.
Github user kavinderd commented on the pull request:
https://github.com/apache/incubator-hawq/pull/604#issuecomment-209679799
@sansanichfb We should add some, but need to run in a secure environment.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Posted by ictmalili <gi...@git.apache.org>.
Github user ictmalili commented on the pull request:
https://github.com/apache/incubator-hawq/pull/604#issuecomment-209759488
LGTM. +1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---