You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hawq.apache.org by interma <gi...@git.apache.org> on 2016/12/20 06:32:53 UTC
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
GitHub user interma opened a pull request:
https://github.com/apache/incubator-hawq/pull/1058
HAWQ-1226. HAWQ core dump due to enable ranger while RPS is down
also include:
1. fix core dump while RPS not running
2. fix libcurl write_callback() problem
3. add libcurl timeout (hard-code 30s now)
4. fix some compile warnings
5. fix compile error: add json.h to checkinc.py
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/interma/interma-hawq HAWQ-1226
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-hawq/pull/1058.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1058
----
commit c663476c4413fd02ee39ae8b6c55edd9e7f60826
Author: interma <in...@outlook.com>
Date: 2016-12-19T10:17:25Z
core dump while RPS not running
include:
1. core dump while RPS not running
2. libcurl write callback problem
3. libcurl timeout
4. compile warning
commit 695cf1dac824ddf61ab266dd6f7a581c075ced66
Author: interma <in...@outlook.com>
Date: 2016-12-20T05:51:09Z
merge hong wu's code
commit 0ad094b167d4ce7fbccd914c9e5e8b93d39e506a
Author: interma <in...@outlook.com>
Date: 2016-12-20T05:54:48Z
Merge remote-tracking branch 'origin/master' into HAWQ-1226
commit 303b5d1d48f3fd1f13c77488e5bfa0a6dab7734b
Author: interma <in...@outlook.com>
Date: 2016-12-20T06:20:43Z
HAWQ-1226. HAWQ core dump due to enable ranger while RPS is down
include:
1. fix core dump while RPS not running
2. fix libcurl write_callback() problem
3. add libcurl timeout (hard-code 30s now)
4. fix some compile warnings
5. fix compile error: add json.h to checkinc.py
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by xunzhang <gi...@git.apache.org>.
Github user xunzhang commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93180025
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -24,60 +24,72 @@
*
*-------------------------------------------------------------------------
*/
-
#include "utils/rangerrest.h"
-
/*
* A mapping from AclObjectKind to string
*/
char* AclObjectKindStr[] =
{
- "table", /* pg_class */
- "sequence", /* pg_sequence */
- "database", /* pg_database */
- "function", /* pg_proc */
- "operator", /* pg_operator */
- "type", /* pg_type */
- "language", /* pg_language */
- "namespace", /* pg_namespace */
- "oplass", /* pg_opclass */
- "conversion", /* pg_conversion */
- "tablespace", /* pg_tablespace */
- "filespace", /* pg_filespace */
- "filesystem", /* pg_filesystem */
- "fdw", /* pg_foreign_data_wrapper */
- "foreign_server", /* pg_foreign_server */
- "protocol", /* pg_extprotocol */
- "none" /* MUST BE LAST */
+ "table", /* pg_class */
+ "sequence", /* pg_sequence */
+ "database", /* pg_database */
+ "function", /* pg_proc */
+ "operator", /* pg_operator */
+ "type", /* pg_type */
+ "language", /* pg_language */
+ "namespace", /* pg_namespace */
+ "oplass", /* pg_opclass */
+ "conversion", /* pg_conversion */
+ "tablespace", /* pg_tablespace */
+ "filespace", /* pg_filespace */
+ "filesystem", /* pg_filesystem */
+ "fdw", /* pg_foreign_data_wrapper */
+ "foreign_server", /* pg_foreign_server */
+ "protocol", /* pg_extprotocol */
+ "none" /* MUST BE LAST */
};
RangerACLResult parse_ranger_response(char* buffer)
{
- Assert(buffer != NULL);
- if (strlen(buffer) == 0)
- return RANGERCHECK_UNKNOWN;
-
- elog(LOG, "read from Ranger Restful API: %s", buffer);
-
- struct json_object *response = json_tokener_parse(buffer);
- struct json_object *accessObj = json_object_object_get(response, "access");
-
- int arraylen = json_object_array_length(accessObj);
- elog(LOG, "Array Length: %dn",arraylen);
-
- json_object * jvalue;
- json_object * jallow;
- json_bool result;
- // here should return which table's acl check failed in future.
- for (int i=0; i< arraylen; i++){
- jvalue = json_object_array_get_idx(accessObj, i);
- jallow = json_object_object_get(jvalue, "allowed");
- result = json_object_get_boolean(jallow);
- if(result != 1){
- return RANGERCHECK_NO_PRIV;
- }
- }
- return RANGERCHECK_OK;
+ Assert(buffer != NULL);
+ if (strlen(buffer) == 0)
+ return RANGERCHECK_UNKNOWN;
+
+ elog(LOG, "read from Ranger Restful API: %s", buffer);
+
+ struct json_object *response = json_tokener_parse(buffer);
+ if (response == NULL)
+ {
+ elog(WARNING, "json_tokener_parse failed");
+ return RANGERCHECK_UNKNOWN;
+ }
+
+ struct json_object *accessObj = NULL;
+ if (!json_object_object_get_ex(response, "access", &accessObj))
--- End diff --
What about other fields check?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by xunzhang <gi...@git.apache.org>.
Github user xunzhang commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93180221
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -86,132 +98,132 @@ RangerACLResult parse_ranger_response(char* buffer)
*/
json_object *create_ranger_request_json_batch(List *args)
{
- json_object *juser = NULL;
- json_object *jaccess = json_object_new_array();
- json_object *jrequest = json_object_new_object();
- char *user = NULL;
- ListCell *arg;
-
- foreach(arg, args)
- {
- RangerRequestJsonArgs *arg_ptr = (RangerRequestJsonArgs *) lfirst(arg);
- if (user == NULL)
- {
- user = arg_ptr->user;
- juser = json_object_new_string(user);
- }
- AclObjectKind kind = arg_ptr->kind;
- char* object = arg_ptr->object;
- Assert(user != NULL && object != NULL && privilege != NULL && arg_ptr->isAll);
- elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
- user, AclObjectKindStr[kind], object);
-
- json_object *jresource = json_object_new_object();
- json_object *jelement = json_object_new_object();
- json_object *jactions = json_object_new_array();
-
- switch(kind)
- {
- case ACL_KIND_CLASS:
- case ACL_KIND_SEQUENCE:
- case ACL_KIND_PROC:
- case ACL_KIND_NAMESPACE:
- case ACL_KIND_LANGUAGE:
- {
- char *ptr = NULL; char *name = NULL;
- char *first = NULL; // could be a database or protocol or tablespace
- char *second = NULL; // could be a schema or language
- char *third = NULL; // could be a table or sequence or function
- int idx = 0;
- for (name = strtok_r(object, ".", &ptr);
- name;
- name = strtok_r(NULL, ".", &ptr), idx++)
- {
- if (idx == 0)
- {
- first = pstrdup(name);
- }
- else if (idx == 1)
- {
- second = pstrdup(name);
- }
- else
- {
- third = pstrdup(name);
- }
- }
-
- if (first != NULL)
- {
- json_object *jfirst = json_object_new_string(first);
- json_object_object_add(jresource, "database", jfirst);
- }
- if (second != NULL)
- {
- json_object *jsecond = json_object_new_string(second);
- json_object_object_add(jresource,
- (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
- }
- if (third != NULL)
- {
- json_object *jthird = json_object_new_string(third);
- json_object_object_add(jresource,
- (kind == ACL_KIND_CLASS) ? "table" :
- (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
- }
-
- if (first != NULL)
- pfree(first);
- if (second != NULL)
- pfree(second);
- if (third != NULL)
- pfree(third);
- break;
- }
- case ACL_KIND_OPER:
- case ACL_KIND_CONVERSION:
- case ACL_KIND_DATABASE:
- case ACL_KIND_TABLESPACE:
- case ACL_KIND_TYPE:
- case ACL_KIND_FILESYSTEM:
- case ACL_KIND_FDW:
- case ACL_KIND_FOREIGN_SERVER:
- case ACL_KIND_EXTPROTOCOL:
- {
- json_object *jobject = json_object_new_string(object);
- json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
- break;
- }
- default:
- elog(ERROR, "unrecognized objkind: %d", (int) kind);
- } // switch
-
- json_object_object_add(jelement, "resource", jresource);
-
- //ListCell *cell;
- //foreach(cell, arg_ptr->actions)
- //{
- char tmp[7] = "select";
- json_object* jaction = json_object_new_string((char *)tmp);
- //json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
- json_object_array_add(jactions, jaction);
- //}
- json_object_object_add(jelement, "privileges", jactions);
- json_object_array_add(jaccess, jelement);
-
- } // foreach
-
- json_object_object_add(jrequest, "user", juser);
- json_object_object_add(jrequest, "access", jaccess);
-
- json_object *jreqid = json_object_new_string("1");
- json_object_object_add(jrequest, "requestId", jreqid);
- json_object *jclientip = json_object_new_string("123.0.0.21");
- json_object_object_add(jrequest, "clientIp", jclientip);
- json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
- json_object_object_add(jrequest, "context", jcontext);
-
- return jrequest;
+ json_object *juser = NULL;
--- End diff --
Here I think you'd better set the tab/space again to make the diff minimal for review. If it is tricky to do that, let it be.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #1058: HAWQ-1226. HAWQ core dump due to enable ranger w...
Posted by interma <gi...@git.apache.org>.
Github user interma commented on the issue:
https://github.com/apache/incubator-hawq/pull/1058
I will open a new PR, closed this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by stanlyxiang <gi...@git.apache.org>.
Github user stanlyxiang commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93179139
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -245,279 +257,326 @@ json_object *create_ranger_request_json_batch(List *args)
* }
*/
json_object* create_ranger_request_json(char* user, AclObjectKind kind, char* object,
- List* actions, bool isAll)
+ List* actions, bool isAll)
{
- Assert(user != NULL && object != NULL && privilege != NULL
- && isAll);
- ListCell *cell;
-
- elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
- user, AclObjectKindStr[kind], object);
- json_object *jrequest = json_object_new_object();
- json_object *juser = json_object_new_string(user);
-
- json_object *jaccess = json_object_new_array();
- json_object *jelement = json_object_new_object();
-
- json_object *jresource = json_object_new_object();
- switch(kind)
- {
- case ACL_KIND_CLASS:
- case ACL_KIND_SEQUENCE:
- case ACL_KIND_PROC:
- case ACL_KIND_NAMESPACE:
- case ACL_KIND_LANGUAGE:
- {
- char *ptr = NULL; char *name = NULL;
- char *first = NULL; // could be a database or protocol or tablespace
- char *second = NULL; // could be a schema or language
- char *third = NULL; // could be a table or sequence or function
- int idx = 0;
- for (name = strtok_r(object, ".", &ptr);
- name;
- name = strtok_r(NULL, ".", &ptr), idx++)
- {
- if (idx == 0)
- {
- first = pstrdup(name);
- }
- else if (idx == 1)
- {
- second = pstrdup(name);
- }
- else
- {
- third = pstrdup(name);
- }
- }
-
- if (first != NULL)
- {
- json_object *jfirst = json_object_new_string(first);
- json_object_object_add(jresource, "database", jfirst);
- }
- if (second != NULL)
- {
- json_object *jsecond = json_object_new_string(second);
- json_object_object_add(jresource,
- (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
- }
- if (third != NULL)
- {
- json_object *jthird = json_object_new_string(third);
- json_object_object_add(jresource,
- (kind == ACL_KIND_CLASS) ? "table" :
- (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
- }
-
- if (first != NULL)
- pfree(first);
- if (second != NULL)
- pfree(second);
- if (third != NULL)
- pfree(third);
- break;
- }
- case ACL_KIND_OPER:
- case ACL_KIND_CONVERSION:
- case ACL_KIND_DATABASE:
- case ACL_KIND_TABLESPACE:
- case ACL_KIND_TYPE:
- case ACL_KIND_FILESYSTEM:
- case ACL_KIND_FDW:
- case ACL_KIND_FOREIGN_SERVER:
- case ACL_KIND_EXTPROTOCOL:
- {
- json_object *jobject = json_object_new_string(object);
- json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
- break;
- }
- default:
- elog(ERROR, "unrecognized objkind: %d", (int) kind);
- }
-
- json_object *jactions = json_object_new_array();
- foreach(cell, actions)
- {
- json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
- json_object_array_add(jactions, jaction);
- }
- json_object_object_add(jelement, "resource", jresource);
- json_object_object_add(jelement, "privileges", jactions);
- json_object_array_add(jaccess, jelement);
-
- json_object_object_add(jrequest, "user", juser);
- json_object_object_add(jrequest, "access", jaccess);
- json_object *jreqid = json_object_new_string("1");
- json_object_object_add(jrequest, "requestId", jreqid);
- json_object *jclientip = json_object_new_string("123.0.0.21");
- json_object_object_add(jrequest, "clientIp", jclientip);
- json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
- json_object_object_add(jrequest, "context", jcontext);
-
-
- return jrequest;
+ Assert(user != NULL && object != NULL && privilege != NULL
+ && isAll);
+ ListCell *cell;
+
+ elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
+ user, AclObjectKindStr[kind], object);
+ json_object *jrequest = json_object_new_object();
+ json_object *juser = json_object_new_string(user);
+
+ json_object *jaccess = json_object_new_array();
+ json_object *jelement = json_object_new_object();
+
+ json_object *jresource = json_object_new_object();
+ switch(kind)
+ {
+ case ACL_KIND_CLASS:
+ case ACL_KIND_SEQUENCE:
+ case ACL_KIND_PROC:
+ case ACL_KIND_NAMESPACE:
+ case ACL_KIND_LANGUAGE:
+ {
+ char *ptr = NULL; char *name = NULL;
+ char *first = NULL; // could be a database or protocol or tablespace
+ char *second = NULL; // could be a schema or language
+ char *third = NULL; // could be a table or sequence or function
+ int idx = 0;
+ for (name = strtok_r(object, ".", &ptr);
+ name;
+ name = strtok_r(NULL, ".", &ptr), idx++)
+ {
+ if (idx == 0)
+ {
+ first = pstrdup(name);
+ }
+ else if (idx == 1)
+ {
+ second = pstrdup(name);
+ }
+ else
+ {
+ third = pstrdup(name);
+ }
+ }
+
+ if (first != NULL)
+ {
+ json_object *jfirst = json_object_new_string(first);
+ json_object_object_add(jresource, "database", jfirst);
+ }
+ if (second != NULL)
+ {
+ json_object *jsecond = json_object_new_string(second);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
+ }
+ if (third != NULL)
+ {
+ json_object *jthird = json_object_new_string(third);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_CLASS) ? "table" :
+ (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
+ }
+
+ if (first != NULL)
+ pfree(first);
+ if (second != NULL)
+ pfree(second);
+ if (third != NULL)
+ pfree(third);
+ break;
+ }
+ case ACL_KIND_OPER:
+ case ACL_KIND_CONVERSION:
+ case ACL_KIND_DATABASE:
+ case ACL_KIND_TABLESPACE:
+ case ACL_KIND_TYPE:
+ case ACL_KIND_FILESYSTEM:
+ case ACL_KIND_FDW:
+ case ACL_KIND_FOREIGN_SERVER:
+ case ACL_KIND_EXTPROTOCOL:
+ {
+ json_object *jobject = json_object_new_string(object);
+ json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
+ break;
+ }
+ default:
+ elog(ERROR, "unrecognized objkind: %d", (int) kind);
+ }
+
+ json_object *jactions = json_object_new_array();
+ foreach(cell, actions)
+ {
+ json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
+ json_object_array_add(jactions, jaction);
+ }
+ json_object_object_add(jelement, "resource", jresource);
+ json_object_object_add(jelement, "privileges", jactions);
+ json_object_array_add(jaccess, jelement);
+
+ json_object_object_add(jrequest, "user", juser);
+ json_object_object_add(jrequest, "access", jaccess);
+ json_object *jreqid = json_object_new_string("1");
+ json_object_object_add(jrequest, "requestId", jreqid);
+ json_object *jclientip = json_object_new_string("123.0.0.21");
+ json_object_object_add(jrequest, "clientIp", jclientip);
+ json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
+ json_object_object_add(jrequest, "context", jcontext);
+
+
+ return jrequest;
}
static size_t write_callback(char *contents, size_t size, size_t nitems,
- void *userp)
+ void *userp)
{
- size_t realsize = size * nitems;
- CURL_HANDLE curl = (curl_context_t *) userp;
-
- curl->response.buffer = palloc0(realsize + 1);
- memset(curl->response.buffer, 0, realsize + 1);
- if (curl->response.buffer == NULL)
- {
- /* out of memory! */
- elog(WARNING, "not enough memory for Ranger response");
- return 0;
- }
-
- memcpy(curl->response.buffer, contents, realsize);
- curl->response.size = realsize + 1;
- elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
-
- return realsize;
+ size_t realsize = size * nitems;
+ CURL_HANDLE curl = (CURL_HANDLE) userp;
+ Assert(curl != NULL);
+
+ if (curl->response.buffer == NULL)
+ {
+ curl->response.buffer = palloc0(realsize + 1);
+ }
+ else
+ {
+ //Note:
+ //our repalloc is not same as realloc, repalloc's first param(buffer) can not be NULL
+ curl->response.buffer = repalloc(curl->response.buffer, curl->response.size + realsize + 1);
+ }
+
+ if (curl->response.buffer == NULL)
+ {
+ /* out of memory! */
+ elog(WARNING, "not enough memory for Ranger response");
+ return 0;
+ }
+
+ memcpy(curl->response.buffer + curl->response.size, contents, realsize);
+ curl->response.size += realsize;
+ curl->response.buffer[curl->response.size] = '\0';
+ elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
+
+ return realsize;
}
-void call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
+
--- End diff --
too many blank line.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by xunzhang <gi...@git.apache.org>.
Github user xunzhang commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93179978
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -24,60 +24,72 @@
*
*-------------------------------------------------------------------------
*/
-
#include "utils/rangerrest.h"
-
/*
* A mapping from AclObjectKind to string
*/
char* AclObjectKindStr[] =
{
- "table", /* pg_class */
- "sequence", /* pg_sequence */
- "database", /* pg_database */
- "function", /* pg_proc */
- "operator", /* pg_operator */
- "type", /* pg_type */
- "language", /* pg_language */
- "namespace", /* pg_namespace */
- "oplass", /* pg_opclass */
- "conversion", /* pg_conversion */
- "tablespace", /* pg_tablespace */
- "filespace", /* pg_filespace */
- "filesystem", /* pg_filesystem */
- "fdw", /* pg_foreign_data_wrapper */
- "foreign_server", /* pg_foreign_server */
- "protocol", /* pg_extprotocol */
- "none" /* MUST BE LAST */
+ "table", /* pg_class */
+ "sequence", /* pg_sequence */
+ "database", /* pg_database */
+ "function", /* pg_proc */
+ "operator", /* pg_operator */
+ "type", /* pg_type */
+ "language", /* pg_language */
+ "namespace", /* pg_namespace */
+ "oplass", /* pg_opclass */
+ "conversion", /* pg_conversion */
+ "tablespace", /* pg_tablespace */
+ "filespace", /* pg_filespace */
+ "filesystem", /* pg_filesystem */
+ "fdw", /* pg_foreign_data_wrapper */
+ "foreign_server", /* pg_foreign_server */
+ "protocol", /* pg_extprotocol */
+ "none" /* MUST BE LAST */
};
RangerACLResult parse_ranger_response(char* buffer)
{
- Assert(buffer != NULL);
- if (strlen(buffer) == 0)
- return RANGERCHECK_UNKNOWN;
-
- elog(LOG, "read from Ranger Restful API: %s", buffer);
-
- struct json_object *response = json_tokener_parse(buffer);
- struct json_object *accessObj = json_object_object_get(response, "access");
-
- int arraylen = json_object_array_length(accessObj);
- elog(LOG, "Array Length: %dn",arraylen);
-
- json_object * jvalue;
- json_object * jallow;
- json_bool result;
- // here should return which table's acl check failed in future.
- for (int i=0; i< arraylen; i++){
- jvalue = json_object_array_get_idx(accessObj, i);
- jallow = json_object_object_get(jvalue, "allowed");
- result = json_object_get_boolean(jallow);
- if(result != 1){
- return RANGERCHECK_NO_PRIV;
- }
- }
- return RANGERCHECK_OK;
+ Assert(buffer != NULL);
+ if (strlen(buffer) == 0)
+ return RANGERCHECK_UNKNOWN;
+
+ elog(LOG, "read from Ranger Restful API: %s", buffer);
+
+ struct json_object *response = json_tokener_parse(buffer);
+ if (response == NULL)
--- End diff --
Looks good \U0001f44b
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by interma <gi...@git.apache.org>.
Github user interma commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93182919
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -245,279 +257,326 @@ json_object *create_ranger_request_json_batch(List *args)
* }
*/
json_object* create_ranger_request_json(char* user, AclObjectKind kind, char* object,
- List* actions, bool isAll)
+ List* actions, bool isAll)
{
- Assert(user != NULL && object != NULL && privilege != NULL
- && isAll);
- ListCell *cell;
-
- elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
- user, AclObjectKindStr[kind], object);
- json_object *jrequest = json_object_new_object();
- json_object *juser = json_object_new_string(user);
-
- json_object *jaccess = json_object_new_array();
- json_object *jelement = json_object_new_object();
-
- json_object *jresource = json_object_new_object();
- switch(kind)
- {
- case ACL_KIND_CLASS:
- case ACL_KIND_SEQUENCE:
- case ACL_KIND_PROC:
- case ACL_KIND_NAMESPACE:
- case ACL_KIND_LANGUAGE:
- {
- char *ptr = NULL; char *name = NULL;
- char *first = NULL; // could be a database or protocol or tablespace
- char *second = NULL; // could be a schema or language
- char *third = NULL; // could be a table or sequence or function
- int idx = 0;
- for (name = strtok_r(object, ".", &ptr);
- name;
- name = strtok_r(NULL, ".", &ptr), idx++)
- {
- if (idx == 0)
- {
- first = pstrdup(name);
- }
- else if (idx == 1)
- {
- second = pstrdup(name);
- }
- else
- {
- third = pstrdup(name);
- }
- }
-
- if (first != NULL)
- {
- json_object *jfirst = json_object_new_string(first);
- json_object_object_add(jresource, "database", jfirst);
- }
- if (second != NULL)
- {
- json_object *jsecond = json_object_new_string(second);
- json_object_object_add(jresource,
- (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
- }
- if (third != NULL)
- {
- json_object *jthird = json_object_new_string(third);
- json_object_object_add(jresource,
- (kind == ACL_KIND_CLASS) ? "table" :
- (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
- }
-
- if (first != NULL)
- pfree(first);
- if (second != NULL)
- pfree(second);
- if (third != NULL)
- pfree(third);
- break;
- }
- case ACL_KIND_OPER:
- case ACL_KIND_CONVERSION:
- case ACL_KIND_DATABASE:
- case ACL_KIND_TABLESPACE:
- case ACL_KIND_TYPE:
- case ACL_KIND_FILESYSTEM:
- case ACL_KIND_FDW:
- case ACL_KIND_FOREIGN_SERVER:
- case ACL_KIND_EXTPROTOCOL:
- {
- json_object *jobject = json_object_new_string(object);
- json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
- break;
- }
- default:
- elog(ERROR, "unrecognized objkind: %d", (int) kind);
- }
-
- json_object *jactions = json_object_new_array();
- foreach(cell, actions)
- {
- json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
- json_object_array_add(jactions, jaction);
- }
- json_object_object_add(jelement, "resource", jresource);
- json_object_object_add(jelement, "privileges", jactions);
- json_object_array_add(jaccess, jelement);
-
- json_object_object_add(jrequest, "user", juser);
- json_object_object_add(jrequest, "access", jaccess);
- json_object *jreqid = json_object_new_string("1");
- json_object_object_add(jrequest, "requestId", jreqid);
- json_object *jclientip = json_object_new_string("123.0.0.21");
- json_object_object_add(jrequest, "clientIp", jclientip);
- json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
- json_object_object_add(jrequest, "context", jcontext);
-
-
- return jrequest;
+ Assert(user != NULL && object != NULL && privilege != NULL
+ && isAll);
+ ListCell *cell;
+
+ elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
+ user, AclObjectKindStr[kind], object);
+ json_object *jrequest = json_object_new_object();
+ json_object *juser = json_object_new_string(user);
+
+ json_object *jaccess = json_object_new_array();
+ json_object *jelement = json_object_new_object();
+
+ json_object *jresource = json_object_new_object();
+ switch(kind)
+ {
+ case ACL_KIND_CLASS:
+ case ACL_KIND_SEQUENCE:
+ case ACL_KIND_PROC:
+ case ACL_KIND_NAMESPACE:
+ case ACL_KIND_LANGUAGE:
+ {
+ char *ptr = NULL; char *name = NULL;
+ char *first = NULL; // could be a database or protocol or tablespace
+ char *second = NULL; // could be a schema or language
+ char *third = NULL; // could be a table or sequence or function
+ int idx = 0;
+ for (name = strtok_r(object, ".", &ptr);
+ name;
+ name = strtok_r(NULL, ".", &ptr), idx++)
+ {
+ if (idx == 0)
+ {
+ first = pstrdup(name);
+ }
+ else if (idx == 1)
+ {
+ second = pstrdup(name);
+ }
+ else
+ {
+ third = pstrdup(name);
+ }
+ }
+
+ if (first != NULL)
+ {
+ json_object *jfirst = json_object_new_string(first);
+ json_object_object_add(jresource, "database", jfirst);
+ }
+ if (second != NULL)
+ {
+ json_object *jsecond = json_object_new_string(second);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
+ }
+ if (third != NULL)
+ {
+ json_object *jthird = json_object_new_string(third);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_CLASS) ? "table" :
+ (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
+ }
+
+ if (first != NULL)
+ pfree(first);
+ if (second != NULL)
+ pfree(second);
+ if (third != NULL)
+ pfree(third);
+ break;
+ }
+ case ACL_KIND_OPER:
+ case ACL_KIND_CONVERSION:
+ case ACL_KIND_DATABASE:
+ case ACL_KIND_TABLESPACE:
+ case ACL_KIND_TYPE:
+ case ACL_KIND_FILESYSTEM:
+ case ACL_KIND_FDW:
+ case ACL_KIND_FOREIGN_SERVER:
+ case ACL_KIND_EXTPROTOCOL:
+ {
+ json_object *jobject = json_object_new_string(object);
+ json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
+ break;
+ }
+ default:
+ elog(ERROR, "unrecognized objkind: %d", (int) kind);
+ }
+
+ json_object *jactions = json_object_new_array();
+ foreach(cell, actions)
+ {
+ json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
+ json_object_array_add(jactions, jaction);
+ }
+ json_object_object_add(jelement, "resource", jresource);
+ json_object_object_add(jelement, "privileges", jactions);
+ json_object_array_add(jaccess, jelement);
+
+ json_object_object_add(jrequest, "user", juser);
+ json_object_object_add(jrequest, "access", jaccess);
+ json_object *jreqid = json_object_new_string("1");
+ json_object_object_add(jrequest, "requestId", jreqid);
+ json_object *jclientip = json_object_new_string("123.0.0.21");
+ json_object_object_add(jrequest, "clientIp", jclientip);
+ json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
+ json_object_object_add(jrequest, "context", jcontext);
+
+
+ return jrequest;
}
static size_t write_callback(char *contents, size_t size, size_t nitems,
- void *userp)
+ void *userp)
{
- size_t realsize = size * nitems;
- CURL_HANDLE curl = (curl_context_t *) userp;
-
- curl->response.buffer = palloc0(realsize + 1);
- memset(curl->response.buffer, 0, realsize + 1);
- if (curl->response.buffer == NULL)
- {
- /* out of memory! */
- elog(WARNING, "not enough memory for Ranger response");
- return 0;
- }
-
- memcpy(curl->response.buffer, contents, realsize);
- curl->response.size = realsize + 1;
- elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
-
- return realsize;
+ size_t realsize = size * nitems;
+ CURL_HANDLE curl = (CURL_HANDLE) userp;
+ Assert(curl != NULL);
+
+ if (curl->response.buffer == NULL)
+ {
+ curl->response.buffer = palloc0(realsize + 1);
+ }
+ else
+ {
+ //Note:
+ //our repalloc is not same as realloc, repalloc's first param(buffer) can not be NULL
+ curl->response.buffer = repalloc(curl->response.buffer, curl->response.size + realsize + 1);
+ }
+
+ if (curl->response.buffer == NULL)
+ {
+ /* out of memory! */
+ elog(WARNING, "not enough memory for Ranger response");
+ return 0;
+ }
+
+ memcpy(curl->response.buffer + curl->response.size, contents, realsize);
+ curl->response.size += realsize;
+ curl->response.buffer[curl->response.size] = '\0';
+ elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
+
+ return realsize;
}
-void call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/**
+ * @returns: 0 curl success; -1 curl failed
+ */
+int call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
{
- CURLcode res;
- Assert(request != NULL);
-
- curl_global_init(CURL_GLOBAL_ALL);
-
- /* init the curl session */
- curl_handle->curl_handle = curl_easy_init();
- if (curl_handle->curl_handle == NULL)
- {
- goto _exit;
- }
-
- /* timeout */
- // curl_easy_setopt(curl_handle, CURLOPT_TIMEOUT, 1);
-
- /* specify URL to get */
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, "http://localhost:8089/checkprivilege");
- StringInfoData tname;
- initStringInfo(&tname);
- appendStringInfo(&tname, "http://");
- appendStringInfo(&tname, rps_addr_host);
- appendStringInfo(&tname, ":");
- appendStringInfo(&tname, "%d", rps_addr_port);
- appendStringInfo(&tname, "/rps");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, tname.data);
-
- /* specify format */
- // struct curl_slist *plist = curl_slist_append(NULL, "Content-Type:application/json;charset=UTF-8");
- // curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, plist);
-
-
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDSIZE_LARGE, 1000);
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPGET, 0);
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_CUSTOMREQUEST, "POST");
-
- struct curl_slist *headers = NULL;
- //curl_slist_append(headers, "Accept: application/json");
- headers = curl_slist_append(headers, "Content-Type:application/json");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPHEADER, headers);
-
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POST, 1L);
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDS,request);
- //"{\"requestId\": 1,\"user\": \"hubert\",\"clientIp\":\"123.0.0.21\",\"context\": \"SELECT * FROM sales\",\"access\":[{\"resource\":{\"database\":\"a-database\",\"schema\":\"a-schema\",\"table\":\"sales\"},\"privileges\": [\"select\"]}]}");
- /* send all data to this function */
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEFUNCTION, write_callback);
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEDATA, (void *)curl_handle);
-
- res = curl_easy_perform(curl_handle->curl_handle);
-
- /* check for errors */
- if(res != CURLE_OK)
- {
- elog(WARNING, "curl_easy_perform() failed: %s\n",
- curl_easy_strerror(res));
- }
- else
- {
- elog(LOG, "%d bytes retrieved from Ranger Restful API.",
- curl_handle->response.size);
- }
+ int ret = -1;
+ CURLcode res;
+ Assert(request != NULL);
+
+ curl_global_init(CURL_GLOBAL_ALL);
+
+ /* init the curl session */
+ curl_handle->curl_handle = curl_easy_init();
+ if (curl_handle->curl_handle == NULL)
+ {
+ goto _exit;
+ }
+
+ // hard-coded timeout
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_TIMEOUT, 30L);
--- End diff --
Agreed, hard-code timeout now and we wil investigate in future.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #1058: HAWQ-1226. HAWQ core dump due to enable ranger w...
Posted by interma <gi...@git.apache.org>.
Github user interma commented on the issue:
https://github.com/apache/incubator-hawq/pull/1058
@xunzhang warning:
be-secure.c:322:1: warning: unused function 'report_commerror' [-Wunused-function]
report_commerror(const char *err_msg)
^
1 warning generated.
rangerrest.c:63:37: warning: 'json_object_object_get' is deprecated [-Wdeprecated-declarations]
struct json_object *accessObj = json_object_object_get(response, "access");
^
/usr/local/include/json-c/json_object.h:290:56: note: 'json_object_object_get' has been explicitly marked deprecated here
THIS_FUNCTION_IS_DEPRECATED(extern struct json_object* json_object_object_get(struct json_object* obj,
^
rangerrest.c:74:16: warning: 'json_object_object_get' is deprecated [-Wdeprecated-declarations]
jallow = json_object_object_get(jvalue, "allowed");
^
/usr/local/include/json-c/json_object.h:290:56: note: 'json_object_object_get' has been explicitly marked deprecated here
THIS_FUNCTION_IS_DEPRECATED(extern struct json_object* json_object_object_get(struct json_object* obj,
^
rangerrest.c:406:30: warning: format string is not a string literal (potentially insecure) [-Wformat-security]
appendStringInfo(&tname, rps_addr_host);
^~~~~~~~~~~~~
rangerrest.c:406:30: note: treat the string as an argument to avoid this
appendStringInfo(&tname, rps_addr_host);
^
"%s",
3 warnings generated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by xunzhang <gi...@git.apache.org>.
Github user xunzhang commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93190518
--- Diff: src/test/regress/checkinc.py ---
@@ -78,9 +78,10 @@
'winsock.h': [],
'winsock2.h': [],
'ws2tcpip.h': [],
- 'hdfs/hdfs.h': [],
+ 'hdfs/hdfs.h': [],
'quicklz1.h': [],
'quicklz3.h': [],
+ 'json-c/json.h': [],
--- End diff --
Make sense since Travis CI is green with this commit. Good fix.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by xunzhang <gi...@git.apache.org>.
Github user xunzhang commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93179777
--- Diff: src/test/regress/checkinc.py ---
@@ -78,9 +78,10 @@
'winsock.h': [],
'winsock2.h': [],
'ws2tcpip.h': [],
- 'hdfs/hdfs.h': [],
+ 'hdfs/hdfs.h': [],
'quicklz1.h': [],
'quicklz3.h': [],
+ 'json-c/json.h': [],
--- End diff --
Does `checkinc.py` apply only when executing `make install`? I think this fix is necessary but I don't have a Mac environment to test it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by zhangh43 <gi...@git.apache.org>.
Github user zhangh43 commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93183185
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -245,279 +257,326 @@ json_object *create_ranger_request_json_batch(List *args)
* }
*/
json_object* create_ranger_request_json(char* user, AclObjectKind kind, char* object,
- List* actions, bool isAll)
+ List* actions, bool isAll)
{
- Assert(user != NULL && object != NULL && privilege != NULL
- && isAll);
- ListCell *cell;
-
- elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
- user, AclObjectKindStr[kind], object);
- json_object *jrequest = json_object_new_object();
- json_object *juser = json_object_new_string(user);
-
- json_object *jaccess = json_object_new_array();
- json_object *jelement = json_object_new_object();
-
- json_object *jresource = json_object_new_object();
- switch(kind)
- {
- case ACL_KIND_CLASS:
- case ACL_KIND_SEQUENCE:
- case ACL_KIND_PROC:
- case ACL_KIND_NAMESPACE:
- case ACL_KIND_LANGUAGE:
- {
- char *ptr = NULL; char *name = NULL;
- char *first = NULL; // could be a database or protocol or tablespace
- char *second = NULL; // could be a schema or language
- char *third = NULL; // could be a table or sequence or function
- int idx = 0;
- for (name = strtok_r(object, ".", &ptr);
- name;
- name = strtok_r(NULL, ".", &ptr), idx++)
- {
- if (idx == 0)
- {
- first = pstrdup(name);
- }
- else if (idx == 1)
- {
- second = pstrdup(name);
- }
- else
- {
- third = pstrdup(name);
- }
- }
-
- if (first != NULL)
- {
- json_object *jfirst = json_object_new_string(first);
- json_object_object_add(jresource, "database", jfirst);
- }
- if (second != NULL)
- {
- json_object *jsecond = json_object_new_string(second);
- json_object_object_add(jresource,
- (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
- }
- if (third != NULL)
- {
- json_object *jthird = json_object_new_string(third);
- json_object_object_add(jresource,
- (kind == ACL_KIND_CLASS) ? "table" :
- (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
- }
-
- if (first != NULL)
- pfree(first);
- if (second != NULL)
- pfree(second);
- if (third != NULL)
- pfree(third);
- break;
- }
- case ACL_KIND_OPER:
- case ACL_KIND_CONVERSION:
- case ACL_KIND_DATABASE:
- case ACL_KIND_TABLESPACE:
- case ACL_KIND_TYPE:
- case ACL_KIND_FILESYSTEM:
- case ACL_KIND_FDW:
- case ACL_KIND_FOREIGN_SERVER:
- case ACL_KIND_EXTPROTOCOL:
- {
- json_object *jobject = json_object_new_string(object);
- json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
- break;
- }
- default:
- elog(ERROR, "unrecognized objkind: %d", (int) kind);
- }
-
- json_object *jactions = json_object_new_array();
- foreach(cell, actions)
- {
- json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
- json_object_array_add(jactions, jaction);
- }
- json_object_object_add(jelement, "resource", jresource);
- json_object_object_add(jelement, "privileges", jactions);
- json_object_array_add(jaccess, jelement);
-
- json_object_object_add(jrequest, "user", juser);
- json_object_object_add(jrequest, "access", jaccess);
- json_object *jreqid = json_object_new_string("1");
- json_object_object_add(jrequest, "requestId", jreqid);
- json_object *jclientip = json_object_new_string("123.0.0.21");
- json_object_object_add(jrequest, "clientIp", jclientip);
- json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
- json_object_object_add(jrequest, "context", jcontext);
-
-
- return jrequest;
+ Assert(user != NULL && object != NULL && privilege != NULL
+ && isAll);
+ ListCell *cell;
+
+ elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
+ user, AclObjectKindStr[kind], object);
+ json_object *jrequest = json_object_new_object();
+ json_object *juser = json_object_new_string(user);
+
+ json_object *jaccess = json_object_new_array();
+ json_object *jelement = json_object_new_object();
+
+ json_object *jresource = json_object_new_object();
+ switch(kind)
+ {
+ case ACL_KIND_CLASS:
+ case ACL_KIND_SEQUENCE:
+ case ACL_KIND_PROC:
+ case ACL_KIND_NAMESPACE:
+ case ACL_KIND_LANGUAGE:
+ {
+ char *ptr = NULL; char *name = NULL;
+ char *first = NULL; // could be a database or protocol or tablespace
+ char *second = NULL; // could be a schema or language
+ char *third = NULL; // could be a table or sequence or function
+ int idx = 0;
+ for (name = strtok_r(object, ".", &ptr);
+ name;
+ name = strtok_r(NULL, ".", &ptr), idx++)
+ {
+ if (idx == 0)
+ {
+ first = pstrdup(name);
+ }
+ else if (idx == 1)
+ {
+ second = pstrdup(name);
+ }
+ else
+ {
+ third = pstrdup(name);
+ }
+ }
+
+ if (first != NULL)
+ {
+ json_object *jfirst = json_object_new_string(first);
+ json_object_object_add(jresource, "database", jfirst);
+ }
+ if (second != NULL)
+ {
+ json_object *jsecond = json_object_new_string(second);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
+ }
+ if (third != NULL)
+ {
+ json_object *jthird = json_object_new_string(third);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_CLASS) ? "table" :
+ (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
+ }
+
+ if (first != NULL)
+ pfree(first);
+ if (second != NULL)
+ pfree(second);
+ if (third != NULL)
+ pfree(third);
+ break;
+ }
+ case ACL_KIND_OPER:
+ case ACL_KIND_CONVERSION:
+ case ACL_KIND_DATABASE:
+ case ACL_KIND_TABLESPACE:
+ case ACL_KIND_TYPE:
+ case ACL_KIND_FILESYSTEM:
+ case ACL_KIND_FDW:
+ case ACL_KIND_FOREIGN_SERVER:
+ case ACL_KIND_EXTPROTOCOL:
+ {
+ json_object *jobject = json_object_new_string(object);
+ json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
+ break;
+ }
+ default:
+ elog(ERROR, "unrecognized objkind: %d", (int) kind);
+ }
+
+ json_object *jactions = json_object_new_array();
+ foreach(cell, actions)
+ {
+ json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
+ json_object_array_add(jactions, jaction);
+ }
+ json_object_object_add(jelement, "resource", jresource);
+ json_object_object_add(jelement, "privileges", jactions);
+ json_object_array_add(jaccess, jelement);
+
+ json_object_object_add(jrequest, "user", juser);
+ json_object_object_add(jrequest, "access", jaccess);
+ json_object *jreqid = json_object_new_string("1");
+ json_object_object_add(jrequest, "requestId", jreqid);
+ json_object *jclientip = json_object_new_string("123.0.0.21");
+ json_object_object_add(jrequest, "clientIp", jclientip);
+ json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
+ json_object_object_add(jrequest, "context", jcontext);
+
+
+ return jrequest;
}
static size_t write_callback(char *contents, size_t size, size_t nitems,
- void *userp)
+ void *userp)
{
- size_t realsize = size * nitems;
- CURL_HANDLE curl = (curl_context_t *) userp;
-
- curl->response.buffer = palloc0(realsize + 1);
- memset(curl->response.buffer, 0, realsize + 1);
- if (curl->response.buffer == NULL)
- {
- /* out of memory! */
- elog(WARNING, "not enough memory for Ranger response");
- return 0;
- }
-
- memcpy(curl->response.buffer, contents, realsize);
- curl->response.size = realsize + 1;
- elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
-
- return realsize;
+ size_t realsize = size * nitems;
+ CURL_HANDLE curl = (CURL_HANDLE) userp;
+ Assert(curl != NULL);
+
+ if (curl->response.buffer == NULL)
+ {
+ curl->response.buffer = palloc0(realsize + 1);
+ }
+ else
+ {
+ //Note:
+ //our repalloc is not same as realloc, repalloc's first param(buffer) can not be NULL
+ curl->response.buffer = repalloc(curl->response.buffer, curl->response.size + realsize + 1);
+ }
+
+ if (curl->response.buffer == NULL)
+ {
+ /* out of memory! */
+ elog(WARNING, "not enough memory for Ranger response");
+ return 0;
+ }
+
+ memcpy(curl->response.buffer + curl->response.size, contents, realsize);
+ curl->response.size += realsize;
+ curl->response.buffer[curl->response.size] = '\0';
+ elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
+
+ return realsize;
}
-void call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/**
+ * @returns: 0 curl success; -1 curl failed
+ */
+int call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
{
- CURLcode res;
- Assert(request != NULL);
-
- curl_global_init(CURL_GLOBAL_ALL);
-
- /* init the curl session */
- curl_handle->curl_handle = curl_easy_init();
- if (curl_handle->curl_handle == NULL)
- {
- goto _exit;
- }
-
- /* timeout */
- // curl_easy_setopt(curl_handle, CURLOPT_TIMEOUT, 1);
-
- /* specify URL to get */
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, "http://localhost:8089/checkprivilege");
- StringInfoData tname;
- initStringInfo(&tname);
- appendStringInfo(&tname, "http://");
- appendStringInfo(&tname, rps_addr_host);
- appendStringInfo(&tname, ":");
- appendStringInfo(&tname, "%d", rps_addr_port);
- appendStringInfo(&tname, "/rps");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, tname.data);
-
- /* specify format */
- // struct curl_slist *plist = curl_slist_append(NULL, "Content-Type:application/json;charset=UTF-8");
- // curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, plist);
-
-
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDSIZE_LARGE, 1000);
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPGET, 0);
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_CUSTOMREQUEST, "POST");
-
- struct curl_slist *headers = NULL;
- //curl_slist_append(headers, "Accept: application/json");
- headers = curl_slist_append(headers, "Content-Type:application/json");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPHEADER, headers);
-
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POST, 1L);
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDS,request);
- //"{\"requestId\": 1,\"user\": \"hubert\",\"clientIp\":\"123.0.0.21\",\"context\": \"SELECT * FROM sales\",\"access\":[{\"resource\":{\"database\":\"a-database\",\"schema\":\"a-schema\",\"table\":\"sales\"},\"privileges\": [\"select\"]}]}");
- /* send all data to this function */
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEFUNCTION, write_callback);
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEDATA, (void *)curl_handle);
-
- res = curl_easy_perform(curl_handle->curl_handle);
-
- /* check for errors */
- if(res != CURLE_OK)
- {
- elog(WARNING, "curl_easy_perform() failed: %s\n",
- curl_easy_strerror(res));
- }
- else
- {
- elog(LOG, "%d bytes retrieved from Ranger Restful API.",
- curl_handle->response.size);
- }
+ int ret = -1;
+ CURLcode res;
+ Assert(request != NULL);
+
+ curl_global_init(CURL_GLOBAL_ALL);
+
+ /* init the curl session */
+ curl_handle->curl_handle = curl_easy_init();
+ if (curl_handle->curl_handle == NULL)
+ {
+ goto _exit;
+ }
+
+ // hard-coded timeout
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_TIMEOUT, 30L);
+
+ /* specify URL to get */
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, "http://localhost:8089/checkprivilege");
+ StringInfoData tname;
+ initStringInfo(&tname);
+ appendStringInfo(&tname, "http://");
+ appendStringInfo(&tname, "%s", rps_addr_host);
+ appendStringInfo(&tname, ":");
+ appendStringInfo(&tname, "%d", rps_addr_port);
+ appendStringInfo(&tname, "/rps");
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, tname.data);
+
+ /* specify format */
+ // struct curl_slist *plist = curl_slist_append(NULL, "Content-Type:application/json;charset=UTF-8");
+ // curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, plist);
+
+
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDSIZE_LARGE, 1000);
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPGET, 0);
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_CUSTOMREQUEST, "POST");
+
+ struct curl_slist *headers = NULL;
+ //curl_slist_append(headers, "Accept: application/json");
+ headers = curl_slist_append(headers, "Content-Type:application/json");
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPHEADER, headers);
+
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POST, 1L);
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDS,request);
+ //"{\"requestId\": 1,\"user\": \"hubert\",\"clientIp\":\"123.0.0.21\",\"context\": \"SELECT * FROM sales\",\"access\":[{\"resource\":{\"database\":\"a-database\",\"schema\":\"a-schema\",\"table\":\"sales\"},\"privileges\": [\"select\"]}]}");
+ /* send all data to this function */
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEFUNCTION, write_callback);
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEDATA, (void *)curl_handle);
+
+ res = curl_easy_perform(curl_handle->curl_handle);
+
+ /* check for errors */
+ if(res != CURLE_OK)
+ {
+ elog(WARNING, "curl_easy_perform() failed: %s\n",
+ curl_easy_strerror(res));
+ }
+ else
+ {
+ ret = 0;
+ elog(LOG, "%d bytes retrieved from Ranger Restful API.",
+ curl_handle->response.size);
+ }
_exit:
- /* cleanup curl stuff */
- if (curl_handle->curl_handle)
- {
- curl_easy_cleanup(curl_handle->curl_handle);
- }
-
- /* we're done with libcurl, so clean it up */
- curl_global_cleanup();
+ /* cleanup curl stuff */
+ if (curl_handle->curl_handle)
+ {
+ curl_easy_cleanup(curl_handle->curl_handle);
+ }
+
+ /* we're done with libcurl, so clean it up */
+ curl_global_cleanup();
+ return ret;
}
/*
* arg_list: List of RangerRequestJsonArgs
*/
int check_privilege_from_ranger_batch(List *arg_list)
{
- json_object* jrequest = create_ranger_request_json_batch(arg_list);
- Assert(jrequest != NULL);
- const char *request = json_object_to_json_string(jrequest);
- elog(LOG, "Send JSON request to Ranger: %s", request);
- Assert(request != NULL);
- struct curl_context_t curl_context;
- memset(&curl_context, 0, sizeof(struct curl_context_t));
-
- /* call GET method to send request*/
- call_ranger_rest(&curl_context, request);
-
- /* free the JSON object */
- json_object_put(jrequest);
-
- /* parse the JSON-format result */
- RangerACLResult ret = parse_ranger_response(curl_context.response.buffer);
-
- /* free response buffer */
- if (curl_context.response.buffer != NULL)
- {
- pfree(curl_context.response.buffer);
- }
-
- return ret;
+ json_object* jrequest = create_ranger_request_json_batch(arg_list);
+ Assert(jrequest != NULL);
+ const char *request = json_object_to_json_string(jrequest);
+ elog(LOG, "Send JSON request to Ranger: %s", request);
+ Assert(request != NULL);
+ struct curl_context_t curl_context;
+ memset(&curl_context, 0, sizeof(struct curl_context_t));
+
+ /* call GET method to send request*/
+ if (call_ranger_rest(&curl_context, request) < 0)
+ {
+ return RANGERCHECK_UNKNOWN;
+ }
+
+ /* free the JSON object */
+ json_object_put(jrequest);
+
+ /* parse the JSON-format result */
+ RangerACLResult ret = parse_ranger_response(curl_context.response.buffer);
+ /* free response buffer */
+ if (curl_context.response.buffer != NULL)
+ {
+ pfree(curl_context.response.buffer);
+ }
+
+ return ret;
}
/*
* Check the privilege from Ranger for one role
*/
int check_privilege_from_ranger(char* user, AclObjectKind kind, char* object,
- List* actions, bool isAll)
+ List* actions, bool isAll)
{
- json_object* jrequest = create_ranger_request_json(user, kind, object,
- actions, isAll);
+ json_object* jrequest = create_ranger_request_json(user, kind, object,
+ actions, isAll);
- Assert(jrequest != NULL);
- const char* request = json_object_to_json_string(jrequest);
- elog(LOG, "send JSON request to Ranger: %s", request);
- Assert(request != NULL);
+ Assert(jrequest != NULL);
+ const char* request = json_object_to_json_string(jrequest);
+ elog(LOG, "send JSON request to Ranger: %s", request);
+ Assert(request != NULL);
- struct curl_context_t curl_context;
- memset(&curl_context, 0, sizeof(struct curl_context_t));
+ struct curl_context_t curl_context;
+ memset(&curl_context, 0, sizeof(struct curl_context_t));
- /* call GET method to send request*/
- call_ranger_rest(&curl_context, request);
+ /* call GET method to send request*/
+ if (call_ranger_rest(&curl_context, request) < 0)
+ {
+ return RANGERCHECK_UNKNOWN;
--- End diff --
Here return RANGERCHECK_UNKNOWN has some problem.
check_privilege_from_ranger is call by pg_rangercheck which return AclResult. But RANGERCHECK_UNKNOWN is belong to RangerACLResult. We need to unify them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by xunzhang <gi...@git.apache.org>.
Github user xunzhang commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93179924
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -245,279 +257,326 @@ json_object *create_ranger_request_json_batch(List *args)
* }
*/
json_object* create_ranger_request_json(char* user, AclObjectKind kind, char* object,
- List* actions, bool isAll)
+ List* actions, bool isAll)
{
- Assert(user != NULL && object != NULL && privilege != NULL
- && isAll);
- ListCell *cell;
-
- elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
- user, AclObjectKindStr[kind], object);
- json_object *jrequest = json_object_new_object();
- json_object *juser = json_object_new_string(user);
-
- json_object *jaccess = json_object_new_array();
- json_object *jelement = json_object_new_object();
-
- json_object *jresource = json_object_new_object();
- switch(kind)
- {
- case ACL_KIND_CLASS:
- case ACL_KIND_SEQUENCE:
- case ACL_KIND_PROC:
- case ACL_KIND_NAMESPACE:
- case ACL_KIND_LANGUAGE:
- {
- char *ptr = NULL; char *name = NULL;
- char *first = NULL; // could be a database or protocol or tablespace
- char *second = NULL; // could be a schema or language
- char *third = NULL; // could be a table or sequence or function
- int idx = 0;
- for (name = strtok_r(object, ".", &ptr);
- name;
- name = strtok_r(NULL, ".", &ptr), idx++)
- {
- if (idx == 0)
- {
- first = pstrdup(name);
- }
- else if (idx == 1)
- {
- second = pstrdup(name);
- }
- else
- {
- third = pstrdup(name);
- }
- }
-
- if (first != NULL)
- {
- json_object *jfirst = json_object_new_string(first);
- json_object_object_add(jresource, "database", jfirst);
- }
- if (second != NULL)
- {
- json_object *jsecond = json_object_new_string(second);
- json_object_object_add(jresource,
- (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
- }
- if (third != NULL)
- {
- json_object *jthird = json_object_new_string(third);
- json_object_object_add(jresource,
- (kind == ACL_KIND_CLASS) ? "table" :
- (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
- }
-
- if (first != NULL)
- pfree(first);
- if (second != NULL)
- pfree(second);
- if (third != NULL)
- pfree(third);
- break;
- }
- case ACL_KIND_OPER:
- case ACL_KIND_CONVERSION:
- case ACL_KIND_DATABASE:
- case ACL_KIND_TABLESPACE:
- case ACL_KIND_TYPE:
- case ACL_KIND_FILESYSTEM:
- case ACL_KIND_FDW:
- case ACL_KIND_FOREIGN_SERVER:
- case ACL_KIND_EXTPROTOCOL:
- {
- json_object *jobject = json_object_new_string(object);
- json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
- break;
- }
- default:
- elog(ERROR, "unrecognized objkind: %d", (int) kind);
- }
-
- json_object *jactions = json_object_new_array();
- foreach(cell, actions)
- {
- json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
- json_object_array_add(jactions, jaction);
- }
- json_object_object_add(jelement, "resource", jresource);
- json_object_object_add(jelement, "privileges", jactions);
- json_object_array_add(jaccess, jelement);
-
- json_object_object_add(jrequest, "user", juser);
- json_object_object_add(jrequest, "access", jaccess);
- json_object *jreqid = json_object_new_string("1");
- json_object_object_add(jrequest, "requestId", jreqid);
- json_object *jclientip = json_object_new_string("123.0.0.21");
- json_object_object_add(jrequest, "clientIp", jclientip);
- json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
- json_object_object_add(jrequest, "context", jcontext);
-
-
- return jrequest;
+ Assert(user != NULL && object != NULL && privilege != NULL
+ && isAll);
+ ListCell *cell;
+
+ elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
+ user, AclObjectKindStr[kind], object);
+ json_object *jrequest = json_object_new_object();
+ json_object *juser = json_object_new_string(user);
+
+ json_object *jaccess = json_object_new_array();
+ json_object *jelement = json_object_new_object();
+
+ json_object *jresource = json_object_new_object();
+ switch(kind)
+ {
+ case ACL_KIND_CLASS:
+ case ACL_KIND_SEQUENCE:
+ case ACL_KIND_PROC:
+ case ACL_KIND_NAMESPACE:
+ case ACL_KIND_LANGUAGE:
+ {
+ char *ptr = NULL; char *name = NULL;
+ char *first = NULL; // could be a database or protocol or tablespace
+ char *second = NULL; // could be a schema or language
+ char *third = NULL; // could be a table or sequence or function
+ int idx = 0;
+ for (name = strtok_r(object, ".", &ptr);
+ name;
+ name = strtok_r(NULL, ".", &ptr), idx++)
+ {
+ if (idx == 0)
+ {
+ first = pstrdup(name);
+ }
+ else if (idx == 1)
+ {
+ second = pstrdup(name);
+ }
+ else
+ {
+ third = pstrdup(name);
+ }
+ }
+
+ if (first != NULL)
+ {
+ json_object *jfirst = json_object_new_string(first);
+ json_object_object_add(jresource, "database", jfirst);
+ }
+ if (second != NULL)
+ {
+ json_object *jsecond = json_object_new_string(second);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
+ }
+ if (third != NULL)
+ {
+ json_object *jthird = json_object_new_string(third);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_CLASS) ? "table" :
+ (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
+ }
+
+ if (first != NULL)
+ pfree(first);
+ if (second != NULL)
+ pfree(second);
+ if (third != NULL)
+ pfree(third);
+ break;
+ }
+ case ACL_KIND_OPER:
+ case ACL_KIND_CONVERSION:
+ case ACL_KIND_DATABASE:
+ case ACL_KIND_TABLESPACE:
+ case ACL_KIND_TYPE:
+ case ACL_KIND_FILESYSTEM:
+ case ACL_KIND_FDW:
+ case ACL_KIND_FOREIGN_SERVER:
+ case ACL_KIND_EXTPROTOCOL:
+ {
+ json_object *jobject = json_object_new_string(object);
+ json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
+ break;
+ }
+ default:
+ elog(ERROR, "unrecognized objkind: %d", (int) kind);
+ }
+
+ json_object *jactions = json_object_new_array();
+ foreach(cell, actions)
+ {
+ json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
+ json_object_array_add(jactions, jaction);
+ }
+ json_object_object_add(jelement, "resource", jresource);
+ json_object_object_add(jelement, "privileges", jactions);
+ json_object_array_add(jaccess, jelement);
+
+ json_object_object_add(jrequest, "user", juser);
+ json_object_object_add(jrequest, "access", jaccess);
+ json_object *jreqid = json_object_new_string("1");
+ json_object_object_add(jrequest, "requestId", jreqid);
+ json_object *jclientip = json_object_new_string("123.0.0.21");
+ json_object_object_add(jrequest, "clientIp", jclientip);
+ json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
+ json_object_object_add(jrequest, "context", jcontext);
+
+
+ return jrequest;
}
static size_t write_callback(char *contents, size_t size, size_t nitems,
- void *userp)
+ void *userp)
{
- size_t realsize = size * nitems;
- CURL_HANDLE curl = (curl_context_t *) userp;
-
- curl->response.buffer = palloc0(realsize + 1);
- memset(curl->response.buffer, 0, realsize + 1);
- if (curl->response.buffer == NULL)
- {
- /* out of memory! */
- elog(WARNING, "not enough memory for Ranger response");
- return 0;
- }
-
- memcpy(curl->response.buffer, contents, realsize);
- curl->response.size = realsize + 1;
- elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
-
- return realsize;
+ size_t realsize = size * nitems;
+ CURL_HANDLE curl = (CURL_HANDLE) userp;
+ Assert(curl != NULL);
+
+ if (curl->response.buffer == NULL)
+ {
+ curl->response.buffer = palloc0(realsize + 1);
+ }
+ else
+ {
+ //Note:
+ //our repalloc is not same as realloc, repalloc's first param(buffer) can not be NULL
+ curl->response.buffer = repalloc(curl->response.buffer, curl->response.size + realsize + 1);
+ }
+
+ if (curl->response.buffer == NULL)
+ {
+ /* out of memory! */
+ elog(WARNING, "not enough memory for Ranger response");
+ return 0;
+ }
+
+ memcpy(curl->response.buffer + curl->response.size, contents, realsize);
+ curl->response.size += realsize;
+ curl->response.buffer[curl->response.size] = '\0';
+ elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
+
+ return realsize;
}
-void call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
+
--- End diff --
You don't need to send another PR, you can append further commits in this PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #1058: HAWQ-1226. HAWQ core dump due to enable ranger w...
Posted by zhangh43 <gi...@git.apache.org>.
Github user zhangh43 commented on the issue:
https://github.com/apache/incubator-hawq/pull/1058
Return RANGERCHECK_UNKNOWN has some problem.
check_privilege_from_ranger is call by pg_rangercheck which returns AclResult. But RANGERCHECK_UNKNOWN is belong to RangerACLResult. We need to unify them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by zhangh43 <gi...@git.apache.org>.
Github user zhangh43 commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93183193
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -245,279 +257,326 @@ json_object *create_ranger_request_json_batch(List *args)
* }
*/
json_object* create_ranger_request_json(char* user, AclObjectKind kind, char* object,
- List* actions, bool isAll)
+ List* actions, bool isAll)
{
- Assert(user != NULL && object != NULL && privilege != NULL
- && isAll);
- ListCell *cell;
-
- elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
- user, AclObjectKindStr[kind], object);
- json_object *jrequest = json_object_new_object();
- json_object *juser = json_object_new_string(user);
-
- json_object *jaccess = json_object_new_array();
- json_object *jelement = json_object_new_object();
-
- json_object *jresource = json_object_new_object();
- switch(kind)
- {
- case ACL_KIND_CLASS:
- case ACL_KIND_SEQUENCE:
- case ACL_KIND_PROC:
- case ACL_KIND_NAMESPACE:
- case ACL_KIND_LANGUAGE:
- {
- char *ptr = NULL; char *name = NULL;
- char *first = NULL; // could be a database or protocol or tablespace
- char *second = NULL; // could be a schema or language
- char *third = NULL; // could be a table or sequence or function
- int idx = 0;
- for (name = strtok_r(object, ".", &ptr);
- name;
- name = strtok_r(NULL, ".", &ptr), idx++)
- {
- if (idx == 0)
- {
- first = pstrdup(name);
- }
- else if (idx == 1)
- {
- second = pstrdup(name);
- }
- else
- {
- third = pstrdup(name);
- }
- }
-
- if (first != NULL)
- {
- json_object *jfirst = json_object_new_string(first);
- json_object_object_add(jresource, "database", jfirst);
- }
- if (second != NULL)
- {
- json_object *jsecond = json_object_new_string(second);
- json_object_object_add(jresource,
- (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
- }
- if (third != NULL)
- {
- json_object *jthird = json_object_new_string(third);
- json_object_object_add(jresource,
- (kind == ACL_KIND_CLASS) ? "table" :
- (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
- }
-
- if (first != NULL)
- pfree(first);
- if (second != NULL)
- pfree(second);
- if (third != NULL)
- pfree(third);
- break;
- }
- case ACL_KIND_OPER:
- case ACL_KIND_CONVERSION:
- case ACL_KIND_DATABASE:
- case ACL_KIND_TABLESPACE:
- case ACL_KIND_TYPE:
- case ACL_KIND_FILESYSTEM:
- case ACL_KIND_FDW:
- case ACL_KIND_FOREIGN_SERVER:
- case ACL_KIND_EXTPROTOCOL:
- {
- json_object *jobject = json_object_new_string(object);
- json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
- break;
- }
- default:
- elog(ERROR, "unrecognized objkind: %d", (int) kind);
- }
-
- json_object *jactions = json_object_new_array();
- foreach(cell, actions)
- {
- json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
- json_object_array_add(jactions, jaction);
- }
- json_object_object_add(jelement, "resource", jresource);
- json_object_object_add(jelement, "privileges", jactions);
- json_object_array_add(jaccess, jelement);
-
- json_object_object_add(jrequest, "user", juser);
- json_object_object_add(jrequest, "access", jaccess);
- json_object *jreqid = json_object_new_string("1");
- json_object_object_add(jrequest, "requestId", jreqid);
- json_object *jclientip = json_object_new_string("123.0.0.21");
- json_object_object_add(jrequest, "clientIp", jclientip);
- json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
- json_object_object_add(jrequest, "context", jcontext);
-
-
- return jrequest;
+ Assert(user != NULL && object != NULL && privilege != NULL
+ && isAll);
+ ListCell *cell;
+
+ elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
+ user, AclObjectKindStr[kind], object);
+ json_object *jrequest = json_object_new_object();
+ json_object *juser = json_object_new_string(user);
+
+ json_object *jaccess = json_object_new_array();
+ json_object *jelement = json_object_new_object();
+
+ json_object *jresource = json_object_new_object();
+ switch(kind)
+ {
+ case ACL_KIND_CLASS:
+ case ACL_KIND_SEQUENCE:
+ case ACL_KIND_PROC:
+ case ACL_KIND_NAMESPACE:
+ case ACL_KIND_LANGUAGE:
+ {
+ char *ptr = NULL; char *name = NULL;
+ char *first = NULL; // could be a database or protocol or tablespace
+ char *second = NULL; // could be a schema or language
+ char *third = NULL; // could be a table or sequence or function
+ int idx = 0;
+ for (name = strtok_r(object, ".", &ptr);
+ name;
+ name = strtok_r(NULL, ".", &ptr), idx++)
+ {
+ if (idx == 0)
+ {
+ first = pstrdup(name);
+ }
+ else if (idx == 1)
+ {
+ second = pstrdup(name);
+ }
+ else
+ {
+ third = pstrdup(name);
+ }
+ }
+
+ if (first != NULL)
+ {
+ json_object *jfirst = json_object_new_string(first);
+ json_object_object_add(jresource, "database", jfirst);
+ }
+ if (second != NULL)
+ {
+ json_object *jsecond = json_object_new_string(second);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
+ }
+ if (third != NULL)
+ {
+ json_object *jthird = json_object_new_string(third);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_CLASS) ? "table" :
+ (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
+ }
+
+ if (first != NULL)
+ pfree(first);
+ if (second != NULL)
+ pfree(second);
+ if (third != NULL)
+ pfree(third);
+ break;
+ }
+ case ACL_KIND_OPER:
+ case ACL_KIND_CONVERSION:
+ case ACL_KIND_DATABASE:
+ case ACL_KIND_TABLESPACE:
+ case ACL_KIND_TYPE:
+ case ACL_KIND_FILESYSTEM:
+ case ACL_KIND_FDW:
+ case ACL_KIND_FOREIGN_SERVER:
+ case ACL_KIND_EXTPROTOCOL:
+ {
+ json_object *jobject = json_object_new_string(object);
+ json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
+ break;
+ }
+ default:
+ elog(ERROR, "unrecognized objkind: %d", (int) kind);
+ }
+
+ json_object *jactions = json_object_new_array();
+ foreach(cell, actions)
+ {
+ json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
+ json_object_array_add(jactions, jaction);
+ }
+ json_object_object_add(jelement, "resource", jresource);
+ json_object_object_add(jelement, "privileges", jactions);
+ json_object_array_add(jaccess, jelement);
+
+ json_object_object_add(jrequest, "user", juser);
+ json_object_object_add(jrequest, "access", jaccess);
+ json_object *jreqid = json_object_new_string("1");
+ json_object_object_add(jrequest, "requestId", jreqid);
+ json_object *jclientip = json_object_new_string("123.0.0.21");
+ json_object_object_add(jrequest, "clientIp", jclientip);
+ json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
+ json_object_object_add(jrequest, "context", jcontext);
+
+
+ return jrequest;
}
static size_t write_callback(char *contents, size_t size, size_t nitems,
- void *userp)
+ void *userp)
{
- size_t realsize = size * nitems;
- CURL_HANDLE curl = (curl_context_t *) userp;
-
- curl->response.buffer = palloc0(realsize + 1);
- memset(curl->response.buffer, 0, realsize + 1);
- if (curl->response.buffer == NULL)
- {
- /* out of memory! */
- elog(WARNING, "not enough memory for Ranger response");
- return 0;
- }
-
- memcpy(curl->response.buffer, contents, realsize);
- curl->response.size = realsize + 1;
- elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
-
- return realsize;
+ size_t realsize = size * nitems;
+ CURL_HANDLE curl = (CURL_HANDLE) userp;
+ Assert(curl != NULL);
+
+ if (curl->response.buffer == NULL)
+ {
+ curl->response.buffer = palloc0(realsize + 1);
+ }
+ else
+ {
+ //Note:
+ //our repalloc is not same as realloc, repalloc's first param(buffer) can not be NULL
+ curl->response.buffer = repalloc(curl->response.buffer, curl->response.size + realsize + 1);
+ }
+
+ if (curl->response.buffer == NULL)
+ {
+ /* out of memory! */
+ elog(WARNING, "not enough memory for Ranger response");
+ return 0;
+ }
+
+ memcpy(curl->response.buffer + curl->response.size, contents, realsize);
+ curl->response.size += realsize;
+ curl->response.buffer[curl->response.size] = '\0';
+ elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
+
+ return realsize;
}
-void call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/**
+ * @returns: 0 curl success; -1 curl failed
+ */
+int call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
{
- CURLcode res;
- Assert(request != NULL);
-
- curl_global_init(CURL_GLOBAL_ALL);
-
- /* init the curl session */
- curl_handle->curl_handle = curl_easy_init();
- if (curl_handle->curl_handle == NULL)
- {
- goto _exit;
- }
-
- /* timeout */
- // curl_easy_setopt(curl_handle, CURLOPT_TIMEOUT, 1);
-
- /* specify URL to get */
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, "http://localhost:8089/checkprivilege");
- StringInfoData tname;
- initStringInfo(&tname);
- appendStringInfo(&tname, "http://");
- appendStringInfo(&tname, rps_addr_host);
- appendStringInfo(&tname, ":");
- appendStringInfo(&tname, "%d", rps_addr_port);
- appendStringInfo(&tname, "/rps");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, tname.data);
-
- /* specify format */
- // struct curl_slist *plist = curl_slist_append(NULL, "Content-Type:application/json;charset=UTF-8");
- // curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, plist);
-
-
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDSIZE_LARGE, 1000);
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPGET, 0);
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_CUSTOMREQUEST, "POST");
-
- struct curl_slist *headers = NULL;
- //curl_slist_append(headers, "Accept: application/json");
- headers = curl_slist_append(headers, "Content-Type:application/json");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPHEADER, headers);
-
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POST, 1L);
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDS,request);
- //"{\"requestId\": 1,\"user\": \"hubert\",\"clientIp\":\"123.0.0.21\",\"context\": \"SELECT * FROM sales\",\"access\":[{\"resource\":{\"database\":\"a-database\",\"schema\":\"a-schema\",\"table\":\"sales\"},\"privileges\": [\"select\"]}]}");
- /* send all data to this function */
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEFUNCTION, write_callback);
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEDATA, (void *)curl_handle);
-
- res = curl_easy_perform(curl_handle->curl_handle);
-
- /* check for errors */
- if(res != CURLE_OK)
- {
- elog(WARNING, "curl_easy_perform() failed: %s\n",
- curl_easy_strerror(res));
- }
- else
- {
- elog(LOG, "%d bytes retrieved from Ranger Restful API.",
- curl_handle->response.size);
- }
+ int ret = -1;
+ CURLcode res;
+ Assert(request != NULL);
+
+ curl_global_init(CURL_GLOBAL_ALL);
+
+ /* init the curl session */
+ curl_handle->curl_handle = curl_easy_init();
+ if (curl_handle->curl_handle == NULL)
+ {
+ goto _exit;
+ }
+
+ // hard-coded timeout
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_TIMEOUT, 30L);
+
+ /* specify URL to get */
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, "http://localhost:8089/checkprivilege");
+ StringInfoData tname;
+ initStringInfo(&tname);
+ appendStringInfo(&tname, "http://");
+ appendStringInfo(&tname, "%s", rps_addr_host);
+ appendStringInfo(&tname, ":");
+ appendStringInfo(&tname, "%d", rps_addr_port);
+ appendStringInfo(&tname, "/rps");
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, tname.data);
+
+ /* specify format */
+ // struct curl_slist *plist = curl_slist_append(NULL, "Content-Type:application/json;charset=UTF-8");
+ // curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, plist);
+
+
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDSIZE_LARGE, 1000);
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPGET, 0);
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_CUSTOMREQUEST, "POST");
+
+ struct curl_slist *headers = NULL;
+ //curl_slist_append(headers, "Accept: application/json");
+ headers = curl_slist_append(headers, "Content-Type:application/json");
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPHEADER, headers);
+
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POST, 1L);
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDS,request);
+ //"{\"requestId\": 1,\"user\": \"hubert\",\"clientIp\":\"123.0.0.21\",\"context\": \"SELECT * FROM sales\",\"access\":[{\"resource\":{\"database\":\"a-database\",\"schema\":\"a-schema\",\"table\":\"sales\"},\"privileges\": [\"select\"]}]}");
+ /* send all data to this function */
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEFUNCTION, write_callback);
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEDATA, (void *)curl_handle);
+
+ res = curl_easy_perform(curl_handle->curl_handle);
+
+ /* check for errors */
+ if(res != CURLE_OK)
+ {
+ elog(WARNING, "curl_easy_perform() failed: %s\n",
+ curl_easy_strerror(res));
+ }
+ else
+ {
+ ret = 0;
+ elog(LOG, "%d bytes retrieved from Ranger Restful API.",
+ curl_handle->response.size);
+ }
_exit:
- /* cleanup curl stuff */
- if (curl_handle->curl_handle)
- {
- curl_easy_cleanup(curl_handle->curl_handle);
- }
-
- /* we're done with libcurl, so clean it up */
- curl_global_cleanup();
+ /* cleanup curl stuff */
+ if (curl_handle->curl_handle)
+ {
+ curl_easy_cleanup(curl_handle->curl_handle);
+ }
+
+ /* we're done with libcurl, so clean it up */
+ curl_global_cleanup();
+ return ret;
}
/*
* arg_list: List of RangerRequestJsonArgs
*/
int check_privilege_from_ranger_batch(List *arg_list)
{
- json_object* jrequest = create_ranger_request_json_batch(arg_list);
- Assert(jrequest != NULL);
- const char *request = json_object_to_json_string(jrequest);
- elog(LOG, "Send JSON request to Ranger: %s", request);
- Assert(request != NULL);
- struct curl_context_t curl_context;
- memset(&curl_context, 0, sizeof(struct curl_context_t));
-
- /* call GET method to send request*/
- call_ranger_rest(&curl_context, request);
-
- /* free the JSON object */
- json_object_put(jrequest);
-
- /* parse the JSON-format result */
- RangerACLResult ret = parse_ranger_response(curl_context.response.buffer);
-
- /* free response buffer */
- if (curl_context.response.buffer != NULL)
- {
- pfree(curl_context.response.buffer);
- }
-
- return ret;
+ json_object* jrequest = create_ranger_request_json_batch(arg_list);
+ Assert(jrequest != NULL);
+ const char *request = json_object_to_json_string(jrequest);
+ elog(LOG, "Send JSON request to Ranger: %s", request);
+ Assert(request != NULL);
+ struct curl_context_t curl_context;
+ memset(&curl_context, 0, sizeof(struct curl_context_t));
+
+ /* call GET method to send request*/
+ if (call_ranger_rest(&curl_context, request) < 0)
+ {
+ return RANGERCHECK_UNKNOWN;
+ }
+
+ /* free the JSON object */
+ json_object_put(jrequest);
+
+ /* parse the JSON-format result */
+ RangerACLResult ret = parse_ranger_response(curl_context.response.buffer);
+ /* free response buffer */
+ if (curl_context.response.buffer != NULL)
+ {
+ pfree(curl_context.response.buffer);
+ }
+
+ return ret;
}
/*
* Check the privilege from Ranger for one role
*/
int check_privilege_from_ranger(char* user, AclObjectKind kind, char* object,
- List* actions, bool isAll)
+ List* actions, bool isAll)
{
- json_object* jrequest = create_ranger_request_json(user, kind, object,
- actions, isAll);
+ json_object* jrequest = create_ranger_request_json(user, kind, object,
+ actions, isAll);
- Assert(jrequest != NULL);
- const char* request = json_object_to_json_string(jrequest);
- elog(LOG, "send JSON request to Ranger: %s", request);
- Assert(request != NULL);
+ Assert(jrequest != NULL);
+ const char* request = json_object_to_json_string(jrequest);
+ elog(LOG, "send JSON request to Ranger: %s", request);
+ Assert(request != NULL);
- struct curl_context_t curl_context;
- memset(&curl_context, 0, sizeof(struct curl_context_t));
+ struct curl_context_t curl_context;
+ memset(&curl_context, 0, sizeof(struct curl_context_t));
- /* call GET method to send request*/
- call_ranger_rest(&curl_context, request);
+ /* call GET method to send request*/
+ if (call_ranger_rest(&curl_context, request) < 0)
+ {
+ return RANGERCHECK_UNKNOWN;
--- End diff --
Here return RANGERCHECK_UNKNOWN has some problem.
check_privilege_from_ranger is call by pg_rangercheck which return AclResult. But RANGERCHECK_UNKNOWN is belong to RangerACLResult. We need to unify them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #1058: HAWQ-1226. HAWQ core dump due to enable ranger w...
Posted by xunzhang <gi...@git.apache.org>.
Github user xunzhang commented on the issue:
https://github.com/apache/incubator-hawq/pull/1058
@interma Could you please attach the compile warning message you fixed here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by linwen <gi...@git.apache.org>.
Github user linwen commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93189866
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -272,280 +286,300 @@ json_object *create_ranger_request_json_batch(List *args)
* ]
* }
*/
-json_object* create_ranger_request_json(char* user, AclObjectKind kind, char* object,
- List* actions, char* how)
+static json_object* create_ranger_request_json(char* user, AclObjectKind kind, char* object,
+ List* actions, char* how)
{
- Assert(user != NULL && object != NULL && privilege != NULL
- && how != NULL);
- ListCell *cell;
-
- elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
- user, AclObjectKindStr[kind], object);
- json_object *jrequest = json_object_new_object();
- json_object *juser = json_object_new_string(user);
-
- json_object *jaccess = json_object_new_array();
- json_object *jelement = json_object_new_object();
-
- json_object *jresource = json_object_new_object();
- switch(kind)
- {
- case ACL_KIND_CLASS:
- case ACL_KIND_SEQUENCE:
- case ACL_KIND_PROC:
- case ACL_KIND_NAMESPACE:
- case ACL_KIND_LANGUAGE:
- {
- char *ptr = NULL; char *name = NULL;
- char *first = NULL; // could be a database or protocol or tablespace
- char *second = NULL; // could be a schema or language
- char *third = NULL; // could be a table or sequence or function
- int idx = 0;
- for (name = strtok_r(object, ".", &ptr);
- name;
- name = strtok_r(NULL, ".", &ptr), idx++)
- {
- if (idx == 0)
- {
- first = pstrdup(name);
- }
- else if (idx == 1)
- {
- second = pstrdup(name);
- }
- else
- {
- third = pstrdup(name);
- }
- }
-
- if (first != NULL)
- {
- json_object *jfirst = json_object_new_string(first);
- json_object_object_add(jresource, "database", jfirst);
- }
- if (second != NULL)
- {
- json_object *jsecond = json_object_new_string(second);
- json_object_object_add(jresource,
- (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
- }
- if (third != NULL)
- {
- json_object *jthird = json_object_new_string(third);
- json_object_object_add(jresource,
- (kind == ACL_KIND_CLASS) ? "table" :
- (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
- }
-
- if (first != NULL)
- pfree(first);
- if (second != NULL)
- pfree(second);
- if (third != NULL)
- pfree(third);
- break;
- }
- case ACL_KIND_OPER:
- case ACL_KIND_CONVERSION:
- case ACL_KIND_DATABASE:
- case ACL_KIND_TABLESPACE:
- case ACL_KIND_TYPE:
- case ACL_KIND_FILESYSTEM:
- case ACL_KIND_FDW:
- case ACL_KIND_FOREIGN_SERVER:
- case ACL_KIND_EXTPROTOCOL:
- {
- json_object *jobject = json_object_new_string(object);
- json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
- break;
- }
- default:
- elog(ERROR, "unrecognized objkind: %d", (int) kind);
- }
-
- json_object *jactions = json_object_new_array();
- foreach(cell, actions)
- {
- json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
- json_object_array_add(jactions, jaction);
- }
- json_object_object_add(jelement, "resource", jresource);
- json_object_object_add(jelement, "privileges", jactions);
- json_object_array_add(jaccess, jelement);
-
- json_object_object_add(jrequest, "user", juser);
- json_object_object_add(jrequest, "access", jaccess);
- json_object *jreqid = json_object_new_string("1");
- json_object_object_add(jrequest, "requestId", jreqid);
- json_object *jclientip = json_object_new_string("123.0.0.21");
- json_object_object_add(jrequest, "clientIp", jclientip);
- json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
- json_object_object_add(jrequest, "context", jcontext);
-
-
- return jrequest;
+ Assert(user != NULL && object != NULL && privilege != NULL
+ && how != NULL);
+ ListCell *cell;
+
+ elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
+ user, AclObjectKindStr[kind], object);
+ json_object *jrequest = json_object_new_object();
+ json_object *juser = json_object_new_string(user);
+
+ json_object *jaccess = json_object_new_array();
+ json_object *jelement = json_object_new_object();
+
+ json_object *jresource = json_object_new_object();
+ switch(kind)
+ {
+ case ACL_KIND_CLASS:
+ case ACL_KIND_SEQUENCE:
+ case ACL_KIND_PROC:
+ case ACL_KIND_NAMESPACE:
+ case ACL_KIND_LANGUAGE:
+ {
+ char *ptr = NULL; char *name = NULL;
+ char *first = NULL; // could be a database or protocol or tablespace
+ char *second = NULL; // could be a schema or language
+ char *third = NULL; // could be a table or sequence or function
+ int idx = 0;
+ for (name = strtok_r(object, ".", &ptr);
+ name;
+ name = strtok_r(NULL, ".", &ptr), idx++)
+ {
+ if (idx == 0)
+ {
+ first = pstrdup(name);
+ }
+ else if (idx == 1)
+ {
+ second = pstrdup(name);
+ }
+ else
+ {
+ third = pstrdup(name);
+ }
+ }
+
+ if (first != NULL)
+ {
+ json_object *jfirst = json_object_new_string(first);
+ json_object_object_add(jresource, "database", jfirst);
+ }
+ if (second != NULL)
+ {
+ json_object *jsecond = json_object_new_string(second);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
+ }
+ if (third != NULL)
+ {
+ json_object *jthird = json_object_new_string(third);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_CLASS) ? "table" :
+ (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
+ }
+
+ if (first != NULL)
+ pfree(first);
+ if (second != NULL)
+ pfree(second);
+ if (third != NULL)
+ pfree(third);
+ break;
+ }
+ case ACL_KIND_OPER:
+ case ACL_KIND_CONVERSION:
+ case ACL_KIND_DATABASE:
+ case ACL_KIND_TABLESPACE:
+ case ACL_KIND_TYPE:
+ case ACL_KIND_FILESYSTEM:
+ case ACL_KIND_FDW:
+ case ACL_KIND_FOREIGN_SERVER:
+ case ACL_KIND_EXTPROTOCOL:
+ {
+ json_object *jobject = json_object_new_string(object);
+ json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
+ break;
+ }
+ default:
+ elog(ERROR, "unrecognized objkind: %d", (int) kind);
+ }
+
+ json_object *jactions = json_object_new_array();
+ foreach(cell, actions)
+ {
+ json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
+ json_object_array_add(jactions, jaction);
+ }
+ json_object_object_add(jelement, "resource", jresource);
+ json_object_object_add(jelement, "privileges", jactions);
+ json_object_array_add(jaccess, jelement);
+
+ json_object_object_add(jrequest, "user", juser);
+ json_object_object_add(jrequest, "access", jaccess);
+ json_object *jreqid = json_object_new_string("1");
+ json_object_object_add(jrequest, "requestId", jreqid);
+ json_object *jclientip = json_object_new_string("123.0.0.21");
+ json_object_object_add(jrequest, "clientIp", jclientip);
+ json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
+ json_object_object_add(jrequest, "context", jcontext);
+
+
+ return jrequest;
}
static size_t write_callback(char *contents, size_t size, size_t nitems,
- void *userp)
+ void *userp)
{
- size_t realsize = size * nitems;
- CURL_HANDLE curl = (struct curl_context *) userp;
-
- curl->response.buffer = palloc0(realsize + 1);
- memset(curl->response.buffer, 0, realsize + 1);
- if (curl->response.buffer == NULL)
- {
- /* out of memory! */
- elog(WARNING, "not enough memory for Ranger response");
- return 0;
- }
-
- memcpy(curl->response.buffer, contents, realsize);
- curl->response.size = realsize + 1;
- elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
-
- return realsize;
+ size_t realsize = size * nitems;
+ CURL_HANDLE curl = (CURL_HANDLE) userp;
+ Assert(curl != NULL);
+
+ if (curl->response.buffer == NULL)
+ {
+ curl->response.buffer = palloc0(realsize + 1);
+ //memset(curl->response.buffer, 0, realsize + 1);
+ }
+ else
+ {
+ //Note: if curl->response.buffer==NULL, repalloc will core dump
+ curl->response.buffer = repalloc(curl->response.buffer, curl->response.size + realsize + 1);
+ }
+
+ if (curl->response.buffer == NULL)
+ {
+ /* out of memory! */
+ elog(WARNING, "not enough memory for Ranger response");
+ return 0;
+ }
+
+ memcpy(curl->response.buffer + curl->response.size, contents, realsize);
+ curl->response.size += realsize;
+ curl->response.buffer[curl->response.size] = '\0';
+ elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
+
+ return realsize;
}
-void call_ranger_rest(CURL_HANDLE curl_handle, char* request)
+/**
+ * @returns: 0 curl success; -1 curl failed
+ */
+static int call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
{
- CURLcode res;
- Assert(request != NULL);
-
- curl_global_init(CURL_GLOBAL_ALL);
-
- /* init the curl session */
- curl_handle->curl_handle = curl_easy_init();
- if (curl_handle->curl_handle == NULL)
- {
- goto _exit;
- }
-
- /* timeout */
- // curl_easy_setopt(curl_handle, CURLOPT_TIMEOUT, 1);
-
- /* specify URL to get */
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, "http://localhost:8089/checkprivilege");
- StringInfoData tname;
- initStringInfo(&tname);
- appendStringInfo(&tname, "http://");
- appendStringInfo(&tname, rps_addr_host);
- appendStringInfo(&tname, ":");
- appendStringInfo(&tname, "%d", rps_addr_port);
- appendStringInfo(&tname, "/rps");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, tname.data);
-
- /* specify format */
- // struct curl_slist *plist = curl_slist_append(NULL, "Content-Type:application/json;charset=UTF-8");
- // curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, plist);
-
-
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDSIZE_LARGE, 1000);
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPGET, 0);
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_CUSTOMREQUEST, "POST");
-
- struct curl_slist *headers = NULL;
- //curl_slist_append(headers, "Accept: application/json");
- headers = curl_slist_append(headers, "Content-Type:application/json");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPHEADER, headers);
-
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POST, 1L);
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDS,request);
- //"{\"requestId\": 1,\"user\": \"hubert\",\"clientIp\":\"123.0.0.21\",\"context\": \"SELECT * FROM sales\",\"access\":[{\"resource\":{\"database\":\"a-database\",\"schema\":\"a-schema\",\"table\":\"sales\"},\"privileges\": [\"select\"]}]}");
- /* send all data to this function */
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEFUNCTION, write_callback);
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEDATA, (void *)curl_handle);
-
- res = curl_easy_perform(curl_handle->curl_handle);
-
- /* check for errors */
- if(res != CURLE_OK)
- {
- elog(WARNING, "curl_easy_perform() failed: %s\n",
- curl_easy_strerror(res));
- }
- else
- {
- elog(LOG, "%lu bytes retrieved from Ranger Restful API.",
- curl_handle->response.size);
- }
+ int ret = -1;
+ CURLcode res;
+ Assert(request != NULL);
+
+ curl_global_init(CURL_GLOBAL_ALL);
+
+ /* init the curl session */
+ curl_handle->curl_handle = curl_easy_init();
+ if (curl_handle->curl_handle == NULL)
+ {
+ goto _exit;
+ }
+
+ // hard core timeout
--- End diff --
\uff0f* timeout is 30 seconds *\uff0f
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by interma <gi...@git.apache.org>.
Github user interma closed the pull request at:
https://github.com/apache/incubator-hawq/pull/1058
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by interma <gi...@git.apache.org>.
Github user interma commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93179635
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -245,279 +257,326 @@ json_object *create_ranger_request_json_batch(List *args)
* }
*/
json_object* create_ranger_request_json(char* user, AclObjectKind kind, char* object,
- List* actions, bool isAll)
+ List* actions, bool isAll)
{
- Assert(user != NULL && object != NULL && privilege != NULL
- && isAll);
- ListCell *cell;
-
- elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
- user, AclObjectKindStr[kind], object);
- json_object *jrequest = json_object_new_object();
- json_object *juser = json_object_new_string(user);
-
- json_object *jaccess = json_object_new_array();
- json_object *jelement = json_object_new_object();
-
- json_object *jresource = json_object_new_object();
- switch(kind)
- {
- case ACL_KIND_CLASS:
- case ACL_KIND_SEQUENCE:
- case ACL_KIND_PROC:
- case ACL_KIND_NAMESPACE:
- case ACL_KIND_LANGUAGE:
- {
- char *ptr = NULL; char *name = NULL;
- char *first = NULL; // could be a database or protocol or tablespace
- char *second = NULL; // could be a schema or language
- char *third = NULL; // could be a table or sequence or function
- int idx = 0;
- for (name = strtok_r(object, ".", &ptr);
- name;
- name = strtok_r(NULL, ".", &ptr), idx++)
- {
- if (idx == 0)
- {
- first = pstrdup(name);
- }
- else if (idx == 1)
- {
- second = pstrdup(name);
- }
- else
- {
- third = pstrdup(name);
- }
- }
-
- if (first != NULL)
- {
- json_object *jfirst = json_object_new_string(first);
- json_object_object_add(jresource, "database", jfirst);
- }
- if (second != NULL)
- {
- json_object *jsecond = json_object_new_string(second);
- json_object_object_add(jresource,
- (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
- }
- if (third != NULL)
- {
- json_object *jthird = json_object_new_string(third);
- json_object_object_add(jresource,
- (kind == ACL_KIND_CLASS) ? "table" :
- (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
- }
-
- if (first != NULL)
- pfree(first);
- if (second != NULL)
- pfree(second);
- if (third != NULL)
- pfree(third);
- break;
- }
- case ACL_KIND_OPER:
- case ACL_KIND_CONVERSION:
- case ACL_KIND_DATABASE:
- case ACL_KIND_TABLESPACE:
- case ACL_KIND_TYPE:
- case ACL_KIND_FILESYSTEM:
- case ACL_KIND_FDW:
- case ACL_KIND_FOREIGN_SERVER:
- case ACL_KIND_EXTPROTOCOL:
- {
- json_object *jobject = json_object_new_string(object);
- json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
- break;
- }
- default:
- elog(ERROR, "unrecognized objkind: %d", (int) kind);
- }
-
- json_object *jactions = json_object_new_array();
- foreach(cell, actions)
- {
- json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
- json_object_array_add(jactions, jaction);
- }
- json_object_object_add(jelement, "resource", jresource);
- json_object_object_add(jelement, "privileges", jactions);
- json_object_array_add(jaccess, jelement);
-
- json_object_object_add(jrequest, "user", juser);
- json_object_object_add(jrequest, "access", jaccess);
- json_object *jreqid = json_object_new_string("1");
- json_object_object_add(jrequest, "requestId", jreqid);
- json_object *jclientip = json_object_new_string("123.0.0.21");
- json_object_object_add(jrequest, "clientIp", jclientip);
- json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
- json_object_object_add(jrequest, "context", jcontext);
-
-
- return jrequest;
+ Assert(user != NULL && object != NULL && privilege != NULL
+ && isAll);
+ ListCell *cell;
+
+ elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
+ user, AclObjectKindStr[kind], object);
+ json_object *jrequest = json_object_new_object();
+ json_object *juser = json_object_new_string(user);
+
+ json_object *jaccess = json_object_new_array();
+ json_object *jelement = json_object_new_object();
+
+ json_object *jresource = json_object_new_object();
+ switch(kind)
+ {
+ case ACL_KIND_CLASS:
+ case ACL_KIND_SEQUENCE:
+ case ACL_KIND_PROC:
+ case ACL_KIND_NAMESPACE:
+ case ACL_KIND_LANGUAGE:
+ {
+ char *ptr = NULL; char *name = NULL;
+ char *first = NULL; // could be a database or protocol or tablespace
+ char *second = NULL; // could be a schema or language
+ char *third = NULL; // could be a table or sequence or function
+ int idx = 0;
+ for (name = strtok_r(object, ".", &ptr);
+ name;
+ name = strtok_r(NULL, ".", &ptr), idx++)
+ {
+ if (idx == 0)
+ {
+ first = pstrdup(name);
+ }
+ else if (idx == 1)
+ {
+ second = pstrdup(name);
+ }
+ else
+ {
+ third = pstrdup(name);
+ }
+ }
+
+ if (first != NULL)
+ {
+ json_object *jfirst = json_object_new_string(first);
+ json_object_object_add(jresource, "database", jfirst);
+ }
+ if (second != NULL)
+ {
+ json_object *jsecond = json_object_new_string(second);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
+ }
+ if (third != NULL)
+ {
+ json_object *jthird = json_object_new_string(third);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_CLASS) ? "table" :
+ (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
+ }
+
+ if (first != NULL)
+ pfree(first);
+ if (second != NULL)
+ pfree(second);
+ if (third != NULL)
+ pfree(third);
+ break;
+ }
+ case ACL_KIND_OPER:
+ case ACL_KIND_CONVERSION:
+ case ACL_KIND_DATABASE:
+ case ACL_KIND_TABLESPACE:
+ case ACL_KIND_TYPE:
+ case ACL_KIND_FILESYSTEM:
+ case ACL_KIND_FDW:
+ case ACL_KIND_FOREIGN_SERVER:
+ case ACL_KIND_EXTPROTOCOL:
+ {
+ json_object *jobject = json_object_new_string(object);
+ json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
+ break;
+ }
+ default:
+ elog(ERROR, "unrecognized objkind: %d", (int) kind);
+ }
+
+ json_object *jactions = json_object_new_array();
+ foreach(cell, actions)
+ {
+ json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
+ json_object_array_add(jactions, jaction);
+ }
+ json_object_object_add(jelement, "resource", jresource);
+ json_object_object_add(jelement, "privileges", jactions);
+ json_object_array_add(jaccess, jelement);
+
+ json_object_object_add(jrequest, "user", juser);
+ json_object_object_add(jrequest, "access", jaccess);
+ json_object *jreqid = json_object_new_string("1");
+ json_object_object_add(jrequest, "requestId", jreqid);
+ json_object *jclientip = json_object_new_string("123.0.0.21");
+ json_object_object_add(jrequest, "clientIp", jclientip);
+ json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
+ json_object_object_add(jrequest, "context", jcontext);
+
+
+ return jrequest;
}
static size_t write_callback(char *contents, size_t size, size_t nitems,
- void *userp)
+ void *userp)
{
- size_t realsize = size * nitems;
- CURL_HANDLE curl = (curl_context_t *) userp;
-
- curl->response.buffer = palloc0(realsize + 1);
- memset(curl->response.buffer, 0, realsize + 1);
- if (curl->response.buffer == NULL)
- {
- /* out of memory! */
- elog(WARNING, "not enough memory for Ranger response");
- return 0;
- }
-
- memcpy(curl->response.buffer, contents, realsize);
- curl->response.size = realsize + 1;
- elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
-
- return realsize;
+ size_t realsize = size * nitems;
+ CURL_HANDLE curl = (CURL_HANDLE) userp;
+ Assert(curl != NULL);
+
+ if (curl->response.buffer == NULL)
+ {
+ curl->response.buffer = palloc0(realsize + 1);
+ }
+ else
+ {
+ //Note:
+ //our repalloc is not same as realloc, repalloc's first param(buffer) can not be NULL
+ curl->response.buffer = repalloc(curl->response.buffer, curl->response.size + realsize + 1);
+ }
+
+ if (curl->response.buffer == NULL)
+ {
+ /* out of memory! */
+ elog(WARNING, "not enough memory for Ranger response");
+ return 0;
+ }
+
+ memcpy(curl->response.buffer + curl->response.size, contents, realsize);
+ curl->response.size += realsize;
+ curl->response.buffer[curl->response.size] = '\0';
+ elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
+
+ return realsize;
}
-void call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
+
--- End diff --
Tks!
I will fix it in next PR by me or others.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #1058: HAWQ-1226. HAWQ core dump due to enable ranger w...
Posted by stanlyxiang <gi...@git.apache.org>.
Github user stanlyxiang commented on the issue:
https://github.com/apache/incubator-hawq/pull/1058
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by stanlyxiang <gi...@git.apache.org>.
Github user stanlyxiang commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93179019
--- Diff: src/backend/libpq/be-secure.c ---
@@ -329,6 +331,7 @@ report_commerror(const char *err_msg)
RESUME_INTERRUPTS();
}
+*/
--- End diff --
If it is unused now, could we remove it directly ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #1058: HAWQ-1226. HAWQ core dump due to enable ranger w...
Posted by interma <gi...@git.apache.org>.
Github user interma commented on the issue:
https://github.com/apache/incubator-hawq/pull/1058
@linwen @xunzhang @stanlyxiang @zhangh43 help to review, thanks.
1. fix core dump while RPS not running
rangerrest.c:435
2. fix libcurl write_callback() problem
rangerrest.c:374
3. add libcurl timeout (hard-code 30s now)
rangerrest.c:451
4. fix some compile warnings
5. fix compile error: add json.h to checkinc.py
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by interma <gi...@git.apache.org>.
Github user interma commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93184941
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -245,279 +257,326 @@ json_object *create_ranger_request_json_batch(List *args)
* }
*/
json_object* create_ranger_request_json(char* user, AclObjectKind kind, char* object,
- List* actions, bool isAll)
+ List* actions, bool isAll)
{
- Assert(user != NULL && object != NULL && privilege != NULL
- && isAll);
- ListCell *cell;
-
- elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
- user, AclObjectKindStr[kind], object);
- json_object *jrequest = json_object_new_object();
- json_object *juser = json_object_new_string(user);
-
- json_object *jaccess = json_object_new_array();
- json_object *jelement = json_object_new_object();
-
- json_object *jresource = json_object_new_object();
- switch(kind)
- {
- case ACL_KIND_CLASS:
- case ACL_KIND_SEQUENCE:
- case ACL_KIND_PROC:
- case ACL_KIND_NAMESPACE:
- case ACL_KIND_LANGUAGE:
- {
- char *ptr = NULL; char *name = NULL;
- char *first = NULL; // could be a database or protocol or tablespace
- char *second = NULL; // could be a schema or language
- char *third = NULL; // could be a table or sequence or function
- int idx = 0;
- for (name = strtok_r(object, ".", &ptr);
- name;
- name = strtok_r(NULL, ".", &ptr), idx++)
- {
- if (idx == 0)
- {
- first = pstrdup(name);
- }
- else if (idx == 1)
- {
- second = pstrdup(name);
- }
- else
- {
- third = pstrdup(name);
- }
- }
-
- if (first != NULL)
- {
- json_object *jfirst = json_object_new_string(first);
- json_object_object_add(jresource, "database", jfirst);
- }
- if (second != NULL)
- {
- json_object *jsecond = json_object_new_string(second);
- json_object_object_add(jresource,
- (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
- }
- if (third != NULL)
- {
- json_object *jthird = json_object_new_string(third);
- json_object_object_add(jresource,
- (kind == ACL_KIND_CLASS) ? "table" :
- (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
- }
-
- if (first != NULL)
- pfree(first);
- if (second != NULL)
- pfree(second);
- if (third != NULL)
- pfree(third);
- break;
- }
- case ACL_KIND_OPER:
- case ACL_KIND_CONVERSION:
- case ACL_KIND_DATABASE:
- case ACL_KIND_TABLESPACE:
- case ACL_KIND_TYPE:
- case ACL_KIND_FILESYSTEM:
- case ACL_KIND_FDW:
- case ACL_KIND_FOREIGN_SERVER:
- case ACL_KIND_EXTPROTOCOL:
- {
- json_object *jobject = json_object_new_string(object);
- json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
- break;
- }
- default:
- elog(ERROR, "unrecognized objkind: %d", (int) kind);
- }
-
- json_object *jactions = json_object_new_array();
- foreach(cell, actions)
- {
- json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
- json_object_array_add(jactions, jaction);
- }
- json_object_object_add(jelement, "resource", jresource);
- json_object_object_add(jelement, "privileges", jactions);
- json_object_array_add(jaccess, jelement);
-
- json_object_object_add(jrequest, "user", juser);
- json_object_object_add(jrequest, "access", jaccess);
- json_object *jreqid = json_object_new_string("1");
- json_object_object_add(jrequest, "requestId", jreqid);
- json_object *jclientip = json_object_new_string("123.0.0.21");
- json_object_object_add(jrequest, "clientIp", jclientip);
- json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
- json_object_object_add(jrequest, "context", jcontext);
-
-
- return jrequest;
+ Assert(user != NULL && object != NULL && privilege != NULL
+ && isAll);
+ ListCell *cell;
+
+ elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
+ user, AclObjectKindStr[kind], object);
+ json_object *jrequest = json_object_new_object();
+ json_object *juser = json_object_new_string(user);
+
+ json_object *jaccess = json_object_new_array();
+ json_object *jelement = json_object_new_object();
+
+ json_object *jresource = json_object_new_object();
+ switch(kind)
+ {
+ case ACL_KIND_CLASS:
+ case ACL_KIND_SEQUENCE:
+ case ACL_KIND_PROC:
+ case ACL_KIND_NAMESPACE:
+ case ACL_KIND_LANGUAGE:
+ {
+ char *ptr = NULL; char *name = NULL;
+ char *first = NULL; // could be a database or protocol or tablespace
+ char *second = NULL; // could be a schema or language
+ char *third = NULL; // could be a table or sequence or function
+ int idx = 0;
+ for (name = strtok_r(object, ".", &ptr);
+ name;
+ name = strtok_r(NULL, ".", &ptr), idx++)
+ {
+ if (idx == 0)
+ {
+ first = pstrdup(name);
+ }
+ else if (idx == 1)
+ {
+ second = pstrdup(name);
+ }
+ else
+ {
+ third = pstrdup(name);
+ }
+ }
+
+ if (first != NULL)
+ {
+ json_object *jfirst = json_object_new_string(first);
+ json_object_object_add(jresource, "database", jfirst);
+ }
+ if (second != NULL)
+ {
+ json_object *jsecond = json_object_new_string(second);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
+ }
+ if (third != NULL)
+ {
+ json_object *jthird = json_object_new_string(third);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_CLASS) ? "table" :
+ (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
+ }
+
+ if (first != NULL)
+ pfree(first);
+ if (second != NULL)
+ pfree(second);
+ if (third != NULL)
+ pfree(third);
+ break;
+ }
+ case ACL_KIND_OPER:
+ case ACL_KIND_CONVERSION:
+ case ACL_KIND_DATABASE:
+ case ACL_KIND_TABLESPACE:
+ case ACL_KIND_TYPE:
+ case ACL_KIND_FILESYSTEM:
+ case ACL_KIND_FDW:
+ case ACL_KIND_FOREIGN_SERVER:
+ case ACL_KIND_EXTPROTOCOL:
+ {
+ json_object *jobject = json_object_new_string(object);
+ json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
+ break;
+ }
+ default:
+ elog(ERROR, "unrecognized objkind: %d", (int) kind);
+ }
+
+ json_object *jactions = json_object_new_array();
+ foreach(cell, actions)
+ {
+ json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
+ json_object_array_add(jactions, jaction);
+ }
+ json_object_object_add(jelement, "resource", jresource);
+ json_object_object_add(jelement, "privileges", jactions);
+ json_object_array_add(jaccess, jelement);
+
+ json_object_object_add(jrequest, "user", juser);
+ json_object_object_add(jrequest, "access", jaccess);
+ json_object *jreqid = json_object_new_string("1");
+ json_object_object_add(jrequest, "requestId", jreqid);
+ json_object *jclientip = json_object_new_string("123.0.0.21");
+ json_object_object_add(jrequest, "clientIp", jclientip);
+ json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
+ json_object_object_add(jrequest, "context", jcontext);
+
+
+ return jrequest;
}
static size_t write_callback(char *contents, size_t size, size_t nitems,
- void *userp)
+ void *userp)
{
- size_t realsize = size * nitems;
- CURL_HANDLE curl = (curl_context_t *) userp;
-
- curl->response.buffer = palloc0(realsize + 1);
- memset(curl->response.buffer, 0, realsize + 1);
- if (curl->response.buffer == NULL)
- {
- /* out of memory! */
- elog(WARNING, "not enough memory for Ranger response");
- return 0;
- }
-
- memcpy(curl->response.buffer, contents, realsize);
- curl->response.size = realsize + 1;
- elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
-
- return realsize;
+ size_t realsize = size * nitems;
+ CURL_HANDLE curl = (CURL_HANDLE) userp;
+ Assert(curl != NULL);
+
+ if (curl->response.buffer == NULL)
+ {
+ curl->response.buffer = palloc0(realsize + 1);
+ }
+ else
+ {
+ //Note:
+ //our repalloc is not same as realloc, repalloc's first param(buffer) can not be NULL
+ curl->response.buffer = repalloc(curl->response.buffer, curl->response.size + realsize + 1);
+ }
+
+ if (curl->response.buffer == NULL)
+ {
+ /* out of memory! */
+ elog(WARNING, "not enough memory for Ranger response");
+ return 0;
+ }
+
+ memcpy(curl->response.buffer + curl->response.size, contents, realsize);
+ curl->response.size += realsize;
+ curl->response.buffer[curl->response.size] = '\0';
+ elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
+
+ return realsize;
}
-void call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/**
+ * @returns: 0 curl success; -1 curl failed
+ */
+int call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
{
- CURLcode res;
- Assert(request != NULL);
-
- curl_global_init(CURL_GLOBAL_ALL);
-
- /* init the curl session */
- curl_handle->curl_handle = curl_easy_init();
- if (curl_handle->curl_handle == NULL)
- {
- goto _exit;
- }
-
- /* timeout */
- // curl_easy_setopt(curl_handle, CURLOPT_TIMEOUT, 1);
-
- /* specify URL to get */
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, "http://localhost:8089/checkprivilege");
- StringInfoData tname;
- initStringInfo(&tname);
- appendStringInfo(&tname, "http://");
- appendStringInfo(&tname, rps_addr_host);
- appendStringInfo(&tname, ":");
- appendStringInfo(&tname, "%d", rps_addr_port);
- appendStringInfo(&tname, "/rps");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, tname.data);
-
- /* specify format */
- // struct curl_slist *plist = curl_slist_append(NULL, "Content-Type:application/json;charset=UTF-8");
- // curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, plist);
-
-
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDSIZE_LARGE, 1000);
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPGET, 0);
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_CUSTOMREQUEST, "POST");
-
- struct curl_slist *headers = NULL;
- //curl_slist_append(headers, "Accept: application/json");
- headers = curl_slist_append(headers, "Content-Type:application/json");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPHEADER, headers);
-
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POST, 1L);
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDS,request);
- //"{\"requestId\": 1,\"user\": \"hubert\",\"clientIp\":\"123.0.0.21\",\"context\": \"SELECT * FROM sales\",\"access\":[{\"resource\":{\"database\":\"a-database\",\"schema\":\"a-schema\",\"table\":\"sales\"},\"privileges\": [\"select\"]}]}");
- /* send all data to this function */
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEFUNCTION, write_callback);
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEDATA, (void *)curl_handle);
-
- res = curl_easy_perform(curl_handle->curl_handle);
-
- /* check for errors */
- if(res != CURLE_OK)
- {
- elog(WARNING, "curl_easy_perform() failed: %s\n",
- curl_easy_strerror(res));
- }
- else
- {
- elog(LOG, "%d bytes retrieved from Ranger Restful API.",
- curl_handle->response.size);
- }
+ int ret = -1;
+ CURLcode res;
+ Assert(request != NULL);
+
+ curl_global_init(CURL_GLOBAL_ALL);
+
+ /* init the curl session */
+ curl_handle->curl_handle = curl_easy_init();
+ if (curl_handle->curl_handle == NULL)
+ {
+ goto _exit;
+ }
+
+ // hard-coded timeout
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_TIMEOUT, 30L);
+
+ /* specify URL to get */
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, "http://localhost:8089/checkprivilege");
+ StringInfoData tname;
+ initStringInfo(&tname);
+ appendStringInfo(&tname, "http://");
+ appendStringInfo(&tname, "%s", rps_addr_host);
+ appendStringInfo(&tname, ":");
+ appendStringInfo(&tname, "%d", rps_addr_port);
+ appendStringInfo(&tname, "/rps");
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, tname.data);
+
+ /* specify format */
+ // struct curl_slist *plist = curl_slist_append(NULL, "Content-Type:application/json;charset=UTF-8");
+ // curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, plist);
+
+
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDSIZE_LARGE, 1000);
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPGET, 0);
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_CUSTOMREQUEST, "POST");
+
+ struct curl_slist *headers = NULL;
+ //curl_slist_append(headers, "Accept: application/json");
+ headers = curl_slist_append(headers, "Content-Type:application/json");
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPHEADER, headers);
+
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POST, 1L);
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDS,request);
+ //"{\"requestId\": 1,\"user\": \"hubert\",\"clientIp\":\"123.0.0.21\",\"context\": \"SELECT * FROM sales\",\"access\":[{\"resource\":{\"database\":\"a-database\",\"schema\":\"a-schema\",\"table\":\"sales\"},\"privileges\": [\"select\"]}]}");
+ /* send all data to this function */
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEFUNCTION, write_callback);
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEDATA, (void *)curl_handle);
+
+ res = curl_easy_perform(curl_handle->curl_handle);
+
+ /* check for errors */
+ if(res != CURLE_OK)
+ {
+ elog(WARNING, "curl_easy_perform() failed: %s\n",
+ curl_easy_strerror(res));
+ }
+ else
+ {
+ ret = 0;
+ elog(LOG, "%d bytes retrieved from Ranger Restful API.",
+ curl_handle->response.size);
+ }
_exit:
- /* cleanup curl stuff */
- if (curl_handle->curl_handle)
- {
- curl_easy_cleanup(curl_handle->curl_handle);
- }
-
- /* we're done with libcurl, so clean it up */
- curl_global_cleanup();
+ /* cleanup curl stuff */
+ if (curl_handle->curl_handle)
+ {
+ curl_easy_cleanup(curl_handle->curl_handle);
+ }
+
+ /* we're done with libcurl, so clean it up */
+ curl_global_cleanup();
+ return ret;
}
/*
* arg_list: List of RangerRequestJsonArgs
*/
int check_privilege_from_ranger_batch(List *arg_list)
{
- json_object* jrequest = create_ranger_request_json_batch(arg_list);
- Assert(jrequest != NULL);
- const char *request = json_object_to_json_string(jrequest);
- elog(LOG, "Send JSON request to Ranger: %s", request);
- Assert(request != NULL);
- struct curl_context_t curl_context;
- memset(&curl_context, 0, sizeof(struct curl_context_t));
-
- /* call GET method to send request*/
- call_ranger_rest(&curl_context, request);
-
- /* free the JSON object */
- json_object_put(jrequest);
-
- /* parse the JSON-format result */
- RangerACLResult ret = parse_ranger_response(curl_context.response.buffer);
-
- /* free response buffer */
- if (curl_context.response.buffer != NULL)
- {
- pfree(curl_context.response.buffer);
- }
-
- return ret;
+ json_object* jrequest = create_ranger_request_json_batch(arg_list);
+ Assert(jrequest != NULL);
+ const char *request = json_object_to_json_string(jrequest);
+ elog(LOG, "Send JSON request to Ranger: %s", request);
+ Assert(request != NULL);
+ struct curl_context_t curl_context;
+ memset(&curl_context, 0, sizeof(struct curl_context_t));
+
+ /* call GET method to send request*/
+ if (call_ranger_rest(&curl_context, request) < 0)
+ {
+ return RANGERCHECK_UNKNOWN;
+ }
+
+ /* free the JSON object */
+ json_object_put(jrequest);
+
+ /* parse the JSON-format result */
+ RangerACLResult ret = parse_ranger_response(curl_context.response.buffer);
+ /* free response buffer */
+ if (curl_context.response.buffer != NULL)
+ {
+ pfree(curl_context.response.buffer);
+ }
+
+ return ret;
}
/*
* Check the privilege from Ranger for one role
*/
int check_privilege_from_ranger(char* user, AclObjectKind kind, char* object,
- List* actions, bool isAll)
+ List* actions, bool isAll)
{
- json_object* jrequest = create_ranger_request_json(user, kind, object,
- actions, isAll);
+ json_object* jrequest = create_ranger_request_json(user, kind, object,
+ actions, isAll);
- Assert(jrequest != NULL);
- const char* request = json_object_to_json_string(jrequest);
- elog(LOG, "send JSON request to Ranger: %s", request);
- Assert(request != NULL);
+ Assert(jrequest != NULL);
+ const char* request = json_object_to_json_string(jrequest);
+ elog(LOG, "send JSON request to Ranger: %s", request);
+ Assert(request != NULL);
- struct curl_context_t curl_context;
- memset(&curl_context, 0, sizeof(struct curl_context_t));
+ struct curl_context_t curl_context;
+ memset(&curl_context, 0, sizeof(struct curl_context_t));
- /* call GET method to send request*/
- call_ranger_rest(&curl_context, request);
+ /* call GET method to send request*/
+ if (call_ranger_rest(&curl_context, request) < 0)
+ {
+ return RANGERCHECK_UNKNOWN;
--- End diff --
tks! I will modify it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by zhangh43 <gi...@git.apache.org>.
Github user zhangh43 commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93183177
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -245,279 +257,326 @@ json_object *create_ranger_request_json_batch(List *args)
* }
*/
json_object* create_ranger_request_json(char* user, AclObjectKind kind, char* object,
- List* actions, bool isAll)
+ List* actions, bool isAll)
{
- Assert(user != NULL && object != NULL && privilege != NULL
- && isAll);
- ListCell *cell;
-
- elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
- user, AclObjectKindStr[kind], object);
- json_object *jrequest = json_object_new_object();
- json_object *juser = json_object_new_string(user);
-
- json_object *jaccess = json_object_new_array();
- json_object *jelement = json_object_new_object();
-
- json_object *jresource = json_object_new_object();
- switch(kind)
- {
- case ACL_KIND_CLASS:
- case ACL_KIND_SEQUENCE:
- case ACL_KIND_PROC:
- case ACL_KIND_NAMESPACE:
- case ACL_KIND_LANGUAGE:
- {
- char *ptr = NULL; char *name = NULL;
- char *first = NULL; // could be a database or protocol or tablespace
- char *second = NULL; // could be a schema or language
- char *third = NULL; // could be a table or sequence or function
- int idx = 0;
- for (name = strtok_r(object, ".", &ptr);
- name;
- name = strtok_r(NULL, ".", &ptr), idx++)
- {
- if (idx == 0)
- {
- first = pstrdup(name);
- }
- else if (idx == 1)
- {
- second = pstrdup(name);
- }
- else
- {
- third = pstrdup(name);
- }
- }
-
- if (first != NULL)
- {
- json_object *jfirst = json_object_new_string(first);
- json_object_object_add(jresource, "database", jfirst);
- }
- if (second != NULL)
- {
- json_object *jsecond = json_object_new_string(second);
- json_object_object_add(jresource,
- (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
- }
- if (third != NULL)
- {
- json_object *jthird = json_object_new_string(third);
- json_object_object_add(jresource,
- (kind == ACL_KIND_CLASS) ? "table" :
- (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
- }
-
- if (first != NULL)
- pfree(first);
- if (second != NULL)
- pfree(second);
- if (third != NULL)
- pfree(third);
- break;
- }
- case ACL_KIND_OPER:
- case ACL_KIND_CONVERSION:
- case ACL_KIND_DATABASE:
- case ACL_KIND_TABLESPACE:
- case ACL_KIND_TYPE:
- case ACL_KIND_FILESYSTEM:
- case ACL_KIND_FDW:
- case ACL_KIND_FOREIGN_SERVER:
- case ACL_KIND_EXTPROTOCOL:
- {
- json_object *jobject = json_object_new_string(object);
- json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
- break;
- }
- default:
- elog(ERROR, "unrecognized objkind: %d", (int) kind);
- }
-
- json_object *jactions = json_object_new_array();
- foreach(cell, actions)
- {
- json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
- json_object_array_add(jactions, jaction);
- }
- json_object_object_add(jelement, "resource", jresource);
- json_object_object_add(jelement, "privileges", jactions);
- json_object_array_add(jaccess, jelement);
-
- json_object_object_add(jrequest, "user", juser);
- json_object_object_add(jrequest, "access", jaccess);
- json_object *jreqid = json_object_new_string("1");
- json_object_object_add(jrequest, "requestId", jreqid);
- json_object *jclientip = json_object_new_string("123.0.0.21");
- json_object_object_add(jrequest, "clientIp", jclientip);
- json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
- json_object_object_add(jrequest, "context", jcontext);
-
-
- return jrequest;
+ Assert(user != NULL && object != NULL && privilege != NULL
+ && isAll);
+ ListCell *cell;
+
+ elog(LOG, "build json for ranger request, user:%s, kind:%s, object:%s",
+ user, AclObjectKindStr[kind], object);
+ json_object *jrequest = json_object_new_object();
+ json_object *juser = json_object_new_string(user);
+
+ json_object *jaccess = json_object_new_array();
+ json_object *jelement = json_object_new_object();
+
+ json_object *jresource = json_object_new_object();
+ switch(kind)
+ {
+ case ACL_KIND_CLASS:
+ case ACL_KIND_SEQUENCE:
+ case ACL_KIND_PROC:
+ case ACL_KIND_NAMESPACE:
+ case ACL_KIND_LANGUAGE:
+ {
+ char *ptr = NULL; char *name = NULL;
+ char *first = NULL; // could be a database or protocol or tablespace
+ char *second = NULL; // could be a schema or language
+ char *third = NULL; // could be a table or sequence or function
+ int idx = 0;
+ for (name = strtok_r(object, ".", &ptr);
+ name;
+ name = strtok_r(NULL, ".", &ptr), idx++)
+ {
+ if (idx == 0)
+ {
+ first = pstrdup(name);
+ }
+ else if (idx == 1)
+ {
+ second = pstrdup(name);
+ }
+ else
+ {
+ third = pstrdup(name);
+ }
+ }
+
+ if (first != NULL)
+ {
+ json_object *jfirst = json_object_new_string(first);
+ json_object_object_add(jresource, "database", jfirst);
+ }
+ if (second != NULL)
+ {
+ json_object *jsecond = json_object_new_string(second);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_LANGUAGE) ? "language" : "schema", jsecond);
+ }
+ if (third != NULL)
+ {
+ json_object *jthird = json_object_new_string(third);
+ json_object_object_add(jresource,
+ (kind == ACL_KIND_CLASS) ? "table" :
+ (kind == ACL_KIND_SEQUENCE) ? "sequence" : "function", jthird);
+ }
+
+ if (first != NULL)
+ pfree(first);
+ if (second != NULL)
+ pfree(second);
+ if (third != NULL)
+ pfree(third);
+ break;
+ }
+ case ACL_KIND_OPER:
+ case ACL_KIND_CONVERSION:
+ case ACL_KIND_DATABASE:
+ case ACL_KIND_TABLESPACE:
+ case ACL_KIND_TYPE:
+ case ACL_KIND_FILESYSTEM:
+ case ACL_KIND_FDW:
+ case ACL_KIND_FOREIGN_SERVER:
+ case ACL_KIND_EXTPROTOCOL:
+ {
+ json_object *jobject = json_object_new_string(object);
+ json_object_object_add(jresource, AclObjectKindStr[kind], jobject);
+ break;
+ }
+ default:
+ elog(ERROR, "unrecognized objkind: %d", (int) kind);
+ }
+
+ json_object *jactions = json_object_new_array();
+ foreach(cell, actions)
+ {
+ json_object* jaction = json_object_new_string((char *)cell->data.ptr_value);
+ json_object_array_add(jactions, jaction);
+ }
+ json_object_object_add(jelement, "resource", jresource);
+ json_object_object_add(jelement, "privileges", jactions);
+ json_object_array_add(jaccess, jelement);
+
+ json_object_object_add(jrequest, "user", juser);
+ json_object_object_add(jrequest, "access", jaccess);
+ json_object *jreqid = json_object_new_string("1");
+ json_object_object_add(jrequest, "requestId", jreqid);
+ json_object *jclientip = json_object_new_string("123.0.0.21");
+ json_object_object_add(jrequest, "clientIp", jclientip);
+ json_object *jcontext = json_object_new_string("SELECT * FROM DDDDDDD");
+ json_object_object_add(jrequest, "context", jcontext);
+
+
+ return jrequest;
}
static size_t write_callback(char *contents, size_t size, size_t nitems,
- void *userp)
+ void *userp)
{
- size_t realsize = size * nitems;
- CURL_HANDLE curl = (curl_context_t *) userp;
-
- curl->response.buffer = palloc0(realsize + 1);
- memset(curl->response.buffer, 0, realsize + 1);
- if (curl->response.buffer == NULL)
- {
- /* out of memory! */
- elog(WARNING, "not enough memory for Ranger response");
- return 0;
- }
-
- memcpy(curl->response.buffer, contents, realsize);
- curl->response.size = realsize + 1;
- elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
-
- return realsize;
+ size_t realsize = size * nitems;
+ CURL_HANDLE curl = (CURL_HANDLE) userp;
+ Assert(curl != NULL);
+
+ if (curl->response.buffer == NULL)
+ {
+ curl->response.buffer = palloc0(realsize + 1);
+ }
+ else
+ {
+ //Note:
+ //our repalloc is not same as realloc, repalloc's first param(buffer) can not be NULL
+ curl->response.buffer = repalloc(curl->response.buffer, curl->response.size + realsize + 1);
+ }
+
+ if (curl->response.buffer == NULL)
+ {
+ /* out of memory! */
+ elog(WARNING, "not enough memory for Ranger response");
+ return 0;
+ }
+
+ memcpy(curl->response.buffer + curl->response.size, contents, realsize);
+ curl->response.size += realsize;
+ curl->response.buffer[curl->response.size] = '\0';
+ elog(LOG, "read from Ranger Restful API: %s", curl->response.buffer);
+
+ return realsize;
}
-void call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/**
+ * @returns: 0 curl success; -1 curl failed
+ */
+int call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
{
- CURLcode res;
- Assert(request != NULL);
-
- curl_global_init(CURL_GLOBAL_ALL);
-
- /* init the curl session */
- curl_handle->curl_handle = curl_easy_init();
- if (curl_handle->curl_handle == NULL)
- {
- goto _exit;
- }
-
- /* timeout */
- // curl_easy_setopt(curl_handle, CURLOPT_TIMEOUT, 1);
-
- /* specify URL to get */
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, "http://localhost:8089/checkprivilege");
- StringInfoData tname;
- initStringInfo(&tname);
- appendStringInfo(&tname, "http://");
- appendStringInfo(&tname, rps_addr_host);
- appendStringInfo(&tname, ":");
- appendStringInfo(&tname, "%d", rps_addr_port);
- appendStringInfo(&tname, "/rps");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, tname.data);
-
- /* specify format */
- // struct curl_slist *plist = curl_slist_append(NULL, "Content-Type:application/json;charset=UTF-8");
- // curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, plist);
-
-
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDSIZE_LARGE, 1000);
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPGET, 0);
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_CUSTOMREQUEST, "POST");
-
- struct curl_slist *headers = NULL;
- //curl_slist_append(headers, "Accept: application/json");
- headers = curl_slist_append(headers, "Content-Type:application/json");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPHEADER, headers);
-
- //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POST, 1L);
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDS,request);
- //"{\"requestId\": 1,\"user\": \"hubert\",\"clientIp\":\"123.0.0.21\",\"context\": \"SELECT * FROM sales\",\"access\":[{\"resource\":{\"database\":\"a-database\",\"schema\":\"a-schema\",\"table\":\"sales\"},\"privileges\": [\"select\"]}]}");
- /* send all data to this function */
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEFUNCTION, write_callback);
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEDATA, (void *)curl_handle);
-
- res = curl_easy_perform(curl_handle->curl_handle);
-
- /* check for errors */
- if(res != CURLE_OK)
- {
- elog(WARNING, "curl_easy_perform() failed: %s\n",
- curl_easy_strerror(res));
- }
- else
- {
- elog(LOG, "%d bytes retrieved from Ranger Restful API.",
- curl_handle->response.size);
- }
+ int ret = -1;
+ CURLcode res;
+ Assert(request != NULL);
+
+ curl_global_init(CURL_GLOBAL_ALL);
+
+ /* init the curl session */
+ curl_handle->curl_handle = curl_easy_init();
+ if (curl_handle->curl_handle == NULL)
+ {
+ goto _exit;
+ }
+
+ // hard-coded timeout
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_TIMEOUT, 30L);
+
+ /* specify URL to get */
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, "http://localhost:8089/checkprivilege");
+ StringInfoData tname;
+ initStringInfo(&tname);
+ appendStringInfo(&tname, "http://");
+ appendStringInfo(&tname, "%s", rps_addr_host);
+ appendStringInfo(&tname, ":");
+ appendStringInfo(&tname, "%d", rps_addr_port);
+ appendStringInfo(&tname, "/rps");
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, tname.data);
+
+ /* specify format */
+ // struct curl_slist *plist = curl_slist_append(NULL, "Content-Type:application/json;charset=UTF-8");
+ // curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, plist);
+
+
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDSIZE_LARGE, 1000);
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPGET, 0);
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_CUSTOMREQUEST, "POST");
+
+ struct curl_slist *headers = NULL;
+ //curl_slist_append(headers, "Accept: application/json");
+ headers = curl_slist_append(headers, "Content-Type:application/json");
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPHEADER, headers);
+
+ //curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POST, 1L);
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDS,request);
+ //"{\"requestId\": 1,\"user\": \"hubert\",\"clientIp\":\"123.0.0.21\",\"context\": \"SELECT * FROM sales\",\"access\":[{\"resource\":{\"database\":\"a-database\",\"schema\":\"a-schema\",\"table\":\"sales\"},\"privileges\": [\"select\"]}]}");
+ /* send all data to this function */
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEFUNCTION, write_callback);
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEDATA, (void *)curl_handle);
+
+ res = curl_easy_perform(curl_handle->curl_handle);
+
+ /* check for errors */
+ if(res != CURLE_OK)
+ {
+ elog(WARNING, "curl_easy_perform() failed: %s\n",
+ curl_easy_strerror(res));
+ }
+ else
+ {
+ ret = 0;
+ elog(LOG, "%d bytes retrieved from Ranger Restful API.",
+ curl_handle->response.size);
+ }
_exit:
- /* cleanup curl stuff */
- if (curl_handle->curl_handle)
- {
- curl_easy_cleanup(curl_handle->curl_handle);
- }
-
- /* we're done with libcurl, so clean it up */
- curl_global_cleanup();
+ /* cleanup curl stuff */
+ if (curl_handle->curl_handle)
+ {
+ curl_easy_cleanup(curl_handle->curl_handle);
+ }
+
+ /* we're done with libcurl, so clean it up */
+ curl_global_cleanup();
+ return ret;
}
/*
* arg_list: List of RangerRequestJsonArgs
*/
int check_privilege_from_ranger_batch(List *arg_list)
{
- json_object* jrequest = create_ranger_request_json_batch(arg_list);
- Assert(jrequest != NULL);
- const char *request = json_object_to_json_string(jrequest);
- elog(LOG, "Send JSON request to Ranger: %s", request);
- Assert(request != NULL);
- struct curl_context_t curl_context;
- memset(&curl_context, 0, sizeof(struct curl_context_t));
-
- /* call GET method to send request*/
- call_ranger_rest(&curl_context, request);
-
- /* free the JSON object */
- json_object_put(jrequest);
-
- /* parse the JSON-format result */
- RangerACLResult ret = parse_ranger_response(curl_context.response.buffer);
-
- /* free response buffer */
- if (curl_context.response.buffer != NULL)
- {
- pfree(curl_context.response.buffer);
- }
-
- return ret;
+ json_object* jrequest = create_ranger_request_json_batch(arg_list);
+ Assert(jrequest != NULL);
+ const char *request = json_object_to_json_string(jrequest);
+ elog(LOG, "Send JSON request to Ranger: %s", request);
+ Assert(request != NULL);
+ struct curl_context_t curl_context;
+ memset(&curl_context, 0, sizeof(struct curl_context_t));
+
+ /* call GET method to send request*/
+ if (call_ranger_rest(&curl_context, request) < 0)
+ {
+ return RANGERCHECK_UNKNOWN;
+ }
+
+ /* free the JSON object */
+ json_object_put(jrequest);
+
+ /* parse the JSON-format result */
+ RangerACLResult ret = parse_ranger_response(curl_context.response.buffer);
+ /* free response buffer */
+ if (curl_context.response.buffer != NULL)
+ {
+ pfree(curl_context.response.buffer);
+ }
+
+ return ret;
}
/*
* Check the privilege from Ranger for one role
*/
int check_privilege_from_ranger(char* user, AclObjectKind kind, char* object,
- List* actions, bool isAll)
+ List* actions, bool isAll)
{
- json_object* jrequest = create_ranger_request_json(user, kind, object,
- actions, isAll);
+ json_object* jrequest = create_ranger_request_json(user, kind, object,
+ actions, isAll);
- Assert(jrequest != NULL);
- const char* request = json_object_to_json_string(jrequest);
- elog(LOG, "send JSON request to Ranger: %s", request);
- Assert(request != NULL);
+ Assert(jrequest != NULL);
+ const char* request = json_object_to_json_string(jrequest);
+ elog(LOG, "send JSON request to Ranger: %s", request);
+ Assert(request != NULL);
- struct curl_context_t curl_context;
- memset(&curl_context, 0, sizeof(struct curl_context_t));
+ struct curl_context_t curl_context;
+ memset(&curl_context, 0, sizeof(struct curl_context_t));
- /* call GET method to send request*/
- call_ranger_rest(&curl_context, request);
+ /* call GET method to send request*/
+ if (call_ranger_rest(&curl_context, request) < 0)
+ {
+ return RANGERCHECK_UNKNOWN;
--- End diff --
Here return RANGERCHECK_UNKNOWN has some problem.
check_privilege_from_ranger is call by pg_rangercheck which return AclResult. But RANGERCHECK_UNKNOWN is belong to RangerACLResult. We need to unify them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #1058: HAWQ-1226. HAWQ core dump due to enable ranger w...
Posted by interma <gi...@git.apache.org>.
Github user interma commented on the issue:
https://github.com/apache/incubator-hawq/pull/1058
Fixed review problems already.
Tks, guys!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by linwen <gi...@git.apache.org>.
Github user linwen commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93190511
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -452,8 +448,8 @@ static int call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
goto _exit;
}
- // hard core timeout
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_TIMEOUT, 15L);
+ // hard-coded timeout
--- End diff --
please use /**/ as comment.
/* hard-coded timeout is 30 seconds */
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1058: HAWQ-1226. HAWQ core dump due to enable r...
Posted by interma <gi...@git.apache.org>.
Github user interma commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1058#discussion_r93184810
--- Diff: src/test/regress/checkinc.py ---
@@ -78,9 +78,10 @@
'winsock.h': [],
'winsock2.h': [],
'ws2tcpip.h': [],
- 'hdfs/hdfs.h': [],
+ 'hdfs/hdfs.h': [],
'quicklz1.h': [],
'quicklz3.h': [],
+ 'json-c/json.h': [],
--- End diff --
Yes, only in "make install".
"make" is ok.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---