You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Anonymous Coward (Code Review)" <ge...@cloudera.org> on 2024/09/18 10:05:41 UTC
[kudu-CR] [WIP] KUDU-3608 REST API
lotzgabriella@gmail.com has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21823
Change subject: [WIP] KUDU-3608 REST API
......................................................................
[WIP] KUDU-3608 REST API
After careful consideration we opted for working on the embedded webserver.
First push just to start the patch set.
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
2 files changed, 27 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/1
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 1
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 18: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/749/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 18
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 04 Dec 2024 19:22:30 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#16).
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
[WIP] KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: No Security Measures Yet. This implementation does not
currently include authentication or authorization.
Deleting, altering, or retrieving a table object requires the table's
ID. Operations using table names are not yet supported.
Testing: A new test file called rest_catalog-test.cc has been created to
validate these endpoints using an internal mini-cluster setup.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
A src/kudu/master/rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
6 files changed, 835 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/16
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 16
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Zoltan Martonka (Code Review)" <ge...@cloudera.org>.
Zoltan Martonka has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 33: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 33
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Mar 2025 13:06:51 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Gabriella Lotz has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 32:
(11 comments)
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog-test.cc
File src/kudu/master/rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog-test.cc@198
PS30, Line 198:
> do we need this line?
Done
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog-test.cc@211
PS30, Line 211: EasyCurl c;
> do we need this line?
Done
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog-test.cc
File src/kudu/master/rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog-test.cc@173
PS31, Line 173: int first_column_id = FindColumnId(buf.ToString());
: ASSERT_NE(first_column_id, -1) << "Column ID not found in the schema";
> nit: can you please add a small comment here?
Done
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog-test.cc@256
PS31, Line 256: faststring buf;
> nit: maybe assert the error message? Curious if we return reasonable error
I checked and it is empty.
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_path_handlers.cc@66
PS31, Line 66: CheckIsInitializedAndIsLeader(Json
> nit rename: CheckIsInitializedAndIsLeader
Done
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_path_handlers.cc@86
PS31, Line 86: RETURN_JSON_ERROR(jw,
> nit: maybe add more error message detail?
Done
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_path_handlers.cc@159
PS31, Line 159: atus_code = HttpStatusCode::Ok;
> does it make sense to only create one JsonWriter instance in the beginning
Done
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_path_handlers.cc@229
PS31, Line 229:
> does it make sense to only create one JsonWriter instance in the beginning
Done
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_test_base.h
File src/kudu/master/rest_catalog_test_base.h:
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_test_base.h@79
PS31, Line 79: DCHECK(table_id);
: kudu::client::sp::shared_ptr<kudu::client::KuduTable> table;
: RETURN_NOT_OK(client_->OpenTable(table_name, &table));
: *table_id = table->id();
: return Status::OK();
: }
:
: kudu::client::sp::shared_ptr<kudu::client::KuduClient> client_;
: s
> please refactor this to return a status, and provide the table id as an out
Done
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_test_base.h
File src/kudu/master/rest_catalog_test_base.h:
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_test_base.h@51
PS30, Line 51: kudu::client::KuduSchema schema;
> make this a class member field, such that else in the tests you dont have t
Done
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/spnego_rest_catalog-test.cc
File src/kudu/master/spnego_rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/spnego_rest_catalog-test.cc@205
PS31, Line 205: ASSERT_OK(client_->OpenTable(kTableName, &table));
: ASSERT_EQ(table->schema().num_columns(), 3);
: ASSERT_STR_CONTAINS(table->owner(), "alice");
: }
:
: TEST_F(SpnegoRestCatalogTest, TestInvalidHeaders) {
: ASSERT_OK(CreateTestTable());
: string table_id;
: ASSERT_OK(GetTableId(kTableName, &table_id));
: EasyCurl c;
: faststring buf;
: Status s = c.FetchURL(
: Substitute("http://$0/api/v1/tables", cluster_->mini_master()->bound_http_addr().ToString()),
: &buf,
: {"Authorization: I am an invalid header"});
: ASSERT_STR_CONTAINS(s.ToString(), "HTTP 500");
:
: s = c.FetchURL(
: Substitute("http://$0/api/v1/tables", cluster_->mini_master()->bound_http_addr().ToString()),
: &buf,
: {"Authorization: Negotiate I am also an invalid header"});
:
> IMO is enough to use one endpoint and have one HTTP 500 and one HTTP 401 te
Done
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 32
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Mar 2025 21:03:45 +0000
Gerrit-HasComments: Yes
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#2).
Change subject: [WIP] KUDU-3608 REST API
......................................................................
[WIP] KUDU-3608 REST API
After careful consideration we opted for implementing a REST API into the Kudu master web server.
The API only handles metadata and will be reachable under the path /api.
It will be implemented using PathHandlers in the master server.
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
2 files changed, 100 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/2
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 7:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/590/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 7
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 14 Oct 2024 13:29:55 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Gabriella Lotz has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 26:
(9 comments)
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/master.cc
File src/kudu/master/master.cc:
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/master.cc@137
PS25, Line 137: The flag webserver_enabled must be "
: "set to true for this flag to take effect.
> Add a group flag validator to enforce this. For reference, check out how t
Done
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/master.cc@259
PS25, Line 259: >set_member_type(resp.membe
> Do we need this if FLAGS_enable_rest_api = false?
Done
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/master.cc@305
PS25, Line 305:
> Since this code is run just once during initialization, there isn't much se
Done
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/rest_catalog-test.cc
File src/kudu/master/rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/rest_catalog-test.cc@82
PS25, Line 82:
: static string ConstructTableSchema(int column_id, bool is_there_new_column = false) {
: string columns = Substitute(
: "{\"id\":$0,\"name\":\"key\",\"type\":\"INT32\",\"is_key\":"
: "true,\"is_nullable\":false,\"encoding\":\"AUTO_ENCODING\",\"compression\":"
: "\"DEFAULT_COMPRESSION\",\"cfile_block_size\":0,\"immutable\":false},{\"id\":"
: "$1,\"name\":\"int_val\",\"type\":\"INT32\",\"is_key\":false,\"is_nullable\":"
: "false,\"encoding\":\"AUTO_ENCODING\",\"compression\":\"DEFAULT_COMPRESSION\","
: "\"cfile_block_size\":0,\"immutable\":false}",
: column_id,
: column_id + 1);
: if (is_there_new_column) {
: string column_id_2_str = std::to_string(column_id + 2);
: string new_column = Substitute(
: ",{\"id\":$0,\"name\":\"new_column\",\"type\":\"STRING\",\"is_key\":false,"
: "\"is_nullable\":true,\"encoding\":\"AUTO_ENCODING\",\"compression\":"
: "\"DEFAULT_COMPRESSION\",\"cfile_block_size\":0,\"immutable\":false}",
: column_id_2_str);
: columns += new_column;
: }
: return Substitute("{\"columns\":[$0]}", columns);
: }
:
: protected:
: unique_ptr<InternalMiniCluster> cluster_;
: const std::string kTablePartitionSchema = "{\"range_schema\":{\"columns\":[{\"id\":10}]}}";
: const std::string kTablePartitionSchemaColumnIdZero =
: "{\"range_schema\":{\"columns\":[{\"id\":0}]}}";
: };
:
:
: TEST_F(RestCatalogTest, TestInvalidMethod) {
: EasyCurl c;
: c.set_custom_method("DELETE");
: faststring buf;
: Status s = c.FetchURL(
: Substitute("http://$0/api/v1/tables", cluster_->mini_master()->bound_http_addr().ToString()),
: &buf);
: A
> This looks like copy-and-paste from spnego_rest_catalog-test.cc. Maybe, in
Done
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/rest_catalog-test.cc@132
PS25, Line 132: table_id),
: &buf);
> There is no need to convert integers to strings if using Substitute.
Done
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/spnego_rest_catalog-test.cc
File src/kudu/master/spnego_rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/spnego_rest_catalog-test.cc@65
PS25, Line 65: ASSERT_OK(StartCluster());
: ASSERT_OK(cluster_->kdc()->Kinit("test-admin"));
: ASSE
> Why to introduce this local constant if it's used only once at line 70?
Done
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/spnego_rest_catalog-test.cc@84
PS25, Line 84:
> Why KUDU_CHECK_OK? There is RETURN_NOT_OK() to handle this in a method ret
Done
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/spnego_rest_catalog-test.cc@89
PS25, Line 89:
> Wrap this into std::unique_ptr?
Done
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/spnego_rest_catalog-test.cc@112
PS25, Line 112:
:
:
> Why to add this extra check if the table was opened successfully?
Done
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 26
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 03 Mar 2025 12:57:54 +0000
Gerrit-HasComments: Yes
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 26:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/1129/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 26
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 03 Mar 2025 12:54:49 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#3).
Change subject: [WIP] KUDU-3608 REST API
......................................................................
[WIP] KUDU-3608 REST API
After careful consideration we opted for implementing a REST API into the Kudu master web server.
The API only handles metadata and will be reachable under the path /api.
It will be implemented using PathHandlers in the master server.
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
2 files changed, 89 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/3
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 5:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/581/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 5
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 08 Oct 2024 16:44:13 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 31:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/1196/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 31
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Mar 2025 14:20:34 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 33: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 33
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Mar 2025 12:53:26 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Zoltan Chovan (Code Review)" <ge...@cloudera.org>.
Zoltan Chovan has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 18:
Please introduce a feature flag to toggle the availability of the REST API. At least until the security aspect is incomplete, this feature should probably behind an unsafe flag as well.
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 18
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 09 Dec 2024 08:39:07 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Gabriella Lotz has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 17:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc
File src/kudu/master/master_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@337
PS17, Line 337: *status_code = HttpStatusCode::NoContent;
> why NoContent status instead of OK? the delete was successful
I chose to use the 204 No Content status code for successful DELETE requests where no additional information needs to be provided to the client. This aligns with the HTTP specification, which allows DELETE methods to return 204, 200, or 202. https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/DELETE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 17
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 04 Dec 2024 14:12:27 +0000
Gerrit-HasComments: Yes
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#19).
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
[WIP] KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: No Security Measures Yet. This implementation does not
currently include authentication or authorization.
Deleting, altering, or retrieving a table object requires the table's
ID. Operations using table names are not yet supported.
Testing: A new test file called rest_catalog-test.cc has been created to
validate these endpoints using an internal mini-cluster setup.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
A src/kudu/master/rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
6 files changed, 815 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/19
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 19
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 33:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/1206/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 33
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 26 Mar 2025 15:19:19 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#33).
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: Deleting, altering, or retrieving a table object requires
the table's ID. Operations using table names are not yet supported.
Testing: Two new test files called rest_catalog-test.cc and
spnego_rest_catalog-test.cc have been created to validate these
endpoints using an internal mini-cluster setup. Both tests are based on
the rest_catalog_test_base.h header file.
These test files have been tested for flakiness and evaluated under high
load to ensure stability.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Follow-up patches:
- Support for multi-master setup
- Adding ranger itests
- Swagger/OpenAI documentation for the endpoint under /api/docs path
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master.cc
M src/kudu/master/master.h
A src/kudu/master/rest_catalog-test.cc
A src/kudu/master/rest_catalog_path_handlers.cc
A src/kudu/master/rest_catalog_path_handlers.h
A src/kudu/master/rest_catalog_test_base.h
A src/kudu/master/spnego_rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
10 files changed, 1,395 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/33
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 33
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 15:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/711/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 15
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Nov 2024 18:19:31 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#9).
Change subject: [WIP] KUDU-3608 REST API
......................................................................
[WIP] KUDU-3608 REST API
After careful consideration we opted for implementing a REST API into the Kudu master web server.
The API only handles metadata and will be reachable under the path /api.
It will be implemented using PathHandlers in the master server.
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/master-test.cc
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
3 files changed, 166 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/9
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 9
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 10: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/667/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 10
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 06 Nov 2024 14:50:09 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#7).
Change subject: [WIP] KUDU-3608 REST API
......................................................................
[WIP] KUDU-3608 REST API
After careful consideration we opted for implementing a REST API into the Kudu master web server.
The API only handles metadata and will be reachable under the path /api.
It will be implemented using PathHandlers in the master server.
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/master-test.cc
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
3 files changed, 167 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/7
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 7
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 19: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/796/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 19
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Dec 2024 10:30:57 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#21).
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
[WIP] KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: No Security Measures Yet. This implementation does not
currently include authentication or authorization.
Deleting, altering, or retrieving a table object requires the table's
ID. Operations using table names are not yet supported.
Testing: A new test file called rest_catalog-test.cc has been created to
validate these endpoints using an internal mini-cluster setup.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master.cc
M src/kudu/master/master.h
A src/kudu/master/rest_catalog-test.cc
A src/kudu/master/rest_catalog_path_handlers.cc
A src/kudu/master/rest_catalog_path_handlers.h
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
8 files changed, 942 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/21
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 21
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 20:
> Uploaded patch set 19.
Please re-base this patch in your local git repo on top of the HEAD in the master branch before submitting next revision to gerrit. That's to avoid Jenkins workspace pollution after the libthrift library was upgraded with f2a6be7a44e3c395c240f982ce7b2a193410cb9b
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 20
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Dec 2024 22:36:08 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 13: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/696/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 13
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 20 Nov 2024 17:49:36 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#30).
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: Deleting, altering, or retrieving a table object requires
the table's ID. Operations using table names are not yet supported.
Testing: Two new test files called rest_catalog-test.cc and
spnego_rest_catalog-test.cc have been created to validate these
endpoints using an internal mini-cluster setup. Both tests are based on
the rest_catalog_test_base.h header file.
These test files have been tested for flakiness and evaluated under high
load to ensure stability.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Follow-up patches:
- Support for multi-master setup
- Adding ranger itests
- Swagger/OpenAI documentation for the endpoint under /api/docs path
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master.cc
M src/kudu/master/master.h
A src/kudu/master/rest_catalog-test.cc
A src/kudu/master/rest_catalog_path_handlers.cc
A src/kudu/master/rest_catalog_path_handlers.h
A src/kudu/master/rest_catalog_test_base.h
A src/kudu/master/spnego_rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
10 files changed, 1,342 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/30
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 30
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 31:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_path_handlers.cc@66
PS31, Line 66: CheckLeaderAndCatalogManagerStatus
nit rename: CheckIsInitializedAndIsLeader
^ to reflect the order of operations in the function body.
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 31
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Mar 2025 18:06:37 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 25:
(24 comments)
Since Attila explicitly asked me not to jump into code reviews of WIP gerrit items, I wasn't following this changelist for some time. However, it seems I missed when WIP tag was removed :)
There were also a few question I needed to respond, so here you go.
This patch has already been committed into the git repo, but maybe consider addressing this feedback in a separate changelist follow-up patch, if it makes sense to you.
Thank you for working on this!
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog-test.cc
File src/kudu/master/rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog-test.cc@537
PS34, Line 537: "name": "int_val"
:
nit for here and elsewhere where the status isn't used elsewhere: it might shortened to
ASSERT_OK(client_->OpenTable(kTableName, &table));
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.h
File src/kudu/master/rest_catalog_path_handlers.h:
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.h@34
PS34, Line 34: // Web page support for the master.
nit: consider making this class 'final'
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.h@36
PS34, Line 36:
nit: consider adding DCHECK_NOT_NULL wrapper around 'master'
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.h@38
PS34, Line 38:
nit: if the destructor is empty, consider using
~RestCatalogPathHandlers() = default;
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@24
PS34, Line 24: #include "google/protobuf/util/json_util.h"
nit: change quotes to '<' and '>' and re-arrange the lines around to be sorted alphabetically
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@71
PS34, Line 71: ()->GetTableIn
Why is this passed by pointer if 'jw' isn't changing in this method?
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@73
PS34, Line 73:
Why is this passed by pointer if 'status_code' isn't changing in this method?
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@90
PS34, Line 90:
What if 'table_id' isn't there? There will be an exception thrown, and the process will crash. It's not acceptable.
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@108
PS34, Line 108:
For here and elsewhere: why not to return InternalServerError or other server-side error code then?
I don't think returning ServiceUnavailable regardless of exact Status is in-line with the semantics of HTTP 503 code: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/503
I'd think that returning HttpStatusCode::ServiceUnavailable would make sense if status.IsServiceUnavailable() were true. In other words, there should be a meaningful conversion from Status into the server-level subset of HttpStatusCode enumeration.
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@146
PS34, Line 146: Http
nit for here and elsewhere: remove the 'std::' prefix since there are corresponding 'using ...' directives in the beginning of this file already
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@322
PS34, Line 322:
What if status is non-OK here?
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@346
PS34, Line 346:
If the method doesn't return anything but Status::OK(), maybe it should have been returning 'void' instead of 'Status'?
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_test_base.h
File src/kudu/master/rest_catalog_test_base.h:
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_test_base.h@43
PS34, Line 43:
What is this for?
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_test_base.h@50
PS34, Line 50:
nit for here and elsewhere: I'd think of removing the 'kudu::' namespace prefix -- this code is already in the 'kudu' namespace and there isn't any ambiguity for compiler to find Status
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_test_base.h@64
PS34, Line 64:
nit: this could be constexpr
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_test_base.h@74
PS34, Line 74:
:
return tableCreator->Create();
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc
File src/kudu/master/spnego_rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc@67
PS34, Line 67:
nit for here and elsewhere in this test: introduce a const/constexpr static member field and use it instead
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc@72
PS34, Line 72:
nit: in C++ that's usually expressed as
InternalMiniClusterOptions opts;
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc@75
PS34, Line 75:
nit: wrap this into std::move() to avoid extra copying
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc@147
PS34, Line 147:
This isn't a reliable way to check for OK status because error messages can contain 'OK' substring.
If you want to check for a particular status, there are Status::ok() and other methods Status::IsXxx(), e.g. Status::IsNotFound(). Use those instead of checks like this.
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc@206
PS34, Line 206:
nit for here and elsewhere: the expected/reference value should come first -- that way it's much easier to comprehend the error message if {ASSERT,EXPECT}_EQ() assertion is ever triggered; you can try to parse this yourself for both options and see which one is easier to comprehend
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc@220
PS34, Line 220:
Here and elsewhere in case of non-OK status: add an assertion for the exact type of type of status/error returned (e.g., Status::IsNotFound()).
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc@229
PS34, Line 229:
nit: please add a small text blurb to explain why this isn't enabled for macOS
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/server/webserver.cc
File src/kudu/server/webserver.cc:
PS25:
> These cases are already used in the code ...
I could not find HttpStatusCode::NoContent, HttpStatusCode::GatewayTimeout in the code, at least as of PS25, but I might be missing something. Could you point where they are used in the code?
> Could you clarify if there's a specific concern that warrants separating it?
Smaller, logically separated patches makes it much easier to reason about, especially if changing something that a thing on its own, like in this particular case. Logically separated patches are easier for the author to revv, for code maintainers to cherry-pick into other branches, and for reviewers to reason about and provide feedback. It's also easier to assess what test coverage is needed, and implement tests that are adequate and meaningful.
In other words, it's just common sense.
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 25
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Fri, 28 Mar 2025 17:39:24 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 30:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/1195/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 30
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Fri, 21 Mar 2025 18:48:12 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#32).
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: Deleting, altering, or retrieving a table object requires
the table's ID. Operations using table names are not yet supported.
Testing: Two new test files called rest_catalog-test.cc and
spnego_rest_catalog-test.cc have been created to validate these
endpoints using an internal mini-cluster setup. Both tests are based on
the rest_catalog_test_base.h header file.
These test files have been tested for flakiness and evaluated under high
load to ensure stability.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Follow-up patches:
- Support for multi-master setup
- Adding ranger itests
- Swagger/OpenAI documentation for the endpoint under /api/docs path
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master.cc
M src/kudu/master/master.h
A src/kudu/master/rest_catalog-test.cc
A src/kudu/master/rest_catalog_path_handlers.cc
A src/kudu/master/rest_catalog_path_handlers.h
A src/kudu/master/rest_catalog_test_base.h
A src/kudu/master/spnego_rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
10 files changed, 1,381 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/32
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 32
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 32:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/1202/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 32
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Mar 2025 19:22:45 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#31).
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: Deleting, altering, or retrieving a table object requires
the table's ID. Operations using table names are not yet supported.
Testing: Two new test files called rest_catalog-test.cc and
spnego_rest_catalog-test.cc have been created to validate these
endpoints using an internal mini-cluster setup. Both tests are based on
the rest_catalog_test_base.h header file.
These test files have been tested for flakiness and evaluated under high
load to ensure stability.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Follow-up patches:
- Support for multi-master setup
- Adding ranger itests
- Swagger/OpenAI documentation for the endpoint under /api/docs path
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master.cc
M src/kudu/master/master.h
A src/kudu/master/rest_catalog-test.cc
A src/kudu/master/rest_catalog_path_handlers.cc
A src/kudu/master/rest_catalog_path_handlers.h
A src/kudu/master/rest_catalog_test_base.h
A src/kudu/master/spnego_rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
10 files changed, 1,373 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/31
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 31
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 2: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/548/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 26 Sep 2024 15:54:07 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#6).
Change subject: [WIP] KUDU-3608 REST API
......................................................................
[WIP] KUDU-3608 REST API
After careful consideration we opted for implementing a REST API into the Kudu master web server.
The API only handles metadata and will be reachable under the path /api.
It will be implemented using PathHandlers in the master server.
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/master-test.cc
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
3 files changed, 150 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/6
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 6
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 11:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/672/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 11
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 07 Nov 2024 12:25:08 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 20: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/802/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 20
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Dec 2024 21:49:24 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Gabriella Lotz has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 18:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc
File src/kudu/master/master_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@125
PS17, Line 125: } // anonymous namespace
> nit: add a few new lines (e.g. before if statements) for readability
Done
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@136
PS17, Line 136: if (table_id.length() != 32) {
: RETURN_JSON_ERROR(jw, "Invalid table ID", resp->status_code, HttpStatusCode::BadRequest);
: }
: CatalogManager::ScopedLeaderSharedLock l(master_->catalog_manager());
: scoped_refptr<TableInfo> table;
: Status status = master_->catalog_manager()->GetTableInfo(table_id, &table);
:
: i
> this kind of status check is present in almost every method introduced, ext
Done
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@145
PS17, Line 145: }
> could use macro/error handling function here as well, and all similar place
Done
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@190
PS17, Line 190:
> probably better to use your github username for TODOs, to make it more spec
Done
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@214
PS17, Line 214: const string& json_str = req.post_data;
> shouldn't the response status code be set to OK here?
Done
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@237
PS17, Line 237:
> shouldn't the response status code be set to OK here?
Done
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/rest_catalog-test.cc
File src/kudu/master/rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/rest_catalog-test.cc@135
PS17, Line 135: cluster_->mini_master()->bound_http_addr().ToString(),
> probably should check the returned http status OK here
ASSERT_OK checks whether the curl was successful or not. FetchURL returns Remote error if the status code is less than 200 or bigger or equal than 300, otherwise it returns OK.
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/rest_catalog-test.cc@165
PS17, Line 165: K(CreateTestTable());
: string table_id = GetTableId("test_table");
: EasyCurl c;
: faststring buf;
: ASSERT_OK(c.FetchURL(Substitute("http://$0/api/v1/tables/$1",
: cluster_->mini_master()->bound_http_addr().ToString(),
: table_id),
: &buf));
> not sure if this schema is reused in the other tests, but if it is, then ex
Done
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 18
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 09 Dec 2024 08:08:23 +0000
Gerrit-HasComments: Yes
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Zoltan Chovan (Code Review)" <ge...@cloudera.org>.
Zoltan Chovan has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 18:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc
File src/kudu/master/master_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@337
PS17, Line 337: jw.Protobuf(l.data().pb.schema());
> I chose to use the 204 No Content status code for successful DELETE request
okay, makes sense, thanks for the clarification!
http://gerrit.cloudera.org:8080/#/c/21823/18/src/kudu/master/master_path_handlers.cc
File src/kudu/master/master_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/18/src/kudu/master/master_path_handlers.cc@82
PS18, Line 82: #define RETURN_JSON_ERROR(jw, error_msg, status_code, error_code) \
add #undef RETURN_JSON_ERROR at the end of the file so that this macro is only present for this file
http://gerrit.cloudera.org:8080/#/c/21823/18/src/kudu/master/rest_catalog-test.cc
File src/kudu/master/rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/18/src/kudu/master/rest_catalog-test.cc@112
PS18, Line 112: std::string kTableSchema = Substitute("{\"columns\":[$0]}", columns);
these strings can be const as well
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 18
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 05 Dec 2024 08:14:48 +0000
Gerrit-HasComments: Yes
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 15: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/711/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 15
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Nov 2024 19:42:02 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 1:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/525/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 1
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 18 Sep 2024 10:05:50 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 30:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@260
PS30, Line 260: RETURN_JSON_ERROR(jw, "Alter table timed out", *status_code, HttpStatusCode::ServiceUnavailable);
nit: Alter table timed out while waiting for operation completion
(maybe it makes sense to add detail that the alter request went in, we just timed out waiting for the completion to arrive)
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 30
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Mar 2025 13:09:23 +0000
Gerrit-HasComments: Yes
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 6:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/584/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 6
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 09 Oct 2024 12:29:32 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 8:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/652/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 8
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 04 Nov 2024 14:58:34 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#10).
Change subject: [WIP] KUDU-3608 REST API
......................................................................
[WIP] KUDU-3608 REST API
After careful consideration we opted for implementing a REST API
into the Kudu master web server.
The API only handles metadata and will be reachable under the
path /api.
It will be implemented using PathHandlers in the master server.
In this patch there will be the following endpoints:
- GET /api/v1/tables - lists all tables by name and id
- POST /api/v1/tables - creates new table
- GET /api/v1/tables/<table_id> - output is the table object
- PUT /api/v1/tables/<table_id> - alters table
- DELTE /api/v1/tables/<table_id> - deletes table
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/master-test.cc
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
3 files changed, 336 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/10
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 10
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 17: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/728/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 17
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Dec 2024 11:41:32 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 16:
Build Aborted
http://jenkins.kudu.apache.org/job/pre_commit/723/ : ABORTED
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 16
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Nov 2024 05:00:32 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#25).
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
[WIP] KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: Deleting, altering, or retrieving a table object requires
the table's ID. Operations using table names are not yet supported.
Testing: A new test file called rest_catalog-test.cc has been created to
validate these endpoints using an internal mini-cluster setup.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Follow-up patches:
- Adding ranger itests
- Swagger/OpenAI documentation for the endpoint under /api/docs path
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master.cc
M src/kudu/master/master.h
A src/kudu/master/rest_catalog-test.cc
A src/kudu/master/rest_catalog_path_handlers.cc
A src/kudu/master/rest_catalog_path_handlers.h
A src/kudu/master/spnego_rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
9 files changed, 1,148 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/25
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 25
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 2:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/548/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 26 Sep 2024 15:48:46 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 4: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/561/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 4
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 03 Oct 2024 14:51:48 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 3: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/553/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 01 Oct 2024 16:16:30 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 24: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/938/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 24
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 16 Jan 2025 19:36:06 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 18:
Take1: Overall the patch is in nice shape IMO.
I would +1 on Zoltans comment to add a flag to enable/disable the rest endpoints.
My other observation is that it would be probably good to separate out the rest catalog logic into a separate file.
One way to do this:
in master.cc we have
"""
if (web_server_) {
RETURN_NOT_OK(path_handlers_->Register(web_server_.get()));
}
"""
so if you separate out the rest catalog path handlers into a rest_catalog_path_handlers.{h/cc}, based on the flag you just need to add another condition to the above piece of code.
In master.cc on the top of the file you can see how gflags are defined.
Let me know if you need help with the refactor. And I will also examine the tests in the meantime.
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 18
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 09 Dec 2024 10:34:51 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 28:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/1135/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 28
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 04 Mar 2025 17:28:05 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 29:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/21823/29/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/29/src/kudu/master/rest_catalog_path_handlers.cc@210
PS29, Line 210: MonoDelta::FromSeconds(60)
Would it be worth to make this configurable? (flag)
maybe?
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 29
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 19 Mar 2025 14:54:06 +0000
Gerrit-HasComments: Yes
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#5).
Change subject: [WIP] KUDU-3608 REST API
......................................................................
[WIP] KUDU-3608 REST API
After careful consideration we opted for implementing a REST API into the Kudu master web server.
The API only handles metadata and will be reachable under the path /api.
It will be implemented using PathHandlers in the master server.
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
2 files changed, 114 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/5
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 5
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#13).
Change subject: [WIP] KUDU-3608 REST API
......................................................................
[WIP] KUDU-3608 REST API
To simplify metadata access and management, we've implemented
a REST API directly in the Kudu master web server. This API,
under the /api path, is built with PathHandlers and is limited
to metadata operations. The following endpoints are introduced:
- GET /api/v1/tables - Lists all tables by name and id
- POST /api/v1/tables - Creates a new table
- GET /api/v1/tables/<table_id> - Returns table details
- PUT /api/v1/tables/<table_id> - Alters table metadata
- DELETE /api/v1/tables/<table_id> - Deletes a table
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
A src/kudu/master/rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
6 files changed, 461 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/13
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 13
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 23: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/910/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 23
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 09 Jan 2025 17:04:41 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 4:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/561/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 4
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 03 Oct 2024 14:35:43 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#24).
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
[WIP] KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: Deleting, altering, or retrieving a table object requires
the table's ID. Operations using table names are not yet supported.
Testing: A new test file called rest_catalog-test.cc has been created to
validate these endpoints using an internal mini-cluster setup.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Follow-up patches:
- Adding ranger itests
- Swagger/OpenAI documentation for the endpoint under /api/docs path
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master.cc
M src/kudu/master/master.h
A src/kudu/master/rest_catalog-test.cc
A src/kudu/master/rest_catalog_path_handlers.cc
A src/kudu/master/rest_catalog_path_handlers.h
A src/kudu/master/spnego_rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
9 files changed, 1,138 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/24
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 24
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 25: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/1057/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 25
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Feb 2025 18:07:33 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 30:
(9 comments)
http://gerrit.cloudera.org:8080/#/c/21823/29/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/29/src/kudu/master/rest_catalog_path_handlers.cc@210
PS29, Line 210: ;
> Would it be worth to make this configurable? (flag)
Please define a flag in this file for this:
rest_catalog_default_request_timeout_ms = 30 * 1000;
(i guess 30 sec should be enough here)
it should have the tags: advanced, and runtime.
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@70
PS30, Line 70: if (!l.catalog_status().ok()) {
: RETURN_JSON_ERROR(
: jw, l.catalog_status().ToString(), resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
: if (!l.leader_status().ok()) {
: RETURN_JSON_ERROR(
: jw, "Master is not the leader", resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
since this comes up in multiple places i think its worth to create a function for this part:
CheckIsInitializedAndIsLeader()
(in the followup patch we can extend that function to CheckIsInitializedAndIsLeaderOrRedirect() or something like this)
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@95
PS30, Line 95: } else {
: RETURN_JSON_ERROR(
: jw, "Method not allowed", resp->status_code, HttpStatusCode::MethodNotAllowed);
: }
IMO it makes sense to do a check for the supported methods earlier.
(if we get an unsupported method we dont even need to do all the catalog manager stuff)
we can do this before the table id check.
maybe its worth to create a function CheckSupportedRequestMethod()
where you supply the webrequest, and the array of supported methods for that endpoint.
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@108
PS30, Line 108: if (!l.catalog_status().ok()) {
: RETURN_JSON_ERROR(
: jw, l.catalog_status().ToString(), resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
: if (!l.leader_status().ok()) {
: RETURN_JSON_ERROR(
: jw, "Master is not the leader", resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
same as above
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@119
PS30, Line 119: if (!l.catalog_status().ok()) {
: RETURN_JSON_ERROR(
: jw, l.catalog_status().ToString(), resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
: if (!l.leader_status().ok()) {
: RETURN_JSON_ERROR(
: jw, "Master is not the leader", resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
same as above
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@128
PS30, Line 128: else {
: RETURN_JSON_ERROR(
: jw, "Method not allowed", resp->status_code, HttpStatusCode::MethodNotAllowed);
: }
same as above
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@172
PS30, Line 172: google::protobuf::util::Status google_status
nit: const auto s = ...
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@213
PS30, Line 213: google::protobuf::util::Status google_status
nit: const auto s = ...
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@237
PS30, Line 237: status = master_->catalog_manager()->IsAlterTableDone(&check_req, &check_resp, user);
I guess since this logic is also available for create table, it would make sense to check also the create table operation whether it is done.
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 30
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Mar 2025 13:06:40 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Zoltan Chovan (Code Review)" <ge...@cloudera.org>.
Zoltan Chovan has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 30:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/master.cc
File src/kudu/master/master.cc:
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/master.cc@179
PS30, Line 179: LOG(ERROR) << "REST API endpoints cannot be enabled when webserver is disabled. "
: << "Please set --webserver_enabled=true to use REST API.";
: return false;
nit: wrong ident
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@69
PS30, Line 69: CatalogManager::ScopedLeaderSharedLock l(master_->catalog_manager());
: if (!l.catalog_status().ok()) {
: RETURN_JSON_ERROR(
: jw, l.catalog_status().ToString(), resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
: if (!l.leader_status().ok()) {
: RETURN_JSON_ERROR(
: jw, "Master is not the leader", resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
consider extracting this to a utility method, to avoid repeated use
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@106
PS30, Line 106: if (req.request_method == "GET") {
why not simplify these checks, so that the locks are only checked once, then route the request to the relevant handler, e.g.
if(LocksAreOkay()) {
if (POST) HandlePostTables()
else if (GET) HandlePostTables()
}
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@234
PS30, Line 234: 60
consider making this a flag instead of a hard coded value
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 30
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Mar 2025 13:15:28 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Gabriella Lotz has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 30:
(14 comments)
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/master.cc
File src/kudu/master/master.cc:
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/master.cc@179
PS30, Line 179: LOG(ERROR) << "REST API endpoints cannot be enabled when webserver is disabled. "
: << "Please set --webserver_enabled=true to use REST API.";
: return false;
> nit: wrong ident
Done
http://gerrit.cloudera.org:8080/#/c/21823/29/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/29/src/kudu/master/rest_catalog_path_handlers.cc@110
PS29, Line 110: jw, l.catalog_status().ToString(),
> here and all the other handler implementations:
Done
http://gerrit.cloudera.org:8080/#/c/21823/29/src/kudu/master/rest_catalog_path_handlers.cc@210
PS29, Line 210: ;
> Please define a flag in this file for this:
Done
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@69
PS30, Line 69: CatalogManager::ScopedLeaderSharedLock l(master_->catalog_manager());
: if (!l.catalog_status().ok()) {
: RETURN_JSON_ERROR(
: jw, l.catalog_status().ToString(), resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
: if (!l.leader_status().ok()) {
: RETURN_JSON_ERROR(
: jw, "Master is not the leader", resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
> consider extracting this to a utility method, to avoid repeated use
Done
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@70
PS30, Line 70: if (!l.catalog_status().ok()) {
: RETURN_JSON_ERROR(
: jw, l.catalog_status().ToString(), resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
: if (!l.leader_status().ok()) {
: RETURN_JSON_ERROR(
: jw, "Master is not the leader", resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
> since this comes up in multiple places i think its worth to create a functi
Done
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@95
PS30, Line 95: } else {
: RETURN_JSON_ERROR(
: jw, "Method not allowed", resp->status_code, HttpStatusCode::MethodNotAllowed);
: }
> IMO it makes sense to do a check for the supported methods earlier.
It makes sense, but checking the request method at the beginning means we first verify whether it's GET/POST or GET/PUT/DELETE. If it doesn't match, we return an error. Then, after additional error handling, we check the method again to differentiate the cases. My concern is that this results in checking the request method twice.
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@106
PS30, Line 106: if (req.request_method == "GET") {
> why not simplify these checks, so that the locks are only checked once, the
Done
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@108
PS30, Line 108: if (!l.catalog_status().ok()) {
: RETURN_JSON_ERROR(
: jw, l.catalog_status().ToString(), resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
: if (!l.leader_status().ok()) {
: RETURN_JSON_ERROR(
: jw, "Master is not the leader", resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
> same as above
Done
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@119
PS30, Line 119: if (!l.catalog_status().ok()) {
: RETURN_JSON_ERROR(
: jw, l.catalog_status().ToString(), resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
: if (!l.leader_status().ok()) {
: RETURN_JSON_ERROR(
: jw, "Master is not the leader", resp->status_code, HttpStatusCode::ServiceUnavailable);
: }
> same as above
Done
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@172
PS30, Line 172: google::protobuf::util::Status google_status
> nit: const auto s = ...
Done
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@213
PS30, Line 213: google::protobuf::util::Status google_status
> nit: const auto s = ...
Done
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@234
PS30, Line 234: 60
> consider making this a flag instead of a hard coded value
Done
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@237
PS30, Line 237: status = master_->catalog_manager()->IsAlterTableDone(&check_req, &check_resp, user);
> I guess since this logic is also available for create table, it would make
Done
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@260
PS30, Line 260: RETURN_JSON_ERROR(jw, "Alter table timed out", *status_code, HttpStatusCode::ServiceUnavailable);
> nit: Alter table timed out while waiting for operation completion
Done
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 30
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Mar 2025 14:20:36 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 30: Verified+1
Build Successful
http://jenkins.kudu.apache.org/job/pre_commit/1195/ : SUCCESS
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 30
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Fri, 21 Mar 2025 19:35:28 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 31: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/1196/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 31
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Mar 2025 15:39:51 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 28: Verified+1
Build Successful
http://jenkins.kudu.apache.org/job/pre_commit/1135/ : SUCCESS
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 28
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 04 Mar 2025 18:24:19 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 25:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/1057/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 25
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Feb 2025 17:11:25 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 11: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/672/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 11
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 07 Nov 2024 13:55:39 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 13:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/696/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 13
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 20 Nov 2024 17:44:14 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Zoltan Chovan (Code Review)" <ge...@cloudera.org>.
Zoltan Chovan has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 17:
(11 comments)
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc
File src/kudu/master/master_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@125
PS17, Line 125: if (table_id.length() != 32) {
nit: add a few new lines (e.g. before if statements) for readability
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@136
PS17, Line 136: if (!status.ok()) {
: jw.StartObject();
: jw.String("error");
: jw.String(status.ToString());
: jw.EndObject();
: resp->status_code = HttpStatusCode::ServiceUnavailable;
: return;
: }
this kind of status check is present in almost every method introduced, extract the majority of these into a common function or macro 'RETURN_JSON_ERROR' or something like:
#define RETURN_JSON_ERROR (error_msg, code) {
JsonWriter jw(output, JsonWriter::COMPACT);
jw.String("error"); \
jw.String(error_msg); \
jw.EndObject(); \
*status_code = code;
return;
}
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@145
PS17, Line 145: jw.StartObject();
could use macro/error handling function here as well, and all similar places across the methods
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@190
PS17, Line 190: Gabi
probably better to use your github username for TODOs, to make it more specific for others
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@214
PS17, Line 214: }
shouldn't the response status code be set to OK here?
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@237
PS17, Line 237: PrintTableObject(output, response.table_id());
shouldn't the response status code be set to OK here?
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/master_path_handlers.cc@337
PS17, Line 337: *status_code = HttpStatusCode::NoContent;
why NoContent status instead of OK? the delete was successful
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/rest_catalog-test.cc
File src/kudu/master/rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/rest_catalog-test.cc@135
PS17, Line 135: ASSERT_STR_CONTAINS(buf.ToString(), "{\"tables\":[]}");
probably should check the returned http status OK here
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/rest_catalog-test.cc@149
PS17, Line 149: }
probably should check the returned http status OK here and everywhere else where we expect a successful outcome
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/rest_catalog-test.cc@165
PS17, Line 165: "{\"name\":\"test_table\",\"id\":\"$0\",\"schema\":{"
: "\"columns\":[{\"id\":10,\"name\":\"key\",\"type\":\"INT32\",\"is_key\":true,\"is_"
: "nullable\":false,\"encoding\":\"AUTO_ENCODING\",\"compression\":\"DEFAULT_COMPRESSION\","
: "\"cfile_block_size\":0,\"immutable\":false},{\"id\":11,\"name\":\"int_val\",\"type\":"
: "\"INT32\",\"is_key\":false,\"is_nullable\":false,\"encoding\":\"AUTO_ENCODING\","
: "\"compression\":\"DEFAULT_COMPRESSION\",\"cfile_block_size\":0,\"immutable\":false}]},"
: "\"partition_schema\":{\"range_schema\":{\"columns\":[{\"id\":10}]}},\"owner\":\"$1\","
: "\"comment\":\"\",\"extra_config\":{}}
not sure if this schema is reused in the other tests, but if it is, then extract it into a const e.g.:
const char kTableSchema = "....";
if it's not exactly the same then try to parameterise it so it can be reused by passing it into a Substitute()
http://gerrit.cloudera.org:8080/#/c/21823/17/src/kudu/master/rest_catalog-test.cc@266
PS17, Line 266: R"({
: "name": "test2",
: "schema": {
: "columns": [
: {"name": "id", "type": "INT32", "is_key": true},
: {"name": "key", "type": "INT64", "is_nullable": false, "is_key": true, "comment": "range partition column"},
: {"name": "name", "type": "STRING", "is_nullable": false, "comment": "user name"}
: ]
: },
: "partition_schema": {
: "hash_schema": {
: "columns": [{"name": "key"}, {"name": "id"}],
: "num_buckets": 2,
: "seed": 8
: },
: "range_schema": {
: "columns": [{"name": "key"}]
: }
: },
: "num_replicas": 1,
: "owner": "root"
: })",
: &buf);
extract to a const string if used multiple times
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 17
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Dec 2024 13:09:01 +0000
Gerrit-HasComments: Yes
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 14:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/697/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 14
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 21 Nov 2024 16:40:09 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 22:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/829/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 22
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 16 Dec 2024 09:28:23 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 27: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/1130/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 27
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 03 Mar 2025 16:06:16 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#28).
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
[WIP] KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: Deleting, altering, or retrieving a table object requires
the table's ID. Operations using table names are not yet supported.
Testing: A new test file called rest_catalog-test.cc has been created to
validate these endpoints using an internal mini-cluster setup.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Follow-up patches:
- Adding ranger itests
- Swagger/OpenAI documentation for the endpoint under /api/docs path
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master.cc
M src/kudu/master/master.h
A src/kudu/master/rest_catalog-test.cc
A src/kudu/master/rest_catalog_path_handlers.cc
A src/kudu/master/rest_catalog_path_handlers.h
A src/kudu/master/rest_catalog_test_base.h
A src/kudu/master/spnego_rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
10 files changed, 1,181 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/28
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 28
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Gabriella Lotz has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 34:
(23 comments)
Thank you for all the comments!
I created a follow-up for these changes, that can be found here: https://gerrit.cloudera.org/c/22744/
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog-test.cc
File src/kudu/master/rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog-test.cc@537
PS34, Line 537: s = client_->OpenTable(kTableName, &table);
: ASSERT_TRUE(s.ok());
> nit for here and elsewhere where the status isn't used elsewhere: it might
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.h
File src/kudu/master/rest_catalog_path_handlers.h:
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.h@34
PS34, Line 34: class RestCatalogPathHandlers {
> nit: consider making this class 'final'
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.h@36
PS34, Line 36: master
> nit: consider adding DCHECK_NOT_NULL wrapper around 'master'
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.h@38
PS34, Line 38: ~RestCatalogPathHandlers();
> nit: if the destructor is empty, consider using
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@24
PS34, Line 24: #include "google/protobuf/util/json_util.h"
> nit: change quotes to '<' and '>' and re-arrange the lines around to be sor
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@71
PS34, Line 71: JsonWriter* jw
> Why is this passed by pointer if 'jw' isn't changing in this method?
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@73
PS34, Line 73: HttpStatusCode* status_code
> Why is this passed by pointer if 'status_code' isn't changing in this metho
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@90
PS34, Line 90: at("table_id")
> What if 'table_id' isn't there? There will be an exception thrown, and the
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@108
PS34, Line 108: HttpStatusCode::ServiceUnavailable
> For here and elsewhere: why not to return InternalServerError or other serv
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@146
PS34, Line 146: std::
> nit for here and elsewhere: remove the 'std::' prefix since there are corre
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@322
PS34, Line 322: Status status = master_->catalog_manager()->GetTableInfo(table_id, &table);
> What if status is non-OK here?
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_path_handlers.cc@346
PS34, Line 346: Status
> If the method doesn't return anything but Status::OK(), maybe it should hav
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_test_base.h
File src/kudu/master/rest_catalog_test_base.h:
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_test_base.h@43
PS34, Line 43: DECLARE_bool(enable_rest_api);
> What is this for?
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_test_base.h@50
PS34, Line 50: kudu::
> nit for here and elsewhere: I'd think of removing the 'kudu::' namespace pr
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_test_base.h@64
PS34, Line 64: int32_t increment
> nit: this could be constexpr
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/rest_catalog_test_base.h@74
PS34, Line 74: kudu::Status s = tableCreator->Create();
: return s;
> return tableCreator->Create();
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc
File src/kudu/master/spnego_rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc@67
PS34, Line 67: "alice"
> nit for here and elsewhere in this test: introduce a const/constexpr static
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc@72
PS34, Line 72: auto opts = InternalMiniClusterOptions();
> nit: in C++ that's usually expressed as
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc@75
PS34, Line 75: opts
> nit: wrap this into std::move() to avoid extra copying
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc@147
PS34, Line 147: ASSERT_STR_CONTAINS(s.ToString(), "OK");
> This isn't a reliable way to check for OK status because error messages can
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc@206
PS34, Line 206: ASSERT_EQ(table->schema().num_columns(), 3);
> nit for here and elsewhere: the expected/reference value should come first
Done
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc@220
PS34, Line 220: ASSERT_STR_CONTAINS(s.ToString(), "HTTP 500");
> Here and elsewhere in case of non-OK status: add an assertion for the exact
Although specific handling is not yet implemented for all error types, I have included an extra check to ensure that the status s is not OK.
http://gerrit.cloudera.org:8080/#/c/21823/34/src/kudu/master/spnego_rest_catalog-test.cc@229
PS34, Line 229: #ifndef __APPLE__
> nit: please add a small text blurb to explain why this isn't enabled for ma
Done
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 34
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 07 Apr 2025 12:00:59 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#29).
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: Deleting, altering, or retrieving a table object requires
the table's ID. Operations using table names are not yet supported.
Testing: A new test file called rest_catalog-test.cc has been created to
validate these endpoints using an internal mini-cluster setup.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Follow-up patches:
- Adding ranger itests
- Swagger/OpenAI documentation for the endpoint under /api/docs path
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master.cc
M src/kudu/master/master.h
A src/kudu/master/rest_catalog-test.cc
A src/kudu/master/rest_catalog_path_handlers.cc
A src/kudu/master/rest_catalog_path_handlers.h
A src/kudu/master/rest_catalog_test_base.h
A src/kudu/master/spnego_rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
10 files changed, 1,319 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/29
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 29
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 32: Verified+1
Build Successful
http://jenkins.kudu.apache.org/job/pre_commit/1202/ : SUCCESS
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 32
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Mar 2025 20:04:57 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 29:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/21823/29/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/29/src/kudu/master/rest_catalog_path_handlers.cc@110
PS29, Line 110: RestCatalogPathHandlers::HandleGetTables
here and all the other handler implementations:
its probably good practice to check if the catalog manager is ready and check for leader status.
Maybe we can just move the lock into the handlers and something like this
https://github.com/apache/kudu/blob/197005b5ec7fa535dce896d7b1ff12f8c3c66157/src/kudu/master/master_path_handlers.cc#L245-L255
if we are not leader, for now we can just error out. and in the followup patch we can do the redirect there.
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 29
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Fri, 21 Mar 2025 11:02:54 +0000
Gerrit-HasComments: Yes
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 7: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/590/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 7
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 14 Oct 2024 13:50:32 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 8: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/652/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 8
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 04 Nov 2024 15:04:12 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#4).
Change subject: [WIP] KUDU-3608 REST API
......................................................................
[WIP] KUDU-3608 REST API
After careful consideration we opted for implementing a REST API into the Kudu master web server.
The API only handles metadata and will be reachable under the path /api.
It will be implemented using PathHandlers in the master server.
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
2 files changed, 121 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/4
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 4
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 1: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/525/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 1
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 18 Sep 2024 10:11:13 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: Deleting, altering, or retrieving a table object requires
the table's ID. Operations using table names are not yet supported.
Testing: Two new test files called rest_catalog-test.cc and
spnego_rest_catalog-test.cc have been created to validate these
endpoints using an internal mini-cluster setup. Both tests are based on
the rest_catalog_test_base.h header file.
These test files have been tested for flakiness and evaluated under high
load to ensure stability.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Follow-up patches:
- Support for multi-master setup
- Adding ranger itests
- Swagger/OpenAI documentation for the endpoint under /api/docs path
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Reviewed-on: http://gerrit.cloudera.org:8080/21823
Tested-by: Kudu Jenkins
Reviewed-by: Marton Greber <gr...@gmail.com>
Reviewed-by: Zoltan Chovan <zc...@cloudera.com>
Reviewed-by: Zoltan Martonka <zm...@cloudera.com>
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master.cc
M src/kudu/master/master.h
A src/kudu/master/rest_catalog-test.cc
A src/kudu/master/rest_catalog_path_handlers.cc
A src/kudu/master/rest_catalog_path_handlers.h
A src/kudu/master/rest_catalog_test_base.h
A src/kudu/master/spnego_rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
10 files changed, 1,395 insertions(+), 1 deletion(-)
Approvals:
Kudu Jenkins: Verified
Marton Greber: Looks good to me, approved
Zoltan Chovan: Looks good to me, but someone else must approve
Zoltan Martonka: Looks good to me, but someone else must approve
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 34
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 29:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/1170/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 29
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 13 Mar 2025 10:54:31 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 21: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/811/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 21
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 11 Dec 2024 14:38:52 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 22: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/829/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 22
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 16 Dec 2024 10:57:27 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Gabriella Lotz has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 24:
(10 comments)
http://gerrit.cloudera.org:8080/#/c/21823/23//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/21823/23//COMMIT_MSG@26
PS23, Line 26: Deleting, altering, or retrieving a table object requires
: the table's ID. Operations using table names are n
> You can remove this line, once the kdc tests are included.
Done
http://gerrit.cloudera.org:8080/#/c/21823/23//COMMIT_MSG@38
PS23, Line 38: - Adding ranger itests
> You can mention at the end that there will be follow-up patches for:
Done
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/master.cc
File src/kudu/master/master.cc:
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/master.cc@135
PS23, Line 135:
> "Enables REST API endpoints for metadata management in the master server."
Done
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/master.cc@136
PS23, Line 136: false,
> Since SPNEGO works now with the api, i think we can remove this line, right
Done
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/master.cc@137
PS23, Line 137: "Enables REST API endpoints in the master server. The flag webserver_enabled must be "
> Looking at webserver_enabled flag, if that uses
Done
http://gerrit.cloudera.org:8080/#/c/21823/18/src/kudu/master/master_path_handlers.cc
File src/kudu/master/master_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/18/src/kudu/master/master_path_handlers.cc@82
PS18, Line 82: using kudu::security::DataFormat;
> add #undef RETURN_JSON_ERROR at the end of the file so that this macro is o
Done
http://gerrit.cloudera.org:8080/#/c/21823/18/src/kudu/master/rest_catalog-test.cc
File src/kudu/master/rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/18/src/kudu/master/rest_catalog-test.cc@112
PS18, Line 112: if (!s.ok()) {
> these strings can be const as well
Done
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/rest_catalog-test.cc
File src/kudu/master/rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/rest_catalog-test.cc@1
PS23, Line 1: // Licensed to the Apache Software Foundation (ASF) under one
> Please add the Apache license in the beginning of this files as well.
Done
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/rest_catalog-test.cc@106
PS23, Line 106: return s;
: }
:
: string GetTableId(const string& table_name) {
: shared_ptr<KuduTable> table;
: Status s = client_->OpenTable(table_name, &table);
: if (!s.ok()) {
: LOG(ERROR) << "OpenTable failed: " << s.ToString();
: return "";
: }
: if (table->name() == table_name) {
: return table->id();
: }
: return "";
: }
> nit: I just saw that we have "class KuduRegex " in src/kudu/util/regex.h.
Done
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/rest_catalog_path_handlers.cc@58
PS23, Line 58: RestCatalogPathHandlers::~RestCatalogPathHandlers() {}
:
: void RestCatalogPathHandlers::HandleApiTableEndpoint(const Webserver::WebRequest& req,
: Webserver::PrerenderedWebResponse* resp) {
: string table_id = req.path_params.at("table_id");
: ostringstream* output = &resp->output;
: JsonWriter jw(output, JsonWriter::COMPACT);
:
: if (table_id.length() != 32) {
: RETURN_JSON_ERROR(jw, "Invalid table ID", resp->status_code, HttpStatusCode::BadRequest);
: }
: CatalogManager::ScopedLeaderSharedLock l(master_->catalog_manager());
: scoped_refptr<TableInfo> table;
: Status status = master_->catalog_manager()->GetTableInfo(table_id, &table);
:
: if (!status.ok()) {
> I dont see this part being used in this file. Can we remove it?
Done
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 24
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 16 Jan 2025 18:17:06 +0000
Gerrit-HasComments: Yes
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 24:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/938/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 24
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 16 Jan 2025 18:16:13 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 26: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/1129/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 26
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 03 Mar 2025 14:13:36 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#27).
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
[WIP] KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: Deleting, altering, or retrieving a table object requires
the table's ID. Operations using table names are not yet supported.
Testing: A new test file called rest_catalog-test.cc has been created to
validate these endpoints using an internal mini-cluster setup.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Follow-up patches:
- Adding ranger itests
- Swagger/OpenAI documentation for the endpoint under /api/docs path
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master.cc
M src/kudu/master/master.h
A src/kudu/master/rest_catalog-test.cc
A src/kudu/master/rest_catalog_path_handlers.cc
A src/kudu/master/rest_catalog_path_handlers.h
A src/kudu/master/rest_catalog_test_base.h
A src/kudu/master/spnego_rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
10 files changed, 1,189 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/27
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 27
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 27:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/1130/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 27
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 03 Mar 2025 16:03:25 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 23:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/21823/23//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/21823/23//COMMIT_MSG@26
PS23, Line 26: No Security Measures Yet. This implementation does not
: currently include authentication or authorization.
You can remove this line, once the kdc tests are included.
http://gerrit.cloudera.org:8080/#/c/21823/23//COMMIT_MSG@38
PS23, Line 38:
You can mention at the end that there will be follow-up patches for:
- writing adding ranger itest
- swagger/docs
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/master.cc
File src/kudu/master/master.cc:
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/master.cc@135
PS23, Line 135: Enables REST API endpoints in the master server.
"Enables REST API endpoints for metadata management in the master server."
Maybe it is also worthwhile to mention that for this to work, the flag webserver_enabled must also be true. (just stating the obvious for users).
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/master.cc@136
PS23, Line 136: TAG_FLAG(enable_rest_api, unsafe);
Since SPNEGO works now with the api, i think we can remove this line, right?
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/master.cc@137
PS23, Line 137:
Looking at webserver_enabled flag, if that uses
TAG_FLAG(webserver_enabled, advanced);
its probably good to have it here as well.
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/rest_catalog-test.cc
File src/kudu/master/rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/rest_catalog-test.cc@1
PS23, Line 1: #include <regex.h>
Please add the Apache license in the beginning of this files as well.
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/rest_catalog-test.cc@106
PS23, Line 106: static int FindColumnId(string const &schema_json) {
: regex_t column_id_regex;
: regmatch_t match[2];
: const char* pattern = "\\{\"id\":([0-9]+),\"name\":\"key\"";
: if (regcomp(&column_id_regex, pattern, REG_EXTENDED) != 0) {
: return -1;
: }
: if (regexec(&column_id_regex, schema_json.c_str(), 2, match, 0) == 0) {
: std::string column_id = schema_json.substr(match[1].rm_so, match[1].rm_eo - match[1].rm_so);
: regfree(&column_id_regex);
: return std::stoi(column_id);
: }
: regfree(&column_id_regex);
: return -1;
: }
nit: I just saw that we have "class KuduRegex " in src/kudu/util/regex.h.
Maybe you can simplify this function by using that class. (its just a wrapper around regex.h)
If this turns out to be too much work, just ignore this.
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/23/src/kudu/master/rest_catalog_path_handlers.cc@58
PS23, Line 58: namespace {
: struct TServerStateInfo {
: // The stringified version of the state.
: string state;
:
: // A human-readable version of the time the tablet server was put into this
: // state.
: string time_entered_state;
:
: // The current status of the tserver (e.g. "Live", "Dead").
: string status;
:
: // A link to the tserver's web UI, if one exists.
: string webserver;
: };
: } // anonymous namespace
I dont see this part being used in this file. Can we remove it?
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 23
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Fri, 10 Jan 2025 11:56:59 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 34:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/21823/34//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/21823/34//COMMIT_MSG@29
PS34, Line 29: Testing
Existing test scenarios in this changelist as of PS34 doesn't provide adequate coverage for the new functionality. Consider adding more test coverage, especially for 'negative' scenarios when some parameters are missing, wrong type of HTTP methods are used, invalid JSON data is supplied (corrupted data, and just missing fields), etc. At least, I could see that with PS34 it's now easy to stage inadvertent DoS attacks by crashing Kudu masters once this code is compiled in and the REST server is enabled.
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 34
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Fri, 28 Mar 2025 17:58:40 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 31:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_path_handlers.cc@95
PS30, Line 95: }
:
: if (!table) {
:
> It makes sense, but checking the request method at the beginning means we f
Make sense, lets keep it as it is. Thanks!
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_path_handlers.cc@159
PS31, Line 159: RestCatalogPathHandlers::HandlePostTables
does it make sense to only create one JsonWriter instance in the beginning of the function and just use that one?
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_path_handlers.cc@229
PS31, Line 229: RestCatalogPathHandlers::HandlePutTable(
does it make sense to only create one JsonWriter instance in the beginning of the function and just use that one?
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 31
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Mar 2025 18:13:56 +0000
Gerrit-HasComments: Yes
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Gabriella Lotz has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 25:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/server/webserver.cc
File src/kudu/server/webserver.cc:
PS25:
> Separate this and other generic changes in web_callback_registry.h into its
These cases are already used in the code, and given the scope of this patch, creating a separate changelist for this change may not be necessary. Could you clarify if there's a specific concern that warrants separating it? The same applies to the changes in webserver.cc.
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 25
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Feb 2025 16:56:02 +0000
Gerrit-HasComments: Yes
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#26).
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
[WIP] KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: Deleting, altering, or retrieving a table object requires
the table's ID. Operations using table names are not yet supported.
Testing: A new test file called rest_catalog-test.cc has been created to
validate these endpoints using an internal mini-cluster setup.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Follow-up patches:
- Adding ranger itests
- Swagger/OpenAI documentation for the endpoint under /api/docs path
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master.cc
M src/kudu/master/master.h
A src/kudu/master/rest_catalog-test.cc
A src/kudu/master/rest_catalog_path_handlers.cc
A src/kudu/master/rest_catalog_path_handlers.h
A src/kudu/master/rest_catalog_test_base.h
A src/kudu/master/spnego_rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
10 files changed, 1,158 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/26
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 26
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 12: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/684/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 12
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 Nov 2024 09:04:56 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 19:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/796/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 19
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Dec 2024 10:19:27 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/21823/2/src/kudu/master/master_path_handlers.cc
File src/kudu/master/master_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/2/src/kudu/master/master_path_handlers.cc@116
PS2, Line 116: void MasterPathHandlers::HandleAPI(const Webserver::WebRequest& req,
: Webserver::PrerenderedWebResponse* resp) {
[question for later]:
put a todo here that we also need to thing about how to organize different rest catalog endpoints: one master handler or spread them
http://gerrit.cloudera.org:8080/#/c/21823/2/src/kudu/master/master_path_handlers.cc@118
PS2, Line 118: if (req.request_method == "GET")
this get request part seems to be okay. can you please take a look at master server test? we have existing tests that hit a given endpoint and assert the response, for example:
https://github.com/apache/kudu/blob/bfa875c00d657b8d5c37ed2032cf82c6c5704f37/src/kudu/master/master-test.cc#L2381C1-L2392C2
can you take a stab and try to implement such a test for the GET request?
http://gerrit.cloudera.org:8080/#/c/21823/2/src/kudu/master/master_path_handlers.cc@163
PS2, Line 163: CreateTableRequestPB request;
: CreateTableResponsePB response;
: // TODO: parse the post_data and fill the request object
iiuc we just put together the api response, but we do not send the populated CreateTableResponsePB right?
Can you look for other places to figure out how to send the create table RPC, in order to actually create the table?
let us know if you need help with this.
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 30 Sep 2024 12:44:35 +0000
Gerrit-HasComments: Yes
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 3:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/553/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 01 Oct 2024 15:32:04 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 6: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/584/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 6
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 09 Oct 2024 12:45:32 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 5:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/21823/5/src/kudu/master/master_path_handlers.cc
File src/kudu/master/master_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/5/src/kudu/master/master_path_handlers.cc@974
PS5, Line 974: *
What semantics does this have? I don't think current implementation of handlers support wildcard paths.
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 5
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 08 Oct 2024 18:30:03 +0000
Gerrit-HasComments: Yes
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 14:
Build Aborted
http://jenkins.kudu.apache.org/job/pre_commit/697/ : ABORTED
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 14
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Nov 2024 04:40:10 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 18:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/749/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 18
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 04 Dec 2024 16:37:44 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 23:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/910/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 23
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 09 Jan 2025 15:58:49 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#15).
Change subject: [WIP] KUDU-3608 REST API
......................................................................
[WIP] KUDU-3608 REST API
To simplify metadata access and management, we've implemented
a REST API directly in the Kudu master web server. This API,
under the /api path, is built with PathHandlers and is limited
to metadata operations. The following endpoints are introduced:
- GET /api/v1/tables - Lists all tables by name and id
- POST /api/v1/tables - Creates a new table
- GET /api/v1/tables/<table_id> - Returns table details
- PUT /api/v1/tables/<table_id> - Alters table metadata
- DELETE /api/v1/tables/<table_id> - Deletes a table
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
A src/kudu/master/rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
6 files changed, 813 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/15
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 15
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 16:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/723/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 16
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Nov 2024 17:00:31 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#14).
Change subject: [WIP] KUDU-3608 REST API
......................................................................
[WIP] KUDU-3608 REST API
To simplify metadata access and management, we've implemented
a REST API directly in the Kudu master web server. This API,
under the /api path, is built with PathHandlers and is limited
to metadata operations. The following endpoints are introduced:
- GET /api/v1/tables - Lists all tables by name and id
- POST /api/v1/tables - Creates a new table
- GET /api/v1/tables/<table_id> - Returns table details
- PUT /api/v1/tables/<table_id> - Alters table metadata
- DELETE /api/v1/tables/<table_id> - Deletes a table
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
A src/kudu/master/rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
6 files changed, 729 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/14
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 14
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 5: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/581/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 5
Gerrit-Owner: Anonymous Coward <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 08 Oct 2024 16:49:32 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 12:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/684/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 12
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 Nov 2024 08:10:08 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Zoltan Chovan (Code Review)" <ge...@cloudera.org>.
Zoltan Chovan has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 21: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 21
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 11 Dec 2024 13:55:46 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 20:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/802/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 20
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Dec 2024 21:12:20 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 21:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/811/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 21
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 11 Dec 2024 13:40:14 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#22).
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
[WIP] KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: No Security Measures Yet. This implementation does not
currently include authentication or authorization.
Deleting, altering, or retrieving a table object requires the table's
ID. Operations using table names are not yet supported.
Testing: A new test file called rest_catalog-test.cc has been created to
validate these endpoints using an internal mini-cluster setup.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master.cc
M src/kudu/master/master.h
A src/kudu/master/rest_catalog-test.cc
A src/kudu/master/rest_catalog_path_handlers.cc
A src/kudu/master/rest_catalog_path_handlers.h
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
8 files changed, 986 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/22
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 22
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#23).
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
[WIP] KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: No Security Measures Yet. This implementation does not
currently include authentication or authorization.
Deleting, altering, or retrieving a table object requires the table's
ID. Operations using table names are not yet supported.
Testing: A new test file called rest_catalog-test.cc has been created to
validate these endpoints using an internal mini-cluster setup.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master.cc
M src/kudu/master/master.h
A src/kudu/master/rest_catalog-test.cc
A src/kudu/master/rest_catalog_path_handlers.cc
A src/kudu/master/rest_catalog_path_handlers.h
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
8 files changed, 1,003 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/23
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 23
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 25:
(11 comments)
Thank you for working on this!
I took a quick look just to see if there is something high-level that I might recommend even at the WIP stage. Left a few suggestions.
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/master.cc
File src/kudu/master/master.cc:
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/master.cc@137
PS25, Line 137: The flag webserver_enabled must be "
: "set to true for this flag to take effect.
Add a group flag validator to enforce this. For reference, check out how the GROUP_FLAG_VALIDATOR macro is used in the existing code.
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/master.cc@259
PS25, Line 259: rest_catalog_path_handlers_
Do we need this if FLAGS_enable_rest_api = false?
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/master.cc@305
PS25, Line 305: FLAGS_enable_rest_api
Since this code is run just once during initialization, there isn't much sense trying to use the `--enable_rest_api` flag as a run-time one. With that, consider switching this criteria based on whether rest_catalog_path_handlers_ is nullptr or not.
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/rest_catalog-test.cc
File src/kudu/master/rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/rest_catalog-test.cc@82
PS25, Line 82: Status CreateTestTable() {
: string kTableName = "test_table";
: KuduSchema schema;
: KuduSchemaBuilder b;
: b.AddColumn("key")->Type(KuduColumnSchema::INT32)->NotNull()->PrimaryKey();
: b.AddColumn("int_val")->Type(KuduColumnSchema::INT32)->NotNull();
: KUDU_CHECK_OK(b.Build(&schema));
: vector<string> columnNames;
: columnNames.emplace_back("key");
:
: // Set the schema and range partition columns.
: KuduTableCreator* tableCreator = client_->NewTableCreator();
: tableCreator->table_name(kTableName).schema(&schema).set_range_partition_columns(columnNames);
:
: // Generate and add the range partition splits for the table.
: int32_t increment = 1000 / 10;
: for (int32_t i = 1; i < 10; i++) {
: KuduPartialRow* row = schema.NewRow();
: KUDU_CHECK_OK(row->SetInt32(0, i * increment));
: tableCreator->add_range_partition_split(row);
: }
: tableCreator->num_replicas(1);
: Status s = tableCreator->Create();
: delete tableCreator;
: return s;
: }
:
: string GetTableId(const string& table_name) {
: shared_ptr<KuduTable> table;
: Status s = client_->OpenTable(table_name, &table);
: if (!s.ok()) {
: LOG(ERROR) << "OpenTable failed: " << s.ToString();
: return "";
: }
: if (table->name() == table_name) {
: return table->id();
: }
: return "";
: }
This looks like copy-and-paste from spnego_rest_catalog-test.cc. Maybe, introduce a base test class or at least separate the common code as utility methods shared across multiple tests?
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/rest_catalog-test.cc@132
PS25, Line 132: string column_id_str = std::to_string(column_id);
: string column_id_1_str = std::to_string(column_id + 1);
There is no need to convert integers to strings if using Substitute.
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/spnego_rest_catalog-test.cc
File src/kudu/master/spnego_rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/spnego_rest_catalog-test.cc@65
PS25, Line 65: const vector<string> kExtraFlags = {
: "--enable_rest_api"
: };
Why to introduce this local constant if it's used only once at line 70?
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/spnego_rest_catalog-test.cc@84
PS25, Line 84: KUDU_CHECK_OK
Why KUDU_CHECK_OK? There is RETURN_NOT_OK() to handle this in a method returning Status.
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/spnego_rest_catalog-test.cc@89
PS25, Line 89: tableCreator
Wrap this into std::unique_ptr?
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/master/spnego_rest_catalog-test.cc@112
PS25, Line 112: if (table->name() == table_name) {
: return table->id();
: }
Why to add this extra check if the table was opened successfully?
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/server/webserver.cc
File src/kudu/server/webserver.cc:
PS25:
Separate this and other generic changes in web_callback_registry.h into its own changelist?
http://gerrit.cloudera.org:8080/#/c/21823/25/src/kudu/util/web_callback_registry.h
File src/kudu/util/web_callback_registry.h:
PS25:
Separate this and other generic changes in webserver.cc into its own changelist?
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 25
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 12 Feb 2025 18:00:00 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 31:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog-test.cc
File src/kudu/master/rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog-test.cc@198
PS30, Line 198: c.set_verbose(true);
do we need this line?
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog-test.cc@211
PS30, Line 211: c.set_verbose(true);
do we need this line?
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog-test.cc
File src/kudu/master/rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog-test.cc@173
PS31, Line 173: string partition_schema =
: (first_column_id == 0) ? kTablePartitionSchemaColumnIdZero : kTablePartitionSchema;
nit: can you please add a small comment here?
In DEBUG builds, column id starts from 10 while in RELEASE builds it starts from 0.
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog-test.cc@256
PS31, Line 256: ASSERT_STR_CONTAINS(s.ToString(), "HTTP 411");
nit: maybe assert the error message? Curious if we return reasonable error message in this case.
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_path_handlers.cc
File src/kudu/master/rest_catalog_path_handlers.cc:
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_path_handlers.cc@86
PS31, Line 86: RETURN_JSON_ERROR(jw, "Invalid table ID", resp->status_code, HttpStatusCode::BadRequest);
nit: maybe add more error message detail?
"Invalid table ID: must be exactly 32 characters long."
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_test_base.h
File src/kudu/master/rest_catalog_test_base.h:
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/rest_catalog_test_base.h@79
PS31, Line 79: std::string GetTableId(const std::string& table_name) {
: kudu::client::sp::shared_ptr<kudu::client::KuduTable> table;
: kudu::Status s = client_->OpenTable(table_name, &table);
: if (!s.ok()) {
: LOG(ERROR) << "OpenTable failed: " << s.ToString();
: return "";
: }
: return table->id();
: }
please refactor this to return a status, and provide the table id as an out-parameter.
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_test_base.h
File src/kudu/master/rest_catalog_test_base.h:
http://gerrit.cloudera.org:8080/#/c/21823/30/src/kudu/master/rest_catalog_test_base.h@51
PS30, Line 51: std::string kTableName = "test_table";
make this a class member field, such that else in the tests you dont have to refer to this table by typing "test_table" rather than kTableName
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/spnego_rest_catalog-test.cc
File src/kudu/master/spnego_rest_catalog-test.cc:
http://gerrit.cloudera.org:8080/#/c/21823/31/src/kudu/master/spnego_rest_catalog-test.cc@205
PS31, Line 205: TEST_F(SpnegoRestCatalogTest, TestInvalidHeaders) {
: ASSERT_OK(CreateTestTable());
: string table_id = GetTableId("test_table");
: EasyCurl c;
: faststring buf;
: Status s = c.FetchURL(
: Substitute("http://$0/api/v1/tables", cluster_->mini_master()->bound_http_addr().ToString()),
: &buf,
: {"Authorization: I am an invalid header"});
: ASSERT_STR_CONTAINS(s.ToString(), "HTTP 500");
: s = c.FetchURL(Substitute("http://$0/api/v1/tables/$1",
: cluster_->mini_master()->bound_http_addr().ToString(),
: table_id),
: &buf,
: {"Authorization: I am an invalid header"});
: ASSERT_STR_CONTAINS(s.ToString(), "HTTP 500");
: s = c.FetchURL(
: Substitute("http://$0/api/v1/tables", cluster_->mini_master()->bound_http_addr().ToString()),
: &buf,
: {"Authorization: Negotiate I am also an invalid header"});
: ASSERT_STR_CONTAINS(s.ToString(), "HTTP 401");
: }
IMO is enough to use one endpoint and have one HTTP 500 and one HTTP 401 test in there.
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 31
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Mar 2025 17:09:56 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 29: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/1170/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 29
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 13 Mar 2025 12:16:04 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 17:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/728/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 17
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Dec 2024 10:43:45 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#17).
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
[WIP] KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: No Security Measures Yet. This implementation does not
currently include authentication or authorization.
Deleting, altering, or retrieving a table object requires the table's
ID. Operations using table names are not yet supported.
Testing: A new test file called rest_catalog-test.cc has been created to
validate these endpoints using an internal mini-cluster setup.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
A src/kudu/master/rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
6 files changed, 835 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/17
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 17
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608: REST API for Metadata Management
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#18).
Change subject: [WIP] KUDU-3608: REST API for Metadata Management
......................................................................
[WIP] KUDU-3608: REST API for Metadata Management
This commit introduces a REST API integrated directly into the Kudu
master web server to simplify metadata access and management. The API
operates under the /api path and focuses exclusively on
metadata-related operations. The following endpoints have been added:
- GET /api/v1/tables - Lists all tables by name and ID.
- POST /api/v1/tables - Creates a new table.
- GET /api/v1/tables/<table_id> - Retrieves table details.
- PUT /api/v1/tables/<table_id> - Updates table metadata.
- DELETE /api/v1/tables/<table_id> - Deletes a table.
HTTP Status Codes: Introduced additional HTTP status codes like 201
Created and 204 No Content for operations such as creating and deleting
tables.
Enhanced the web server's content-length check to include the PUT
method, allowing it to handle errors similar to POST.
Limitations: No Security Measures Yet. This implementation does not
currently include authentication or authorization.
Deleting, altering, or retrieving a table object requires the table's
ID. Operations using table names are not yet supported.
Testing: A new test file called rest_catalog-test.cc has been created to
validate these endpoints using an internal mini-cluster setup.
Usage:
When running Kudu, the endpoints are accessible via the /api/v1/ path
on the master web server. For POST and PUT endpoints, appropriate
data must be provided with the HTTP request (e.g., via curl).
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/CMakeLists.txt
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
A src/kudu/master/rest_catalog-test.cc
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
6 files changed, 801 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/18
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 18
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 10:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/667/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 10
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 06 Nov 2024 13:51:48 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 9:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/653/
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 9
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 04 Nov 2024 16:11:39 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: [WIP] KUDU-3608 REST API
......................................................................
Patch Set 9: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/653/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 9
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Mon, 04 Nov 2024 17:06:57 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Zoltan Chovan (Code Review)" <ge...@cloudera.org>.
Zoltan Chovan has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 33: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 33
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Mar 2025 12:55:38 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3608: REST API for Metadata Management
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21823 )
Change subject: KUDU-3608: REST API for Metadata Management
......................................................................
Patch Set 33: Verified+1
Build Successful
http://jenkins.kudu.apache.org/job/pre_commit/1206/ : SUCCESS
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 33
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Comment-Date: Wed, 26 Mar 2025 16:17:43 +0000
Gerrit-HasComments: No
[kudu-CR] [WIP] KUDU-3608 REST API
Posted by "Gabriella Lotz (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21823
to look at the new patch set (#11).
Change subject: [WIP] KUDU-3608 REST API
......................................................................
[WIP] KUDU-3608 REST API
To simplify metadata access and management, we've implemented
a REST API directly in the Kudu master web server. This API,
under the /api path, is built with PathHandlers and is limited
to metadata operations. The following endpoints are introduced:
- GET /api/v1/tables - Lists all tables by name and id
- POST /api/v1/tables - Creates a new table
- GET /api/v1/tables/<table_id> - Returns table details
- PUT /api/v1/tables/<table_id> - Alters table metadata
- DELETE /api/v1/tables/<table_id> - Deletes a table
Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
---
M src/kudu/master/master-test.cc
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
M src/kudu/server/webserver.cc
M src/kudu/util/web_callback_registry.h
5 files changed, 380 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21823/11
--
To view, visit http://gerrit.cloudera.org:8080/21823
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67f964c4f950edfde31772cafd5c3ed5d6b87413
Gerrit-Change-Number: 21823
Gerrit-PatchSet: 11
Gerrit-Owner: Gabriella Lotz <lo...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>