You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by li...@apache.org on 2022/05/31 03:52:44 UTC
[rocketmq-apis] branch main updated: Release protocol v2 (#42)
This is an automated email from the ASF dual-hosted git repository.
lizhanhui pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/rocketmq-apis.git
The following commit(s) were added to refs/heads/main by this push:
new 04768fb Release protocol v2 (#42)
04768fb is described below
commit 04768fb3f008c80227067eea3a90d462f516a4e5
Author: Zhanhui Li <li...@gmail.com>
AuthorDate: Tue May 31 11:52:40 2022 +0800
Release protocol v2 (#42)
* Upgrade version to 2.0
* Use stream telemetry and receive message response
* Remove unused messages
* Allow send batched messages
* Use plural forms for repeated fields
* Remove Nack API
* Move FIFO flag to ConsumerData
* Format
* Add change proposal
* Minor fix
* WIP: settings
* Revert ReceiveMessage RPC to unary type
* Remove HealthCheck unary RPC
* Refactor the response common (#15)
* Refactor the response common
* Update CI config
* Fix typo
* Rename some of the code
* Add 500 Internal Server Error
* Change code field name
* Add gateway timeout error code
* Add not-impelement code
* Fix typo
* Fix a series of issues (#16)
* Supply more code in status (#19)
* Add code: illegal message (#20)
* Add code: illegal message
* Remove redundant fields
* Support batch ack
* Support batch ack of messages
* Refactor client settings (#21)
* Remove unused RPCs and introduce server side stream to mitigate performance issue
* Add RPC-level status field
* Typo fix
* Refactor telemetry settings (#22)
* Add documents
* Add clang-format to format proto files
* Remove message_queue from SendMessageRequest
* Polish code (#23)
* Support customized backoff duration for cases of sending throttle or message consumption failure
* Add more comments for ReceiveMessageResponse (#24)
* Remove repeated support of messages in ReceiveMessageResponse as it is targeted use case is stream
* Simplify ReceiveMessageResponse struct
* Optimize exponential backoff
* Externalize backoff configuration for retries
* [ISSUE #26] Make interface orthogonal
* Add message_id in ChangeInvisibleDurationRequest for tracing
* Add optional for RetryPolicy
* Add message queue in RecoverOrphanedTransactionCommand (#29)
* Remove NackMessage (#28)
* [ISSUE #26] Add ClientType in HeartbeatRequest (#30)
* Include user agent details in settings
* Add status for TelemetryCommand
* remove status in ThreadStackTrace and VerifyMessageResult (#35)
* Add message_queue for VerifyMessageCommand (#36)
* Allow partial batch-send-failure in protocol perspsective
* Add status code: multiple results (#37)
* Add metric endpoints (#38)
* Rename field name of metrics collector endpoints
* Add response_delivery_time for ReceiveMessageResponse (#40)
* Fix typo (#41)
* Fix grammar issue and import
Co-authored-by: Aaron Ai <ya...@alibaba-inc.com>
Co-authored-by: zhouxiang <zh...@alibaba-inc.com>
Co-authored-by: lk <xd...@outlook.com>
---
.bazelversion | 2 +-
.clang-format | 5 +
.github/workflows/main.yml | 2 +-
.gitignore | 3 +-
BUILD.bazel | 4 +-
ChangeLog.md | 14 +
StyleGuide.md | 5 +
WORKSPACE | 18 +-
apache/rocketmq/v1/definition.proto | 351 ---------------------
apache/rocketmq/v1/service.proto | 551 ---------------------------------
apache/rocketmq/{v1 => v2}/admin.proto | 13 +-
apache/rocketmq/v2/definition.proto | 443 ++++++++++++++++++++++++++
apache/rocketmq/v2/service.proto | 445 ++++++++++++++++++++++++++
cpp/BUILD.bazel | 2 +-
deps.bzl | 4 +-
java/BUILD.bazel | 2 +-
16 files changed, 936 insertions(+), 928 deletions(-)
diff --git a/.bazelversion b/.bazelversion
index fae6e3d..af8c8ec 100644
--- a/.bazelversion
+++ b/.bazelversion
@@ -1 +1 @@
-4.2.1
+4.2.2
diff --git a/.clang-format b/.clang-format
new file mode 100644
index 0000000..1dd4061
--- /dev/null
+++ b/.clang-format
@@ -0,0 +1,5 @@
+Language: Proto
+ColumnLimit: 120
+SpacesInContainerLiterals: false
+AllowShortFunctionsOnASingleLine: None
+ReflowComments: false
\ No newline at end of file
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 3116429..feb9f2d 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -8,7 +8,7 @@ on:
push:
branches: [ main ]
pull_request:
- branches: [ main ]
+ branches: [ main, v2 ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
diff --git a/.gitignore b/.gitignore
index 20af2af..55dbf68 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,4 +1,5 @@
bazel-bin
bazel-out
bazel-rocketmq-apis
-bazel-testlogs
\ No newline at end of file
+bazel-testlogs
+.idea
\ No newline at end of file
diff --git a/BUILD.bazel b/BUILD.bazel
index f4c1f9b..a9d1cb6 100644
--- a/BUILD.bazel
+++ b/BUILD.bazel
@@ -1,8 +1,8 @@
package(default_visibility = ["//visibility:public"])
proto_library(
- name = "rocketmq_v1_proto",
- srcs = glob(["apache/rocketmq/v1/*.proto"]),
+ name = "rocketmq_v2_proto",
+ srcs = glob(["apache/rocketmq/v2/*.proto"]),
deps = [
"@com_google_protobuf//:empty_proto",
"@com_google_protobuf//:field_mask_proto",
diff --git a/ChangeLog.md b/ChangeLog.md
new file mode 100644
index 0000000..17e1a89
--- /dev/null
+++ b/ChangeLog.md
@@ -0,0 +1,14 @@
+Changes:
+1. v1 --> v2
+2. rpc ReceiveMessage(ReceiveMessageRequest) returns (stream ReceiveMessageResponse) {}
+3. Removed Nack RPC to honor ChangeInvisibleDuration RPC in consistent way;
+4. SendMessageRequest supports repeated messages; Corresponding response syncs with repeated SendReceipts
+5. rpc Telemetry(stream TelemetryCommand) returns (stream TelemetryCommand) {}
+6. Removed ProducerGroup;
+7. FIFO flags moved from HeartbeatRequest to ConsumerData
+8. Removed MessageModel enumeration as concept of broadcasting is totally deprecated;
+9. Enums field number = 0 is redefined to meet requirement [Each enum value should end with a semicolon, not a comma. Prefer prefixing enum values instead of surrounding them in an enclosing message. The zero value enum should have the suffix UNSPECIFIED.](https://developers.google.com/protocol-buffers/docs/style)
+10. Nested enumerations are externalized due to the same guide item as above.
+
+Remaining Issues:
+How server publishes conf and conf changes to clients
\ No newline at end of file
diff --git a/StyleGuide.md b/StyleGuide.md
new file mode 100644
index 0000000..540e4bf
--- /dev/null
+++ b/StyleGuide.md
@@ -0,0 +1,5 @@
+https://developers.google.com/protocol-buffers/docs/style
+https://dev.bostondynamics.com/docs/protos/style_guide
+
+In addition to rules described above, Field Number Range [0, 64] are **exclusively** reserved for RocketMQ.
+Third party entities are advised to used numbers greater than 64.
\ No newline at end of file
diff --git a/WORKSPACE b/WORKSPACE
index 1fb8bd0..df587d0 100644
--- a/WORKSPACE
+++ b/WORKSPACE
@@ -38,12 +38,12 @@ load("@rules_jvm_external//:defs.bzl", "maven_install")
maven_install(
artifacts = [
- "com.google.guava:guava:20.0",
- "com.google.protobuf:protobuf-java:3.12.0",
- "io.grpc:grpc-core:1.35.0",
- "io.grpc:grpc-protobuf:1.35.0",
- "io.grpc:grpc-stub:1.35.0",
- "io.grpc:grpc-api:1.35.0",
+ "com.google.guava:guava:31.1-jre",
+ "com.google.protobuf:protobuf-java:3.19.4",
+ "io.grpc:grpc-core:1.45.0",
+ "io.grpc:grpc-protobuf:1.45.0",
+ "io.grpc:grpc-stub:1.45.0",
+ "io.grpc:grpc-api:1.45.0",
"com.google.api.grpc:proto-google-common-protos:2.0.1",
"javax.annotation:javax.annotation-api:1.3.2"
],
@@ -83,9 +83,9 @@ git_repository(
http_archive(
name = "rules_proto_grpc",
- sha256 = "7954abbb6898830cd10ac9714fbcacf092299fda00ed2baf781172f545120419",
- strip_prefix = "rules_proto_grpc-3.1.1",
- urls = ["https://github.com/rules-proto-grpc/rules_proto_grpc/archive/3.1.1.tar.gz"],
+ sha256 = "507e38c8d95c7efa4f3b1c0595a8e8f139c885cb41a76cab7e20e4e67ae87731",
+ strip_prefix = "rules_proto_grpc-4.1.1",
+ urls = ["https://github.com/rules-proto-grpc/rules_proto_grpc/archive/4.1.1.tar.gz"],
)
load("@rules_proto_grpc//:repositories.bzl", "rules_proto_grpc_toolchains", "rules_proto_grpc_repos")
rules_proto_grpc_toolchains()
diff --git a/apache/rocketmq/v1/definition.proto b/apache/rocketmq/v1/definition.proto
deleted file mode 100644
index 723c1e3..0000000
--- a/apache/rocketmq/v1/definition.proto
+++ /dev/null
@@ -1,351 +0,0 @@
-// Licensed to the Apache Software Foundation (ASF) under one or more
-// contributor license agreements. See the NOTICE file distributed with
-// this work for additional information regarding copyright ownership.
-// The ASF licenses this file to You under the Apache License, Version 2.0
-// (the "License"); you may not use this file except in compliance with
-// the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-syntax = "proto3";
-
-import "google/protobuf/timestamp.proto";
-import "google/protobuf/duration.proto";
-
-package apache.rocketmq.v1;
-
-option csharp_namespace = "Apache.Rocketmq.V1";
-option java_multiple_files = true;
-option java_package = "apache.rocketmq.v1";
-option java_generate_equals_and_hash = true;
-option java_string_check_utf8 = true;
-option java_outer_classname = "MQDomain";
-
-enum Permission {
- NONE = 0;
- READ = 1;
- WRITE = 2;
- READ_WRITE = 3;
-
- reserved 4 to 64;
-}
-
-enum FilterType {
- TAG = 0;
- SQL = 1;
-
- reserved 2 to 64;
-}
-
-message FilterExpression {
- FilterType type = 1;
- string expression = 2;
-
- reserved 3 to 64;
-}
-
-// Dead lettering is done on a best effort basis. The same message might be
-// dead lettered multiple times.
-//
-// If validation on any of the fields fails at subscription creation/update,
-// the create/update subscription request will fail.
-message DeadLetterPolicy {
- // The maximum number of delivery attempts for any message.
- //
- // This field will be honored on a best effort basis.
- //
- // If this parameter is 0, a default value of 16 is used.
- int32 max_delivery_attempts = 1;
-
- reserved 2 to 64;
-}
-
-message Resource {
- string resource_namespace = 1;
-
- // Resource name identifier, which remains unique within the abstract resource
- // namespace.
- string name = 2;
-
- reserved 3 to 64;
-}
-
-enum ConsumeModel {
- CLUSTERING = 0;
- BROADCASTING = 1;
-
- reserved 2 to 64;
-}
-
-message ProducerData {
- Resource group = 1;
-
- reserved 2 to 64;
-}
-
-enum ConsumePolicy {
- RESUME = 0;
- PLAYBACK = 1;
- DISCARD = 2;
- TARGET_TIMESTAMP = 3;
-
- reserved 4 to 64;
-}
-
-enum ConsumeMessageType {
- ACTIVE = 0;
- PASSIVE = 1;
-
- reserved 2 to 64;
-}
-
-message ConsumerData {
- Resource group = 1;
-
- repeated SubscriptionEntry subscriptions = 2;
-
- ConsumeModel consume_model = 3;
-
- ConsumePolicy consume_policy = 4;
-
- DeadLetterPolicy dead_letter_policy = 5;
-
- ConsumeMessageType consume_type = 6;
-
- reserved 7 to 64;
-}
-
-message SubscriptionEntry {
- Resource topic = 1;
- FilterExpression expression = 2;
-
- reserved 3 to 64;
-}
-
-enum AddressScheme {
- IPv4 = 0;
- IPv6 = 1;
- DOMAIN_NAME = 2;
-
- reserved 3 to 64;
-}
-
-message Address {
- string host = 1;
- int32 port = 2;
-
- reserved 3 to 64;
-}
-
-message Endpoints {
- AddressScheme scheme = 1;
- repeated Address addresses = 2;
-
- reserved 3 to 64;
-}
-
-message Broker {
- // Name of the broker
- string name = 1;
-
- // Broker index. Canonically, index = 0 implies that the broker is playing
- // leader role while brokers with index > 0 play follower role.
- int32 id = 2;
-
- // Address of the broker, complying with the following scheme
- // 1. dns:[//authority/]host[:port]
- // 2. ipv4:address[:port][,address[:port],...] – IPv4 addresses
- // 3. ipv6:address[:port][,address[:port],...] – IPv6 addresses
- Endpoints endpoints = 3;
-
- reserved 4 to 64;
-}
-
-message Partition {
- Resource topic = 1;
- int32 id = 2;
- Permission permission = 3;
- Broker broker = 4;
- repeated MessageType accept_message_types = 5;
-
- reserved 6 to 64;
-}
-
-enum MessageType {
- NORMAL = 0;
-
- // Sequenced message
- FIFO = 1;
-
- // Messages that are delivered after the specified duration.
- DELAY = 2;
-
- // Messages that are transactional. Only committed messages are delivered to
- // subscribers.
- TRANSACTION = 3;
-
- reserved 4 to 64;
-}
-
-enum DigestType {
- // CRC algorithm achieves goal of detecting random data error with lowest
- // computation overhead.
- CRC32 = 0;
-
- // MD5 algorithm achieves good balance between collision rate and computation
- // overhead.
- MD5 = 1;
-
- // SHA-family has substantially fewer collision with fair amount of
- // computation.
- SHA1 = 2;
-
- reserved 3 to 64;
-}
-
-// When publishing messages to or subscribing messages from brokers, clients
-// shall include or validate digests of message body to ensure data integrity.
-//
-// For message publishment, when an invalid digest were detected, brokers need
-// respond client with BAD_REQUEST.
-//
-// For messags subscription, when an invalid digest were detected, consumers
-// need to handle this case according to message type:
-// 1) Standard messages should be negatively acknowledged instantly, causing
-// immediate re-delivery; 2) FIFO messages require special RPC, to re-fetch
-// previously acquired messages batch;
-//
-// Message consumption model also affects how invalid digest are handled. When
-// messages are consumed in broadcasting way,
-// TODO: define semantics of invalid-digest-when-broadcasting.
-message Digest {
- DigestType type = 1;
- string checksum = 2;
-
- reserved 3 to 64;
-}
-
-enum Encoding {
- IDENTITY = 0;
- GZIP = 1;
-
- reserved 2 to 64;
-}
-
-message SystemAttribute {
- // Tag
- string tag = 1;
-
- // Message keys
- repeated string keys = 2;
-
- // Message identifier, client-side generated, remains unique.
- // if message_id is empty, the send message request will be aborted with
- // status `INVALID_ARGUMENT`
- string message_id = 3;
-
- // Message body digest
- Digest body_digest = 4;
-
- // Message body encoding. Candidate options are identity, gzip, snappy etc.
- Encoding body_encoding = 5;
-
- // Message type, normal, FIFO or transactional.
- MessageType message_type = 6;
-
- // Message born time-point.
- google.protobuf.Timestamp born_timestamp = 7;
-
- // Message born host. Valid options are IPv4, IPv6 or client host domain name.
- string born_host = 8;
-
- // Time-point at which the message is stored in the broker.
- google.protobuf.Timestamp store_timestamp = 9;
-
- // The broker that stores this message. It may be name, IP or arbitrary
- // identifier that uniquely identify the broker.
- string store_host = 10;
-
- oneof timed_delivery {
- // Time-point at which broker delivers to clients.
- google.protobuf.Timestamp delivery_timestamp = 11;
-
- // Level-based delay strategy.
- int32 delay_level = 12;
- }
-
- // If a message is acquired by way of POP, this field holds the receipt.
- // Clients use the receipt to acknowledge or negatively acknowledge the
- // message.
- string receipt_handle = 13;
-
- // Partition identifier in which a message is physically stored.
- int32 partition_id = 14;
-
- // Partition offset at which a message is stored.
- int64 partition_offset = 15;
-
- // Period of time servers would remain invisible once a message is acquired.
- google.protobuf.Duration invisible_period = 16;
-
- // Business code may failed to process messages for the moment. Hence, clients
- // may request servers to deliver them again using certain back-off strategy,
- // the attempt is 1 not 0 if message is delivered first time.
- int32 delivery_attempt = 17;
-
- // Message producer load-balance group if applicable.
- Resource producer_group = 18;
-
- string message_group = 19;
-
- // Trace context.
- string trace_context = 20;
-
- // Delay time of first recover orphaned transaction request from server.
- google.protobuf.Duration orphaned_transaction_recovery_period = 21;
-
- reserved 22 to 64;
-}
-
-message Message {
-
- Resource topic = 1;
-
- // User defined key-value pairs.
- // If user_attribute contains the reserved keys by RocketMQ,
- // the send message request will be aborted with status `INVALID_ARGUMENT`.
- // See below links for the reserved keys
- // https://github.com/apache/rocketmq/blob/master/common/src/main/java/org/apache/rocketmq/common/message/MessageConst.java#L58
- map<string, string> user_attribute = 2;
-
- SystemAttribute system_attribute = 3;
-
- bytes body = 4;
-
- reserved 5 to 64;
-}
-
-message Assignment {
- Partition Partition = 1;
-
- reserved 2 to 64;
-}
-
-enum QueryOffsetPolicy {
- // Use this option if client wishes to playback all existing messages.
- BEGINNING = 0;
-
- // Use this option if client wishes to skip all existing messages.
- END = 1;
-
- // Use this option if time-based seek is targeted.
- TIME_POINT = 2;
-
- reserved 3 to 64;
-}
\ No newline at end of file
diff --git a/apache/rocketmq/v1/service.proto b/apache/rocketmq/v1/service.proto
deleted file mode 100644
index 185eddf..0000000
--- a/apache/rocketmq/v1/service.proto
+++ /dev/null
@@ -1,551 +0,0 @@
-// Licensed to the Apache Software Foundation (ASF) under one or more
-// contributor license agreements. See the NOTICE file distributed with
-// this work for additional information regarding copyright ownership.
-// The ASF licenses this file to You under the Apache License, Version 2.0
-// (the "License"); you may not use this file except in compliance with
-// the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-syntax = "proto3";
-
-import "google/protobuf/duration.proto";
-import "google/protobuf/timestamp.proto";
-import "google/rpc/error_details.proto";
-import "google/rpc/status.proto";
-
-import "apache/rocketmq/v1/definition.proto";
-
-package apache.rocketmq.v1;
-
-option csharp_namespace = "Apache.Rocketmq.V1";
-option java_multiple_files = true;
-option java_package = "apache.rocketmq.v1";
-option java_generate_equals_and_hash = true;
-option java_string_check_utf8 = true;
-option java_outer_classname = "MQService";
-
-message ResponseCommon {
- google.rpc.Status status = 1;
- google.rpc.RequestInfo request_info = 2;
- google.rpc.Help help = 3;
- google.rpc.RetryInfo retry_info = 4;
- google.rpc.DebugInfo debug_info = 5;
- google.rpc.ErrorInfo error_info = 6;
-
- reserved 7 to 64;
-}
-
-// Topics are destination of messages to publish to or subscribe from. Similar
-// to domain names, they will be addressable after resolution through the
-// provided access point.
-//
-// Access points are usually the addresses of name servers, which fulfill
-// service discovery, load-balancing and other auxiliary services. Name servers
-// receive periodic heartbeats from affiliate brokers and erase those which
-// failed to maintain alive status.
-//
-// Name servers answer queries of QueryRouteRequest, responding clients with
-// addressable partitions, which they may directly publish messages to or
-// subscribe messages from.
-//
-// QueryRouteRequest shall include source endpoints, aka, configured
-// access-point, which annotates tenant-id, instance-id or other
-// vendor-specific settings. Purpose-built name servers may respond customized
-// results based on these particular requirements.
-message QueryRouteRequest {
- Resource topic = 1;
-
- Endpoints endpoints = 2;
-
- reserved 3 to 64;
-}
-
-message QueryRouteResponse {
- ResponseCommon common = 1;
-
- repeated Partition partitions = 2;
-
- reserved 3 to 64;
-}
-
-message SendMessageRequest {
- Message message = 1;
- Partition partition = 2;
-
- reserved 3 to 64;
-}
-
-message SendMessageResponse {
- ResponseCommon common = 1;
- string message_id = 2;
- string transaction_id = 3;
-
- reserved 4 to 64;
-}
-
-message QueryAssignmentRequest {
- Resource topic = 1;
- Resource group = 2;
- string client_id = 3;
-
- // Service access point
- Endpoints endpoints = 4;
-
- reserved 5 to 64;
-}
-
-message QueryAssignmentResponse {
- ResponseCommon common = 1;
- repeated Assignment assignments = 2;
-
- reserved 3 to 64;
-}
-
-message ReceiveMessageRequest {
- Resource group = 1;
- string client_id = 2;
- Partition partition = 3;
- FilterExpression filter_expression = 4;
- ConsumePolicy consume_policy = 5;
- google.protobuf.Timestamp initialization_timestamp = 6;
- int32 batch_size = 7;
- google.protobuf.Duration invisible_duration = 8;
- google.protobuf.Duration await_time = 9;
- bool fifo_flag = 10;
-
- reserved 11 to 64;
-}
-
-message ReceiveMessageResponse {
- ResponseCommon common = 1;
- repeated Message messages = 2;
- google.protobuf.Timestamp delivery_timestamp = 3;
- google.protobuf.Duration invisible_duration = 4;
-
- reserved 5 to 64;
-}
-
-message AckMessageRequest {
- Resource group = 1;
- Resource topic = 2;
- string client_id = 3;
- oneof handle {
- string receipt_handle = 4;
- int64 offset = 5;
- }
- string message_id = 6;
-
- reserved 7 to 64;
-}
-
-message AckMessageResponse {
- ResponseCommon common = 1;
-
- reserved 2 to 64;
-}
-
-message NackMessageRequest {
- Resource group = 1;
- Resource topic = 2;
- string client_id = 3;
- string receipt_handle = 4;
- string message_id = 5;
- int32 delivery_attempt = 6;
- int32 max_delivery_attempts = 7;
-
- reserved 8 to 64;
-}
-
-message NackMessageResponse {
- ResponseCommon common = 1;
-
- reserved 2 to 64;
-}
-
-message ForwardMessageToDeadLetterQueueRequest {
- Resource group = 1;
- Resource topic = 2;
- string client_id = 3;
- string receipt_handle = 4;
- string message_id = 5;
- int32 delivery_attempt = 6;
- int32 max_delivery_attempts = 7;
-
- reserved 8 to 64;
-}
-
-message ForwardMessageToDeadLetterQueueResponse {
- ResponseCommon common = 1;
-
- reserved 2 to 64;
-}
-
-message HeartbeatRequest {
- string client_id = 1;
- oneof client_data {
- ProducerData producer_data = 2;
- ConsumerData consumer_data = 3;
- }
- bool fifo_flag = 4;
-
- reserved 5 to 64;
-}
-
-message HeartbeatResponse {
- ResponseCommon common = 1;
-
- reserved 2 to 64;
-}
-
-message HealthCheckRequest {
- Resource group = 1;
- string client_host = 2;
-
- reserved 3 to 64;
-}
-
-message HealthCheckResponse {
- ResponseCommon common = 1;
-
- reserved 2 to 64;
-}
-
-message EndTransactionRequest {
- Resource group = 1;
- string message_id = 2;
- string transaction_id = 3;
- enum TransactionResolution {
- COMMIT = 0;
- ROLLBACK = 1;
- }
- TransactionResolution resolution = 4;
- enum Source {
- CLIENT = 0;
- SERVER_CHECK = 1;
- }
- Source source = 5;
- string trace_context = 6;
-
- reserved 7 to 64;
-}
-
-message EndTransactionResponse {
- ResponseCommon common = 1;
-
- reserved 2 to 64;
-}
-
-message QueryOffsetRequest {
- Partition partition = 1;
- QueryOffsetPolicy policy = 2;
- google.protobuf.Timestamp time_point = 3;
-
- reserved 4 to 64;
-}
-
-message QueryOffsetResponse {
- ResponseCommon common = 1;
- int64 offset = 2;
-
- reserved 3 to 64;
-}
-
-message PullMessageRequest {
- Resource group = 1;
- Partition partition = 2;
- int64 offset = 3;
- int32 batch_size = 4;
- google.protobuf.Duration await_time = 5;
- FilterExpression filter_expression = 6;
- string client_id = 7;
-
- reserved 8 to 64;
-}
-
-message PullMessageResponse {
- ResponseCommon common = 1;
- int64 min_offset = 2;
- int64 next_offset = 3;
- int64 max_offset = 4;
- repeated Message messages = 5;
-
- reserved 6 to 64;
-}
-
-message NoopCommand { reserved 1 to 64; }
-
-message PrintThreadStackTraceCommand {
- string command_id = 1;
-
- reserved 2 to 64;
-}
-
-message ReportThreadStackTraceRequest {
- string command_id = 1;
- string thread_stack_trace = 2;
-
- reserved 3 to 64;
-}
-
-message ReportThreadStackTraceResponse {
- ResponseCommon common = 1;
-
- reserved 2 to 64;
-}
-
-message VerifyMessageConsumptionCommand {
- string command_id = 1;
- Message message = 2;
-
- reserved 3 to 64;
-}
-
-message ReportMessageConsumptionResultRequest {
- string command_id = 1;
-
- // 1. Return `INVALID_ARGUMENT` if message is corrupted.
- // 2. Return `INTERNAL` if failed to consume message.
- // 3. Return `OK` if success.
- google.rpc.Status status = 2;
-
- reserved 3 to 64;
-}
-
-message ReportMessageConsumptionResultResponse {
- ResponseCommon common = 1;
-
- reserved 2 to 64;
-}
-
-message RecoverOrphanedTransactionCommand {
- Message orphaned_transactional_message = 1;
- string transaction_id = 2;
-
- reserved 3 to 64;
-}
-
-message PollCommandRequest {
- string client_id = 1;
- repeated Resource topics = 2;
- oneof group {
- Resource producer_group = 3;
- Resource consumer_group = 4;
- }
-
- reserved 5 to 64;
-}
-
-message PollCommandResponse {
- oneof type {
- // Default command when no new command need to be delivered.
- NoopCommand noop_command = 1;
- // Request client to print thread stack trace.
- PrintThreadStackTraceCommand print_thread_stack_trace_command = 2;
- // Request client to verify the consumption of the appointed message.
- VerifyMessageConsumptionCommand verify_message_consumption_command = 3;
- // Request client to recover the orphaned transaction message.
- RecoverOrphanedTransactionCommand recover_orphaned_transaction_command = 4;
- }
-
- reserved 5 to 64;
-}
-
-message NotifyClientTerminationRequest {
- oneof group {
- Resource producer_group = 1;
- Resource consumer_group = 2;
- }
- string client_id = 3;
-
- reserved 4 to 64;
-}
-
-message NotifyClientTerminationResponse {
- ResponseCommon common = 1;
-
- reserved 2 to 64;
-}
-
-message ChangeInvisibleDurationRequest {
- Resource group = 1;
- Resource topic = 2;
-
- // Unique receipt handle to identify message to change
- string receipt_handle = 3;
-
- // New invisible duration
- google.protobuf.Duration invisible_duration = 4;
- reserved 5 to 64;
-}
-
-message ChangeInvisibleDurationResponse {
- ResponseCommon common = 1;
-
- // Server may generate a new receipt handle for the message.
- string receipt_handle = 2;
-
- reserved 3 to 64;
-}
-
-// For all the RPCs in MessagingService, the following error handling policies
-// apply:
-//
-// If the request doesn't bear a valid authentication credential, return a
-// response with common.status.code == `UNAUTHENTICATED`. If the authenticated
-// user is not granted with sufficient permission to execute the requested
-// operation, return a response with common.status.code == `PERMISSION_DENIED`.
-// If the per-user-resource-based quota is exhausted, return a response with
-// common.status.code == `RESOURCE_EXHAUSTED`. If any unexpected server-side
-// errors raise, return a response with common.status.code == `INTERNAL`.
-service MessagingService {
-
- // Queries the route entries of the requested topic in the perspective of the
- // given endpoints. On success, servers should return a collection of
- // addressable partitions. Note servers may return customized route entries
- // based on endpoints provided.
- //
- // If the requested topic doesn't exist, returns `NOT_FOUND`.
- // If the specific endpoints is emtpy, returns `INVALID_ARGUMENT`.
- rpc QueryRoute(QueryRouteRequest) returns (QueryRouteResponse) {}
-
- // Producer or consumer sends HeartbeatRequest to servers periodically to
- // keep-alive. Additionally, it also reports client-side configuration,
- // including topic subscription, load-balancing group name, etc.
- //
- // Returns `OK` if success.
- //
- // If a client specifies a language that is not yet supported by servers,
- // returns `INVALID_ARGUMENT`
- rpc Heartbeat(HeartbeatRequest) returns (HeartbeatResponse) {}
-
- // Checks the health status of message server, returns `OK` if services are
- // online and serving. Clients may use this RPC to detect availability of
- // messaging service, and take isolation actions when necessary.
- rpc HealthCheck(HealthCheckRequest) returns (HealthCheckResponse) {}
-
- // Delivers messages to brokers.
- // Clients may further:
- // 1. Refine a message destination to topic partition which fulfills parts of
- // FIFO semantic;
- // 2. Flag a message as transactional, which keeps it invisible to consumers
- // until it commits;
- // 3. Time a message, making it invisible to consumers till specified
- // time-point;
- // 4. And more...
- //
- // Returns message-id or transaction-id with status `OK` on success.
- //
- // If the destination topic doesn't exist, returns `NOT_FOUND`.
- rpc SendMessage(SendMessageRequest) returns (SendMessageResponse) {}
-
- // Queries the assigned partition route info of a topic for current consumer,
- // the returned assignment result is decided by server-side load balancer.
- //
- // If the corresponding topic doesn't exist, returns `NOT_FOUND`.
- // If the specific endpoints is empty, returns `INVALID_ARGUMENT`.
- rpc QueryAssignment(QueryAssignmentRequest)
- returns (QueryAssignmentResponse) {}
-
- // Receives messages from the server in batch manner, returns a set of
- // messages if success. The received messages should be acked or nacked after
- // processed.
- //
- // If the pending concurrent receive requests exceed the quota of the given
- // consumer group, returns `UNAVAILABLE`. If the upstream store server hangs,
- // return `DEADLINE_EXCEEDED` in a timely manner. If the corresponding topic
- // or consumer group doesn't exist, returns `NOT_FOUND`. If there is no new
- // message in the specific topic, returns `OK` with an empty message set.
- // Please note that client may suffer from false empty responses.
- rpc ReceiveMessage(ReceiveMessageRequest) returns (ReceiveMessageResponse) {}
-
- // Acknowledges the message associated with the `receipt_handle` or `offset`
- // in the `AckMessageRequest`, it means the message has been successfully
- // processed. Returns `OK` if the message server remove the relevant message
- // successfully.
- //
- // If the given receipt_handle is illegal or out of date, returns
- // `INVALID_ARGUMENT`.
- rpc AckMessage(AckMessageRequest) returns (AckMessageResponse) {}
-
- // Signals that the message has not been successfully processed. The message
- // server should resend the message follow the retry policy defined at
- // server-side.
- //
- // If the corresponding topic or consumer group doesn't exist, returns
- // `NOT_FOUND`.
- rpc NackMessage(NackMessageRequest) returns (NackMessageResponse) {}
-
- // Forwards one message to dead letter queue if the DeadLetterPolicy is
- // triggered by this message at client-side, return `OK` if success.
- rpc ForwardMessageToDeadLetterQueue(ForwardMessageToDeadLetterQueueRequest)
- returns (ForwardMessageToDeadLetterQueueResponse) {}
-
- // Commits or rollback one transactional message.
- rpc EndTransaction(EndTransactionRequest) returns (EndTransactionResponse) {}
-
- // Queries the offset of the specific partition, returns the offset with `OK`
- // if success. The message server should maintain a numerical offset for each
- // message in a partition.
- rpc QueryOffset(QueryOffsetRequest) returns (QueryOffsetResponse) {}
-
- // Pulls messages from the specific partition, returns a set of messages with
- // next pull offset. The pulled messages can't be acked or nacked, while the
- // client is responsible for manage offsets for consumer, typically update
- // consume offset to local memory or a third-party storage service.
- //
- // If the pending concurrent receive requests exceed the quota of the given
- // consumer group, returns `UNAVAILABLE`. If the upstream store server hangs,
- // return `DEADLINE_EXCEEDED` in a timely manner. If the corresponding topic
- // or consumer group doesn't exist, returns `NOT_FOUND`. If there is no new
- // message in the specific topic, returns `OK` with an empty message set.
- // Please note that client may suffer from false empty responses.
- rpc PullMessage(PullMessageRequest) returns (PullMessageResponse) {}
-
- // Multiplexing RPC(s) for various polling requests, which issue different
- // commands to client.
- //
- // Sometimes client may need to receive and process the command from server.
- // To prevent the complexity of streaming RPC(s), a unary RPC using
- // long-polling is another solution.
- //
- // To mark the request-response of corresponding command, `command_id` in
- // message is recorded in the subsequent RPC(s). For example, after receiving
- // command of printing thread stack trace, client would send
- // `ReportMessageConsumptionResultRequest` to server, which contain both of
- // the stack trace and `command_id`.
- //
- // At same time, `NoopCommand` is delivered from server when no new command is
- // needed, it is essential for client to maintain the ping-pong.
- //
- rpc PollCommand(PollCommandRequest) returns (PollCommandResponse) {}
-
- // After receiving the corresponding polling command, the thread stack trace
- // is reported to the server.
- rpc ReportThreadStackTrace(ReportThreadStackTraceRequest)
- returns (ReportThreadStackTraceResponse) {}
-
- // After receiving the corresponding polling command, the consumption result
- // of appointed message is reported to the server.
- rpc ReportMessageConsumptionResult(ReportMessageConsumptionResultRequest)
- returns (ReportMessageConsumptionResultResponse) {}
-
- // Notify the server that the client is terminated.
- rpc NotifyClientTermination(NotifyClientTerminationRequest)
- returns (NotifyClientTerminationResponse) {}
-
- // Once a message is retrieved from consume queue on behalf of the group, it
- // will be kept invisible to other clients of the same group for a period of
- // time. The message is supposed to be processed within the invisible
- // duration. If the client, which is in charge of the invisible message, is
- // not capable of processing the message timely, it may use
- // ChangeInvisibleDuration to lengthen invisible duration.
- rpc ChangeInvisibleDuration(ChangeInvisibleDurationRequest)
- returns (ChangeInvisibleDurationResponse) {}
-}
\ No newline at end of file
diff --git a/apache/rocketmq/v1/admin.proto b/apache/rocketmq/v2/admin.proto
similarity index 86%
rename from apache/rocketmq/v1/admin.proto
rename to apache/rocketmq/v2/admin.proto
index 283ca82..7dbb702 100644
--- a/apache/rocketmq/v1/admin.proto
+++ b/apache/rocketmq/v2/admin.proto
@@ -15,12 +15,12 @@
syntax = "proto3";
-package apache.rocketmq.v1;
+package apache.rocketmq.v2;
option cc_enable_arenas = true;
-option csharp_namespace = "Apache.Rocketmq.V1";
+option csharp_namespace = "Apache.Rocketmq.V2";
option java_multiple_files = true;
-option java_package = "apache.rocketmq.v1";
+option java_package = "apache.rocketmq.v2";
option java_generate_equals_and_hash = true;
option java_string_check_utf8 = true;
option java_outer_classname = "MQAdmin";
@@ -36,11 +36,8 @@ message ChangeLogLevelRequest {
Level level = 1;
}
-message ChangeLogLevelResponse {
- string remark = 1;
-}
+message ChangeLogLevelResponse { string remark = 1; }
service Admin {
- rpc ChangeLogLevel(ChangeLogLevelRequest) returns (ChangeLogLevelResponse) {
- }
+ rpc ChangeLogLevel(ChangeLogLevelRequest) returns (ChangeLogLevelResponse) {}
}
\ No newline at end of file
diff --git a/apache/rocketmq/v2/definition.proto b/apache/rocketmq/v2/definition.proto
new file mode 100644
index 0000000..21a6321
--- /dev/null
+++ b/apache/rocketmq/v2/definition.proto
@@ -0,0 +1,443 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+syntax = "proto3";
+
+import "google/protobuf/timestamp.proto";
+import "google/protobuf/duration.proto";
+
+package apache.rocketmq.v2;
+
+option csharp_namespace = "Apache.Rocketmq.V2";
+option java_multiple_files = true;
+option java_package = "apache.rocketmq.v2";
+option java_generate_equals_and_hash = true;
+option java_string_check_utf8 = true;
+option java_outer_classname = "MQDomain";
+
+enum TransactionResolution {
+ TRANSACTION_RESOLUTION_UNSPECIFIED = 0;
+ COMMIT = 1;
+ ROLLBACK = 2;
+}
+
+enum TransactionSource {
+ SOURCE_UNSPECIFIED = 0;
+ SOURCE_CLIENT = 1;
+ SOURCE_SERVER_CHECK = 2;
+}
+
+enum Permission {
+ PERMISSION_UNSPECIFIED = 0;
+ NONE = 1;
+ READ = 2;
+ WRITE = 3;
+ READ_WRITE = 4;
+}
+
+enum FilterType {
+ FILTER_TYPE_UNSPECIFIED = 0;
+ TAG = 1;
+ SQL = 2;
+}
+
+message FilterExpression {
+ FilterType type = 1;
+ string expression = 2;
+}
+
+message RetryPolicy {
+ int32 max_attempts = 1;
+ oneof strategy {
+ ExponentialBackoff exponential_backoff = 2;
+ CustomizedBackoff customized_backoff = 3;
+ }
+}
+
+// https://en.wikipedia.org/wiki/Exponential_backoff
+message ExponentialBackoff {
+ google.protobuf.Duration initial = 1;
+ google.protobuf.Duration max = 2;
+ float multiplier = 3;
+}
+
+message CustomizedBackoff {
+ // To support classic backoff strategy which is arbitary defined by end users.
+ // Typical values are: `1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h`
+ repeated google.protobuf.Duration next = 1;
+}
+
+message Resource {
+ string resource_namespace = 1;
+
+ // Resource name identifier, which remains unique within the abstract resource
+ // namespace.
+ string name = 2;
+}
+
+message SubscriptionEntry {
+ Resource topic = 1;
+ FilterExpression expression = 2;
+}
+
+enum AddressScheme {
+ ADDRESS_SCHEME_UNSPECIFIED = 0;
+ IPv4 = 1;
+ IPv6 = 2;
+ DOMAIN_NAME = 3;
+}
+
+message Address {
+ string host = 1;
+ int32 port = 2;
+}
+
+message Endpoints {
+ AddressScheme scheme = 1;
+ repeated Address addresses = 2;
+}
+
+message Broker {
+ // Name of the broker
+ string name = 1;
+
+ // Broker index. Canonically, index = 0 implies that the broker is playing
+ // leader role while brokers with index > 0 play follower role.
+ int32 id = 2;
+
+ // Address of the broker, complying with the following scheme
+ // 1. dns:[//authority/]host[:port]
+ // 2. ipv4:address[:port][,address[:port],...] – IPv4 addresses
+ // 3. ipv6:address[:port][,address[:port],...] – IPv6 addresses
+ Endpoints endpoints = 3;
+}
+
+message MessageQueue {
+ Resource topic = 1;
+ int32 id = 2;
+ Permission permission = 3;
+ Broker broker = 4;
+ repeated MessageType accept_message_types = 5;
+}
+
+enum MessageType {
+ MESSAGE_TYPE_UNSPECIFIED = 0;
+
+ NORMAL = 1;
+
+ // Sequenced message
+ FIFO = 2;
+
+ // Messages that are delivered after the specified duration.
+ DELAY = 3;
+
+ // Messages that are transactional. Only committed messages are delivered to
+ // subscribers.
+ TRANSACTION = 4;
+}
+
+enum DigestType {
+ DIGEST_TYPE_UNSPECIFIED = 0;
+
+ // CRC algorithm achieves goal of detecting random data error with lowest
+ // computation overhead.
+ CRC32 = 1;
+
+ // MD5 algorithm achieves good balance between collision rate and computation
+ // overhead.
+ MD5 = 2;
+
+ // SHA-family has substantially fewer collision with fair amount of
+ // computation.
+ SHA1 = 3;
+}
+
+// When publishing messages to or subscribing messages from brokers, clients
+// shall include or validate digests of message body to ensure data integrity.
+//
+// For message publishing, when an invalid digest were detected, brokers need
+// respond client with BAD_REQUEST.
+//
+// For messages subscription, when an invalid digest were detected, consumers
+// need to handle this case according to message type:
+// 1) Standard messages should be negatively acknowledged instantly, causing
+// immediate re-delivery; 2) FIFO messages require special RPC, to re-fetch
+// previously acquired messages batch;
+//
+// Message consumption model also affects how invalid digest are handled. When
+// messages are consumed in broadcasting way,
+// TODO: define semantics of invalid-digest-when-broadcasting.
+message Digest {
+ DigestType type = 1;
+ string checksum = 2;
+}
+
+enum ClientType {
+ CLIENT_TYPE_UNSPECIFIED = 0;
+ PRODUCER = 1;
+ PUSH_CONSUMER = 2;
+ SIMPLE_CONSUMER = 3;
+}
+
+enum Encoding {
+ ENCODING_UNSPECIFIED = 0;
+
+ IDENTITY = 1;
+
+ GZIP = 2;
+}
+
+message SystemProperties {
+ // Tag, which is optional.
+ optional string tag = 1;
+
+ // Message keys
+ repeated string keys = 2;
+
+ // Message identifier, client-side generated, remains unique.
+ // if message_id is empty, the send message request will be aborted with
+ // status `INVALID_ARGUMENT`
+ string message_id = 3;
+
+ // Message body digest
+ Digest body_digest = 4;
+
+ // Message body encoding. Candidate options are identity, gzip, snappy etc.
+ Encoding body_encoding = 5;
+
+ // Message type, normal, FIFO or transactional.
+ MessageType message_type = 6;
+
+ // Message born time-point.
+ google.protobuf.Timestamp born_timestamp = 7;
+
+ // Message born host. Valid options are IPv4, IPv6 or client host domain name.
+ string born_host = 8;
+
+ // Time-point at which the message is stored in the broker, which is absent
+ // for message publishing.
+ optional google.protobuf.Timestamp store_timestamp = 9;
+
+ // The broker that stores this message. It may be broker name, IP or arbitrary
+ // identifier that uniquely identify the server.
+ string store_host = 10;
+
+ // Time-point at which broker delivers to clients, which is optional.
+ optional google.protobuf.Timestamp delivery_timestamp = 11;
+
+ // If a message is acquired by way of POP, this field holds the receipt,
+ // which is absent for message publishing.
+ // Clients use the receipt to acknowledge or negatively acknowledge the
+ // message.
+ optional string receipt_handle = 12;
+
+ // Message queue identifier in which a message is physically stored.
+ int32 queue_id = 13;
+
+ // Message-queue offset at which a message is stored, which is absent for
+ // message publishing.
+ optional int64 queue_offset = 14;
+
+ // Period of time servers would remain invisible once a message is acquired.
+ optional google.protobuf.Duration invisible_duration = 15;
+
+ // Business code may failed to process messages for the moment. Hence, clients
+ // may request servers to deliver them again using certain back-off strategy,
+ // the attempt is 1 not 0 if message is delivered first time, and it is absent
+ // for message publishing.
+ optional int32 delivery_attempt = 16;
+
+ // Define the group name of message in the same topic, which is optional.
+ optional string message_group = 17;
+
+ // Trace context for each message, which is optional.
+ optional string trace_context = 18;
+
+ // If a transactional message stay unresolved for more than
+ // `transaction_orphan_threshold`, it would be regarded as an
+ // orphan. Servers that manages orphan messages would pick up
+ // a capable publisher to resolve
+ optional google.protobuf.Duration orphaned_transaction_recovery_duration = 19;
+}
+
+message Message {
+
+ Resource topic = 1;
+
+ // User defined key-value pairs.
+ // If user_properties contain the reserved keys by RocketMQ,
+ // the send message request will be aborted with status `INVALID_ARGUMENT`.
+ // See below links for the reserved keys
+ // https://github.com/apache/rocketmq/blob/master/common/src/main/java/org/apache/rocketmq/common/message/MessageConst.java#L58
+ map<string, string> user_properties = 2;
+
+ SystemProperties system_properties = 3;
+
+ bytes body = 4;
+}
+
+message Assignment { MessageQueue message_queue = 1; }
+
+enum Code {
+ // Success.
+ OK = 0;
+ // Format of access point is illegal.
+ ILLEGAL_ACCESS_POINT = 1;
+ // Format of topic is illegal.
+ ILLEGAL_TOPIC = 2;
+ // Format of consumer group is illegal.
+ ILLEGAL_CONSUMER_GROUP = 3;
+ // Format of message tag is illegal.
+ ILLEGAL_MESSAGE_TAG = 4;
+ // Format of message key is illegal.
+ ILLEGAL_MESSAGE_KEY = 5;
+ // Size of message keys exceeds the threshold.
+ MESSAGE_KEYS_TOO_LARGE = 6;
+ // Format of message group is illegal.
+ ILLEGAL_MESSAGE_GROUP = 7;
+ // Format of message property key is illegal.
+ ILLEGAL_MESSAGE_PROPERTY_KEY = 8;
+ // Message properties total size exceeds the threshold.
+ MESSAGE_PROPERTIES_TOO_LARGE = 9;
+ // Message body size exceeds the threshold.
+ MESSAGE_BODY_TOO_LARGE = 10;
+
+ // User does not have the permission to operate.
+ // See https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403
+ FORBIDDEN = 403;
+
+ // Code indicates that the client request has not been completed
+ // because it lacks valid authentication credentials for the
+ // requested resource.
+ // See https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401
+ UNAUTHORIZED = 401;
+
+ // Topic resource does not exist.
+ TOPIC_NOT_FOUND = 13;
+
+ // Consumer group resource does not exist.
+ CONSUMER_GROUP_NOT_FOUND = 14;
+
+ // Not allowed to verify message. Chances are that you are verifying
+ // a FIFO message, as is violating FIFO semantics.
+ VERIFY_MESSAGE_FORBIDDEN = 15;
+
+ // Failed to consume message.
+ FAILED_TO_CONSUME_MESSAGE = 16;
+
+ // Message is corrupted.
+ MESSAGE_CORRUPTED = 17;
+
+ // Too many requests are made in short period of duration.
+ // Requests are throttled.
+ TOO_MANY_REQUESTS = 18;
+
+ // Expired receipt-handle is used when trying to acknowledge or change
+ // invisible duration of a message
+ RECEIPT_HANDLE_EXPIRED = 19;
+
+ // Message property is not match the message type.
+ MESSAGE_PROPERTY_DOES_NOT_MATCH_MESSAGE_TYPE = 20;
+
+ // Format of message id is illegal.
+ ILLEGAL_MESSAGE_ID = 21;
+
+ // Transaction id is invalid.
+ INVALID_TRANSACTION_ID = 22;
+
+ // Format of filter expression is illegal.
+ ILLEGAL_FILTER_EXPRESSION = 23;
+
+ // Receipt handle of message is invalid.
+ INVALID_RECEIPT_HANDLE = 24;
+
+ // Message persistence timeout.
+ MASTER_PERSISTENCE_TIMEOUT = 25;
+
+ // Slave persistence timeout.
+ SLAVE_PERSISTENCE_TIMEOUT = 26;
+
+ // The HA-mechanism is not working now.
+ HA_NOT_AVAILABLE = 27;
+
+ // Operation is not allowed in current version.
+ VERSION_UNSUPPORTED = 28;
+
+ // Message not found from server.
+ MESSAGE_NOT_FOUND = 29;
+
+ // Message offset is illegal.
+ ILLEGAL_MESSAGE_OFFSET = 30;
+
+ // Illegal message is for the sake of backward compatibility. In most case,
+ // more definitive code is better, e.g. `ILLEGAL_MESSAGE_TAG`.
+ ILLEGAL_MESSAGE = 31;
+
+ // Client type could not be recognized.
+ UNRECOGNIZED_CLIENT_TYPE = 32;
+
+ // Return different results for entries in composite request.
+ MULTIPLE_RESULTS = 33;
+
+ // Code indicates that the server encountered an unexpected condition
+ // that prevented it from fulfilling the request.
+ // This error response is a generic "catch-all" response.
+ // Usually, this indicates the server cannot find a better alternative
+ // error code to response. Sometimes, server administrators log error
+ // responses like the 500 status code with more details about the request
+ // to prevent the error from happening again in the future.
+ //
+ // See https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
+ INTERNAL_SERVER_ERROR = 500;
+
+ // Code means that the server or client does not support the functionality
+ // required to fulfill the request.
+ NOT_IMPLEMENTED = 501;
+
+ // Code indicates that the server, while acting as a gateway or proxy,
+ // did not get a response in time from the upstream server that
+ // it needed in order to complete the request.
+ // See https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/504
+ GATEWAY_TIMEOUT = 504;
+}
+
+message Status {
+ Code code = 1;
+ string message = 2;
+}
+
+enum Language {
+ LANGUAGE_UNSPECIFIED = 0;
+ JAVA = 1;
+ CPP = 2;
+ DOT_NET = 3;
+ GOLANG = 4;
+ RUST = 5;
+}
+
+// User Agent
+message UA {
+ // SDK language
+ Language language = 1;
+
+ // SDK version
+ string version = 2;
+
+ // Platform details, including OS name, version, arch etc.
+ string platform = 3;
+
+ // Hostname of the node
+ string hostname = 4;
+}
\ No newline at end of file
diff --git a/apache/rocketmq/v2/service.proto b/apache/rocketmq/v2/service.proto
new file mode 100644
index 0000000..c7ce2e9
--- /dev/null
+++ b/apache/rocketmq/v2/service.proto
@@ -0,0 +1,445 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+syntax = "proto3";
+
+import "google/protobuf/duration.proto";
+import "google/protobuf/timestamp.proto";
+
+import "apache/rocketmq/v2/definition.proto";
+
+package apache.rocketmq.v2;
+
+option csharp_namespace = "Apache.Rocketmq.V2";
+option java_multiple_files = true;
+option java_package = "apache.rocketmq.v2";
+option java_generate_equals_and_hash = true;
+option java_string_check_utf8 = true;
+option java_outer_classname = "MQService";
+
+// Topics are destination of messages to publish to or subscribe from. Similar
+// to domain names, they will be addressable after resolution through the
+// provided access point.
+//
+// Access points are usually the addresses of name servers, which fulfill
+// service discovery, load-balancing and other auxiliary services. Name servers
+// receive periodic heartbeats from affiliate brokers and erase those which
+// failed to maintain alive status.
+//
+// Name servers answer queries of QueryRouteRequest, responding clients with
+// addressable message-queues, which they may directly publish messages to or
+// subscribe messages from.
+//
+// QueryRouteRequest shall include source endpoints, aka, configured
+// access-point, which annotates tenant-id, instance-id or other
+// vendor-specific settings. Purpose-built name servers may respond customized
+// results based on these particular requirements.
+message QueryRouteRequest {
+ Resource topic = 1;
+ Endpoints endpoints = 2;
+}
+
+message QueryRouteResponse {
+ Status status = 1;
+
+ repeated MessageQueue message_queues = 2;
+}
+
+message SendMessageRequest {
+ repeated Message messages = 1;
+}
+
+message SendResultEntry {
+ Status status = 1;
+ string message_id = 2;
+ string transaction_id = 3;
+ int64 offset = 4;
+}
+
+message SendMessageResponse {
+ Status status = 1;
+
+ // Some implementation may have partial failure issues. Client SDK developers are expected to inspect
+ // each entry for best certainty.
+ repeated SendResultEntry entries = 2;
+}
+
+message QueryAssignmentRequest {
+ Resource topic = 1;
+ Resource group = 2;
+ Endpoints endpoints = 3;
+}
+
+message QueryAssignmentResponse {
+ Status status = 1;
+ repeated Assignment assignments = 2;
+}
+
+message ReceiveMessageRequest {
+ Resource group = 1;
+ MessageQueue message_queue = 2;
+ FilterExpression filter_expression = 3;
+ int32 batch_size = 4;
+ // Required if client type is simple consumer.
+ optional google.protobuf.Duration invisible_duration = 5;
+ // For message auto renew and clean
+ bool auto_renew = 6;
+}
+
+message ReceiveMessageResponse {
+ oneof content {
+ Status status = 1;
+ Message message = 2;
+ // The timestamp that brokers start to deliver status line or message.
+ google.protobuf.Timestamp delivery_timestamp = 3;
+ }
+}
+
+message AckMessageEntry {
+ string message_id = 1;
+ string receipt_handle = 2;
+}
+
+message AckMessageRequest {
+ Resource group = 1;
+ Resource topic = 2;
+ repeated AckMessageEntry entries = 3;
+}
+
+message AckMessageResultEntry {
+ string message_id = 1;
+ string receipt_handle = 2;
+
+ // Acknowledge result may be acquired through inspecting
+ // `status.code`; In case acknowledgement failed, `status.message`
+ // is the explanation of the failure.
+ Status status = 3;
+}
+
+message AckMessageResponse {
+
+ // RPC tier status, which is used to represent RPC-level errors including
+ // authentication, authorization, throttling and other general failures.
+ Status status = 1;
+
+ repeated AckMessageResultEntry entries = 2;
+}
+
+message ForwardMessageToDeadLetterQueueRequest {
+ Resource group = 1;
+ Resource topic = 2;
+ string receipt_handle = 3;
+ string message_id = 4;
+ int32 delivery_attempt = 5;
+ int32 max_delivery_attempts = 6;
+}
+
+message ForwardMessageToDeadLetterQueueResponse { Status status = 1; }
+
+message HeartbeatRequest {
+ optional Resource group = 1;
+ ClientType client_type = 2;
+}
+
+message HeartbeatResponse { Status status = 1; }
+
+message EndTransactionRequest {
+ Resource topic = 1;
+ string message_id = 2;
+ string transaction_id = 3;
+ TransactionResolution resolution = 4;
+ TransactionSource source = 5;
+ string trace_context = 6;
+}
+
+message EndTransactionResponse { Status status = 1; }
+
+message PrintThreadStackTraceCommand { string nonce = 1; }
+
+message ThreadStackTrace {
+ string nonce = 1;
+ optional string thread_stack_trace = 2;
+}
+
+message VerifyMessageCommand {
+ string nonce = 1;
+ MessageQueue message_queue = 2;
+ Message message = 3;
+}
+
+message VerifyMessageResult {
+ string nonce = 1;
+}
+
+message RecoverOrphanedTransactionCommand {
+ MessageQueue message_queue = 1;
+ Message orphaned_transactional_message = 2;
+ string transaction_id = 3;
+}
+
+message Publishing {
+ // Publishing settings below here is appointed by client, thus it is
+ // unnecessary for server to push at present.
+ //
+ // List of topics to which messages will publish to.
+ repeated Resource topics = 1;
+
+ // Publishing settings below here are from server, it is essential for
+ // server to push.
+ //
+ // Body of message will be deflated if its size in bytes exceeds the
+ // threshold.
+ int32 compress_body_threshold = 2;
+
+ // If the message body size exceeds `max_body_size`, broker servers would
+ // reject the request. As a result, it is advisable that Producer performs
+ // client-side check validation.
+ int32 max_body_size = 3;
+}
+
+message Subscription {
+ // Subscription settings below here is appointed by client, thus it is
+ // unnecessary for server to push at present.
+ //
+ // Consumer group.
+ optional Resource group = 1;
+
+ // Subscription for consumer.
+ repeated SubscriptionEntry subscriptions = 2;
+
+ // Subscription settings below here are from server, it is essential for
+ // server to push.
+ //
+ // When FIFO flag is `true`, messages of the same message group are processed
+ // in first-in-first-out manner.
+ //
+ // Brokers will not deliver further messages of the same group utill prior
+ // ones are completely acknowledged.
+ optional bool fifo = 3;
+
+ // Message receive batch size here is essential for push consumer.
+ optional int32 receive_batch_size = 4;
+
+ // Long-polling timeout for `ReceiveMessageRequest`, which is essential for
+ // push consumer.
+ optional google.protobuf.Duration long_polling_timeout = 5;
+}
+
+message Metric {
+ // Indicates that if client should export local metrics to server.
+ bool on = 1;
+
+ // The endpoint that client metrics should be exported to, which is required if the switch is on.
+ optional Endpoints endpoints = 2;
+}
+
+message Settings {
+ // Configurations for all clients.
+ optional ClientType client_type = 1;
+
+ optional Endpoints access_point = 2;
+
+ // If publishing of messages encounters throttling or server internal errors,
+ // publishers should implement automatic retries after progressive longer
+ // back-offs for consecutive errors.
+ //
+ // When processing message fails, `backoff_policy` describes an interval
+ // after which the message should be available to consume again.
+ //
+ // For FIFO messages, the interval should be relatively small because
+ // messages of the same message group would not be readily available utill
+ // the prior one depletes its lifecycle.
+ optional RetryPolicy backoff_policy = 3;
+
+ // Request timeout for RPCs excluding long-polling.
+ optional google.protobuf.Duration request_timeout = 4;
+
+ oneof pub_sub {
+ Publishing publishing = 5;
+
+ Subscription subscription = 6;
+ }
+
+ // User agent details
+ UA user_agent = 7;
+
+ Metric metric = 8;
+}
+
+message TelemetryCommand {
+ optional Status status = 1;
+
+ oneof command {
+ // Client settings
+ Settings settings = 2;
+
+ // These messages are from client.
+ //
+ // Report thread stack trace to server.
+ ThreadStackTrace thread_stack_trace = 3;
+
+ // Report message verify result to server.
+ VerifyMessageResult verify_message_result = 4;
+
+ // There messages are from server.
+ //
+ // Request client to recover the orphaned transaction message.
+ RecoverOrphanedTransactionCommand recover_orphaned_transaction_command = 5;
+
+ // Request client to print thread stack trace.
+ PrintThreadStackTraceCommand print_thread_stack_trace_command = 6;
+
+ // Request client to verify the consumption of the appointed message.
+ VerifyMessageCommand verify_message_command = 7;
+ }
+}
+
+message NotifyClientTerminationRequest {
+ // Consumer group, which is absent for producer.
+ optional Resource group = 1;
+}
+
+message NotifyClientTerminationResponse { Status status = 1; }
+
+message ChangeInvisibleDurationRequest {
+ Resource group = 1;
+ Resource topic = 2;
+
+ // Unique receipt handle to identify message to change
+ string receipt_handle = 3;
+
+ // New invisible duration
+ google.protobuf.Duration invisible_duration = 4;
+
+ // For message tracing
+ string message_id = 5;
+}
+
+message ChangeInvisibleDurationResponse {
+ Status status = 1;
+
+ // Server may generate a new receipt handle for the message.
+ string receipt_handle = 2;
+}
+
+// For all the RPCs in MessagingService, the following error handling policies
+// apply:
+//
+// If the request doesn't bear a valid authentication credential, return a
+// response with common.status.code == `UNAUTHENTICATED`. If the authenticated
+// user is not granted with sufficient permission to execute the requested
+// operation, return a response with common.status.code == `PERMISSION_DENIED`.
+// If the per-user-resource-based quota is exhausted, return a response with
+// common.status.code == `RESOURCE_EXHAUSTED`. If any unexpected server-side
+// errors raise, return a response with common.status.code == `INTERNAL`.
+service MessagingService {
+
+ // Queries the route entries of the requested topic in the perspective of the
+ // given endpoints. On success, servers should return a collection of
+ // addressable message-queues. Note servers may return customized route
+ // entries based on endpoints provided.
+ //
+ // If the requested topic doesn't exist, returns `NOT_FOUND`.
+ // If the specific endpoints is empty, returns `INVALID_ARGUMENT`.
+ rpc QueryRoute(QueryRouteRequest) returns (QueryRouteResponse) {}
+
+ // Producer or consumer sends HeartbeatRequest to servers periodically to
+ // keep-alive. Additionally, it also reports client-side configuration,
+ // including topic subscription, load-balancing group name, etc.
+ //
+ // Returns `OK` if success.
+ //
+ // If a client specifies a language that is not yet supported by servers,
+ // returns `INVALID_ARGUMENT`
+ rpc Heartbeat(HeartbeatRequest) returns (HeartbeatResponse) {}
+
+ // Delivers messages to brokers.
+ // Clients may further:
+ // 1. Refine a message destination to message-queues which fulfills parts of
+ // FIFO semantic;
+ // 2. Flag a message as transactional, which keeps it invisible to consumers
+ // until it commits;
+ // 3. Time a message, making it invisible to consumers till specified
+ // time-point;
+ // 4. And more...
+ //
+ // Returns message-id or transaction-id with status `OK` on success.
+ //
+ // If the destination topic doesn't exist, returns `NOT_FOUND`.
+ rpc SendMessage(SendMessageRequest) returns (SendMessageResponse) {}
+
+ // Queries the assigned route info of a topic for current consumer,
+ // the returned assignment result is decided by server-side load balancer.
+ //
+ // If the corresponding topic doesn't exist, returns `NOT_FOUND`.
+ // If the specific endpoints is empty, returns `INVALID_ARGUMENT`.
+ rpc QueryAssignment(QueryAssignmentRequest) returns (QueryAssignmentResponse) {
+ }
+
+ // Receives messages from the server in batch manner, returns a set of
+ // messages if success. The received messages should be acked or redelivered
+ // after processed.
+ //
+ // If the pending concurrent receive requests exceed the quota of the given
+ // consumer group, returns `UNAVAILABLE`. If the upstream store server hangs,
+ // return `DEADLINE_EXCEEDED` in a timely manner. If the corresponding topic
+ // or consumer group doesn't exist, returns `NOT_FOUND`. If there is no new
+ // message in the specific topic, returns `OK` with an empty message set.
+ // Please note that client may suffer from false empty responses.
+ //
+ // If failed to receive message from remote, server must return only one
+ // `ReceiveMessageResponse` as the reply to the request, whose `Status` indicates
+ // the specific reason of failure, otherwise, the reply is considered successful.
+ rpc ReceiveMessage(ReceiveMessageRequest) returns (stream ReceiveMessageResponse) {
+ }
+
+ // Acknowledges the message associated with the `receipt_handle` or `offset`
+ // in the `AckMessageRequest`, it means the message has been successfully
+ // processed. Returns `OK` if the message server remove the relevant message
+ // successfully.
+ //
+ // If the given receipt_handle is illegal or out of date, returns
+ // `INVALID_ARGUMENT`.
+ rpc AckMessage(AckMessageRequest) returns (AckMessageResponse) {}
+
+ // Forwards one message to dead letter queue if the max delivery attempts is
+ // exceeded by this message at client-side, return `OK` if success.
+ rpc ForwardMessageToDeadLetterQueue(ForwardMessageToDeadLetterQueueRequest)
+ returns (ForwardMessageToDeadLetterQueueResponse) {}
+
+ // Commits or rollback one transactional message.
+ rpc EndTransaction(EndTransactionRequest) returns (EndTransactionResponse) {}
+
+ // Once a client starts, it would immediately establishes bi-lateral stream
+ // RPCs with brokers, reporting its settings as the initiative command.
+ //
+ // When servers have need of inspecting client status, they would issue
+ // telemetry commands to clients. After executing received instructions,
+ // clients shall report command execution results through client-side streams.
+ rpc Telemetry(stream TelemetryCommand) returns (stream TelemetryCommand) {}
+
+ // Notify the server that the client is terminated.
+ rpc NotifyClientTermination(NotifyClientTerminationRequest) returns (NotifyClientTerminationResponse) {
+ }
+
+ // Once a message is retrieved from consume queue on behalf of the group, it
+ // will be kept invisible to other clients of the same group for a period of
+ // time. The message is supposed to be processed within the invisible
+ // duration. If the client, which is in charge of the invisible message, is
+ // not capable of processing the message timely, it may use
+ // ChangeInvisibleDuration to lengthen invisible duration.
+ rpc ChangeInvisibleDuration(ChangeInvisibleDurationRequest) returns (ChangeInvisibleDurationResponse) {
+ }
+}
\ No newline at end of file
diff --git a/cpp/BUILD.bazel b/cpp/BUILD.bazel
index 8c03db4..5a0726b 100644
--- a/cpp/BUILD.bazel
+++ b/cpp/BUILD.bazel
@@ -26,5 +26,5 @@ proto_plugin(
cpp_grpc_library(
name = "rocketmq-proto-cpp-library",
- deps = ["//:rocketmq_v1_proto"]
+ deps = ["//:rocketmq_v2_proto"]
)
\ No newline at end of file
diff --git a/deps.bzl b/deps.bzl
index b5c20ed..181cb39 100644
--- a/deps.bzl
+++ b/deps.bzl
@@ -33,6 +33,6 @@ def io_grpc_grpc_java(**kwargs):
"""grpc java plugin and jars
"""
name = "io_grpc_grpc_java"
- ref = get_ref(name, "3c24dc6fe1b8f3e5c89b919c38a4eefe216397d3", kwargs) # v1.19.0 and changes up to PR #5456
- sha256 = get_sha256(name, "1eeb136874a58a0a311a0701016aced96919f501ced0372013eb1708724ab046", kwargs)
+ ref = get_ref(name, "8eff2630828a7ec6f4980b5b46f30f875070a4e4", kwargs) # v1.19.0 and changes up to PR #5456
+ sha256 = get_sha256(name, "f0e17fb16a404ba473429144481221e2c970c65596f65129002af3c73dcfe141", kwargs)
github_archive(name, "grpc", "grpc-java", ref, sha256)
\ No newline at end of file
diff --git a/java/BUILD.bazel b/java/BUILD.bazel
index 95cf665..93daa4c 100644
--- a/java/BUILD.bazel
+++ b/java/BUILD.bazel
@@ -20,7 +20,7 @@ proto_plugin(
java_grpc_compile(
name = "rocketmq_proto_src",
deps = [
- "//:rocketmq_v1_proto",
+ "//:rocketmq_v2_proto",
]
)