You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by li...@apache.org on 2022/07/26 06:30:28 UTC
[rocketmq-clients] branch csharp_dev updated (34d4e2d -> 83a7774)
This is an automated email from the ASF dual-hosted git repository.
lizhanhui pushed a change to branch csharp_dev
in repository https://gitbox.apache.org/repos/asf/rocketmq-clients.git
omit 34d4e2d Sync changes from upstream
omit a2bca73 Merge changes from upstream
add ab9353d Bump jacoco to 0.8.8 (#62)
add aaad80d Merge changes from upstream (#61)
add 3cb7fbe Add new label(consumerGroup) for the metrics of consumer (#63)
add 8202ef8 Do not check body size in MessageBuilderImpl#setBody (#64)
add 8d873be Using default aggregationTemporalitySelector (#67)
add 739f754 Prepare to sync examples (#66)
add d0fa7a3 Support license checker (#65)
add d1b3518 Sync consumer examples (#68)
add 5c519ab Polish README.md (#69)
add f7c4302 Update README.md
add b6c4641 Update README.md
add b8145d5 Revert "Support license checker (#65)"
add 8c21cbb Collect stats for local cache (#70)
new 83a7774 Use rocketmq-apis 1.0
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (34d4e2d)
\
N -- N -- N refs/heads/csharp_dev (83a7774)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
cpp/README.md | 64 ++++-
cpp/bazel/rocketmq_deps.bzl | 189 +++++++-------
cpp/examples/BUILD.bazel | 107 +++-----
cpp/examples/BenchmarkPushConsumer.cpp | 83 ------
cpp/examples/ExampleFifoPushConsumer.cpp | 67 -----
cpp/examples/ExampleProducer.cpp | 44 +++-
...ncProducer.cpp => ExampleProducerWithAsync.cpp} | 32 ++-
...ucer.cpp => ExampleProducerWithFifoMessage.cpp} | 29 ++-
...cer.cpp => ExampleProducerWithTimedMessage.cpp} | 38 ++-
...=> ExampleProducerWithTransactionalMessage.cpp} | 65 ++---
cpp/examples/ExamplePushConsumer.cpp | 27 +-
cpp/examples/ExampleSimpleConsumer.cpp | 22 +-
cpp/examples/ExampleTransactionProducer.cpp | 52 ----
cpp/examples/PushConsumerWithCustomExecutor.cpp | 147 -----------
cpp/examples/PushConsumerWithThrottle.cpp | 88 -------
cpp/examples/SqlConsumer.cpp | 61 -----
cpp/examples/SqlProducer.cpp | 60 -----
cpp/include/rocketmq/Message.h | 9 +
cpp/include/rocketmq/Producer.h | 5 +
cpp/include/rocketmq/SendReceipt.h | 2 +
cpp/include/rocketmq/Transaction.h | 10 -
cpp/source/base/Message.cpp | 5 +
cpp/source/base/ThreadPoolImpl.cpp | 2 +-
cpp/source/client/ClientManagerImpl.cpp | 17 +-
cpp/source/client/TelemetryBidiReactor.cpp | 5 +-
.../rocketmq/AsyncReceiveMessageCallback.cpp | 3 +-
cpp/source/rocketmq/ConsumeTask.cpp | 8 +-
cpp/source/rocketmq/ProcessQueueImpl.cpp | 59 ++---
cpp/source/rocketmq/Producer.cpp | 9 +
cpp/source/rocketmq/ProducerImpl.cpp | 79 +++---
cpp/source/rocketmq/PushConsumerImpl.cpp | 59 +++++
cpp/source/rocketmq/SendContext.cpp | 4 +-
cpp/source/rocketmq/TransactionImpl.cpp | 25 +-
cpp/source/rocketmq/include/ProcessQueue.h | 15 +-
cpp/source/rocketmq/include/ProcessQueueImpl.h | 48 +---
cpp/source/rocketmq/include/ProducerImpl.h | 17 +-
cpp/source/rocketmq/include/PushConsumerImpl.h | 4 +
cpp/source/rocketmq/include/TransactionImpl.h | 47 ++--
cpp/source/scheduler/SchedulerImpl.cpp | 2 +-
cpp/source/stats/Tag.cpp | 7 +-
cpp/source/stats/include/Tag.h | 4 +-
.../Protos/apache/rocketmq/v2/definition.proto | 285 ++++++++++++++-------
.../Protos/apache/rocketmq/v2/service.proto | 97 +------
.../client/java/message/MessageBuilderImpl.java | 3 -
.../client/java/metrics/ClientMeterProvider.java | 2 -
java/pom.xml | 2 +-
46 files changed, 800 insertions(+), 1209 deletions(-)
delete mode 100644 cpp/examples/BenchmarkPushConsumer.cpp
delete mode 100644 cpp/examples/ExampleFifoPushConsumer.cpp
copy cpp/examples/{ExampleAsyncProducer.cpp => ExampleProducerWithAsync.cpp} (73%)
copy cpp/examples/{ExampleFifoProducer.cpp => ExampleProducerWithFifoMessage.cpp} (73%)
rename cpp/examples/{ExampleFifoProducer.cpp => ExampleProducerWithTimedMessage.cpp} (67%)
rename cpp/examples/{ExampleAsyncProducer.cpp => ExampleProducerWithTransactionalMessage.cpp} (59%)
delete mode 100644 cpp/examples/ExampleTransactionProducer.cpp
delete mode 100644 cpp/examples/PushConsumerWithCustomExecutor.cpp
delete mode 100644 cpp/examples/PushConsumerWithThrottle.cpp
delete mode 100644 cpp/examples/SqlConsumer.cpp
delete mode 100644 cpp/examples/SqlProducer.cpp
[rocketmq-clients] 01/01: Use rocketmq-apis 1.0
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lizhanhui pushed a commit to branch csharp_dev
in repository https://gitbox.apache.org/repos/asf/rocketmq-clients.git
commit 83a7774a6492aa0de8ce22c91b6428561c6c04c7
Author: Li Zhanhui <li...@gmail.com>
AuthorDate: Tue Jul 26 14:28:59 2022 +0800
Use rocketmq-apis 1.0
---
.../Protos/apache/rocketmq/v2/definition.proto | 285 ++++++++++++++-------
.../Protos/apache/rocketmq/v2/service.proto | 97 +------
2 files changed, 194 insertions(+), 188 deletions(-)
diff --git a/csharp/rocketmq-client-csharp/Protos/apache/rocketmq/v2/definition.proto b/csharp/rocketmq-client-csharp/Protos/apache/rocketmq/v2/definition.proto
index 21a6321..02b009a 100644
--- a/csharp/rocketmq-client-csharp/Protos/apache/rocketmq/v2/definition.proto
+++ b/csharp/rocketmq-client-csharp/Protos/apache/rocketmq/v2/definition.proto
@@ -74,7 +74,7 @@ message ExponentialBackoff {
}
message CustomizedBackoff {
- // To support classic backoff strategy which is arbitary defined by end users.
+ // To support classic backoff strategy which is arbitrary defined by end users.
// Typical values are: `1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h`
repeated google.protobuf.Duration next = 1;
}
@@ -175,10 +175,6 @@ enum DigestType {
// 1) Standard messages should be negatively acknowledged instantly, causing
// immediate re-delivery; 2) FIFO messages require special RPC, to re-fetch
// previously acquired messages batch;
-//
-// Message consumption model also affects how invalid digest are handled. When
-// messages are consumed in broadcasting way,
-// TODO: define semantics of invalid-digest-when-broadcasting.
message Digest {
DigestType type = 1;
string checksum = 2;
@@ -288,109 +284,101 @@ message Message {
bytes body = 4;
}
-message Assignment { MessageQueue message_queue = 1; }
+message Assignment {
+ MessageQueue message_queue = 1;
+}
enum Code {
- // Success.
- OK = 0;
+ CODE_UNSPECIFIED = 0;
+
+ // Generic code for success.
+ OK = 20000;
+
+ // Generic code for multiple return results.
+ MULTIPLE_RESULTS = 30000;
+
+ // Generic code for bad request, indicating that required fields or headers are missing.
+ BAD_REQUEST = 40000;
// Format of access point is illegal.
- ILLEGAL_ACCESS_POINT = 1;
+ ILLEGAL_ACCESS_POINT = 40001;
// Format of topic is illegal.
- ILLEGAL_TOPIC = 2;
+ ILLEGAL_TOPIC = 40002;
// Format of consumer group is illegal.
- ILLEGAL_CONSUMER_GROUP = 3;
+ ILLEGAL_CONSUMER_GROUP = 40003;
// Format of message tag is illegal.
- ILLEGAL_MESSAGE_TAG = 4;
+ ILLEGAL_MESSAGE_TAG = 40004;
// Format of message key is illegal.
- ILLEGAL_MESSAGE_KEY = 5;
- // Size of message keys exceeds the threshold.
- MESSAGE_KEYS_TOO_LARGE = 6;
+ ILLEGAL_MESSAGE_KEY = 40005;
// Format of message group is illegal.
- ILLEGAL_MESSAGE_GROUP = 7;
+ ILLEGAL_MESSAGE_GROUP = 40006;
// Format of message property key is illegal.
- ILLEGAL_MESSAGE_PROPERTY_KEY = 8;
- // Message properties total size exceeds the threshold.
- MESSAGE_PROPERTIES_TOO_LARGE = 9;
- // Message body size exceeds the threshold.
- MESSAGE_BODY_TOO_LARGE = 10;
-
- // User does not have the permission to operate.
- // See https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403
- FORBIDDEN = 403;
-
- // Code indicates that the client request has not been completed
- // because it lacks valid authentication credentials for the
- // requested resource.
- // See https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401
- UNAUTHORIZED = 401;
-
- // Topic resource does not exist.
- TOPIC_NOT_FOUND = 13;
-
- // Consumer group resource does not exist.
- CONSUMER_GROUP_NOT_FOUND = 14;
-
- // Not allowed to verify message. Chances are that you are verifying
- // a FIFO message, as is violating FIFO semantics.
- VERIFY_MESSAGE_FORBIDDEN = 15;
-
- // Failed to consume message.
- FAILED_TO_CONSUME_MESSAGE = 16;
-
- // Message is corrupted.
- MESSAGE_CORRUPTED = 17;
-
- // Too many requests are made in short period of duration.
- // Requests are throttled.
- TOO_MANY_REQUESTS = 18;
-
- // Expired receipt-handle is used when trying to acknowledge or change
- // invisible duration of a message
- RECEIPT_HANDLE_EXPIRED = 19;
-
- // Message property is not match the message type.
- MESSAGE_PROPERTY_DOES_NOT_MATCH_MESSAGE_TYPE = 20;
-
- // Format of message id is illegal.
- ILLEGAL_MESSAGE_ID = 21;
-
+ ILLEGAL_MESSAGE_PROPERTY_KEY = 40007;
// Transaction id is invalid.
- INVALID_TRANSACTION_ID = 22;
-
+ INVALID_TRANSACTION_ID = 40008;
+ // Format of message id is illegal.
+ ILLEGAL_MESSAGE_ID = 40009;
// Format of filter expression is illegal.
- ILLEGAL_FILTER_EXPRESSION = 23;
-
+ ILLEGAL_FILTER_EXPRESSION = 40010;
+ // The invisible time of request is invalid.
+ ILLEGAL_INVISIBLE_TIME = 40011;
+ // The delivery timestamp of message is invalid.
+ ILLEGAL_DELIVERY_TIME = 40012;
// Receipt handle of message is invalid.
- INVALID_RECEIPT_HANDLE = 24;
-
- // Message persistence timeout.
- MASTER_PERSISTENCE_TIMEOUT = 25;
+ INVALID_RECEIPT_HANDLE = 40013;
+ // Message property conflicts with its type.
+ MESSAGE_PROPERTY_CONFLICT_WITH_TYPE = 40014;
+ // Client type could not be recognized.
+ UNRECOGNIZED_CLIENT_TYPE = 40015;
+ // Message is corrupted.
+ MESSAGE_CORRUPTED = 40016;
+ // Request is rejected due to missing of x-mq-client-id header.
+ CLIENT_ID_REQUIRED = 40017;
- // Slave persistence timeout.
- SLAVE_PERSISTENCE_TIMEOUT = 26;
+ // Generic code indicates that the client request lacks valid authentication
+ // credentials for the requested resource.
+ UNAUTHORIZED = 40100;
- // The HA-mechanism is not working now.
- HA_NOT_AVAILABLE = 27;
+ // Generic code indicates that the account is suspended due to overdue of payment.
+ PAYMENT_REQUIRED = 40200;
- // Operation is not allowed in current version.
- VERSION_UNSUPPORTED = 28;
+ // Generic code for the case that user does not have the permission to operate.
+ FORBIDDEN = 40300;
+ // Generic code for resource not found.
+ NOT_FOUND = 40400;
// Message not found from server.
- MESSAGE_NOT_FOUND = 29;
+ MESSAGE_NOT_FOUND = 40401;
+ // Topic resource does not exist.
+ TOPIC_NOT_FOUND = 40402;
+ // Consumer group resource does not exist.
+ CONSUMER_GROUP_NOT_FOUND = 40403;
- // Message offset is illegal.
- ILLEGAL_MESSAGE_OFFSET = 30;
+ // Generic code representing client side timeout when connecting to, reading data from, or write data to server.
+ REQUEST_TIMEOUT = 40800;
- // Illegal message is for the sake of backward compatibility. In most case,
- // more definitive code is better, e.g. `ILLEGAL_MESSAGE_TAG`.
- ILLEGAL_MESSAGE = 31;
+ // Generic code represents that the request entity is larger than limits defined by server.
+ PAYLOAD_TOO_LARGE = 41300;
+ // Message body size exceeds the threshold.
+ MESSAGE_BODY_TOO_LARGE = 41301;
- // Client type could not be recognized.
- UNRECOGNIZED_CLIENT_TYPE = 32;
+ // Generic code for use cases where pre-conditions are not met.
+ // For example, if a producer instance is used to publish messages without prior start() invocation,
+ // this error code will be raised.
+ PRECONDITION_FAILED = 42800;
- // Return different results for entries in composite request.
- MULTIPLE_RESULTS = 33;
+ // Generic code indicates that too many requests are made in short period of duration.
+ // Requests are throttled.
+ TOO_MANY_REQUESTS = 42900;
+
+ // Generic code for the case that the server is unwilling to process the request because its header fields are too large.
+ // The request may be resubmitted after reducing the size of the request header fields.
+ REQUEST_HEADER_FIELDS_TOO_LARGE = 43100;
+ // Message properties total size exceeds the threshold.
+ MESSAGE_PROPERTIES_TOO_LARGE = 43101;
+ // Generic code indicates that server/client encountered an unexpected
+ // condition that prevented it from fulfilling the request.
+ INTERNAL_ERROR = 50000;
// Code indicates that the server encountered an unexpected condition
// that prevented it from fulfilling the request.
// This error response is a generic "catch-all" response.
@@ -400,17 +388,33 @@ enum Code {
// to prevent the error from happening again in the future.
//
// See https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
- INTERNAL_SERVER_ERROR = 500;
+ INTERNAL_SERVER_ERROR = 50001;
+ // The HA-mechanism is not working now.
+ HA_NOT_AVAILABLE = 50002;
- // Code means that the server or client does not support the functionality
- // required to fulfill the request.
- NOT_IMPLEMENTED = 501;
+ // Generic code means that the server or client does not support the
+ // functionality required to fulfill the request.
+ NOT_IMPLEMENTED = 50100;
- // Code indicates that the server, while acting as a gateway or proxy,
- // did not get a response in time from the upstream server that
- // it needed in order to complete the request.
+ // Generic code represents that the server, which acts as a gateway or proxy,
+ // does not get an satisfied response in time from its upstream servers.
// See https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/504
- GATEWAY_TIMEOUT = 504;
+ PROXY_TIMEOUT = 50400;
+ // Message persistence timeout.
+ MASTER_PERSISTENCE_TIMEOUT = 50401;
+ // Slave persistence timeout.
+ SLAVE_PERSISTENCE_TIMEOUT = 50402;
+
+ // Generic code for unsupported operation.
+ UNSUPPORTED = 50500;
+ // Operation is not allowed in current version.
+ VERSION_UNSUPPORTED = 50501;
+ // Not allowed to verify message. Chances are that you are verifying
+ // a FIFO message, as is violating FIFO semantics.
+ VERIFY_FIFO_MESSAGE_UNSUPPORTED = 50502;
+
+ // Generic code for failed message consumption.
+ FAILED_TO_CONSUME_MESSAGE = 60000;
}
message Status {
@@ -425,6 +429,13 @@ enum Language {
DOT_NET = 3;
GOLANG = 4;
RUST = 5;
+ PYTHON = 6;
+ PHP = 7;
+ NODE_JS = 8;
+ RUBY = 9;
+ OBJECTIVE_C = 10;
+ DART = 11;
+ KOTLIN = 12;
}
// User Agent
@@ -440,4 +451,90 @@ message UA {
// Hostname of the node
string hostname = 4;
+}
+
+message Settings {
+ // Configurations for all clients.
+ optional ClientType client_type = 1;
+
+ optional Endpoints access_point = 2;
+
+ // If publishing of messages encounters throttling or server internal errors,
+ // publishers should implement automatic retries after progressive longer
+ // back-offs for consecutive errors.
+ //
+ // When processing message fails, `backoff_policy` describes an interval
+ // after which the message should be available to consume again.
+ //
+ // For FIFO messages, the interval should be relatively small because
+ // messages of the same message group would not be readily available until
+ // the prior one depletes its lifecycle.
+ optional RetryPolicy backoff_policy = 3;
+
+ // Request timeout for RPCs excluding long-polling.
+ optional google.protobuf.Duration request_timeout = 4;
+
+ oneof pub_sub {
+ Publishing publishing = 5;
+
+ Subscription subscription = 6;
+ }
+
+ // User agent details
+ UA user_agent = 7;
+
+ Metric metric = 8;
+}
+
+message Publishing {
+ // Publishing settings below here is appointed by client, thus it is
+ // unnecessary for server to push at present.
+ //
+ // List of topics to which messages will publish to.
+ repeated Resource topics = 1;
+
+ // If the message body size exceeds `max_body_size`, broker servers would
+ // reject the request. As a result, it is advisable that Producer performs
+ // client-side check validation.
+ int32 max_body_size = 2;
+
+ // When `validate_message_type` flag set `false`, no need to validate message's type
+ // with messageQueue's `accept_message_types` before publishing.
+ bool validate_message_type = 3;
+}
+
+message Subscription {
+ // Subscription settings below here is appointed by client, thus it is
+ // unnecessary for server to push at present.
+ //
+ // Consumer group.
+ optional Resource group = 1;
+
+ // Subscription for consumer.
+ repeated SubscriptionEntry subscriptions = 2;
+
+ // Subscription settings below here are from server, it is essential for
+ // server to push.
+ //
+ // When FIFO flag is `true`, messages of the same message group are processed
+ // in first-in-first-out manner.
+ //
+ // Brokers will not deliver further messages of the same group until prior
+ // ones are completely acknowledged.
+ optional bool fifo = 3;
+
+ // Message receive batch size here is essential for push consumer.
+ optional int32 receive_batch_size = 4;
+
+ // Long-polling timeout for `ReceiveMessageRequest`, which is essential for
+ // push consumer.
+ optional google.protobuf.Duration long_polling_timeout = 5;
+}
+
+message Metric {
+ // Indicates that if client should export local metrics to server.
+ bool on = 1;
+
+ // The endpoint that client metrics should be exported to, which is required if the switch is on.
+ optional Endpoints endpoints = 2;
}
\ No newline at end of file
diff --git a/csharp/rocketmq-client-csharp/Protos/apache/rocketmq/v2/service.proto b/csharp/rocketmq-client-csharp/Protos/apache/rocketmq/v2/service.proto
index c7ce2e9..8e880e2 100644
--- a/csharp/rocketmq-client-csharp/Protos/apache/rocketmq/v2/service.proto
+++ b/csharp/rocketmq-client-csharp/Protos/apache/rocketmq/v2/service.proto
@@ -175,8 +175,7 @@ message ThreadStackTrace {
message VerifyMessageCommand {
string nonce = 1;
- MessageQueue message_queue = 2;
- Message message = 3;
+ Message message = 2;
}
message VerifyMessageResult {
@@ -184,98 +183,8 @@ message VerifyMessageResult {
}
message RecoverOrphanedTransactionCommand {
- MessageQueue message_queue = 1;
- Message orphaned_transactional_message = 2;
- string transaction_id = 3;
-}
-
-message Publishing {
- // Publishing settings below here is appointed by client, thus it is
- // unnecessary for server to push at present.
- //
- // List of topics to which messages will publish to.
- repeated Resource topics = 1;
-
- // Publishing settings below here are from server, it is essential for
- // server to push.
- //
- // Body of message will be deflated if its size in bytes exceeds the
- // threshold.
- int32 compress_body_threshold = 2;
-
- // If the message body size exceeds `max_body_size`, broker servers would
- // reject the request. As a result, it is advisable that Producer performs
- // client-side check validation.
- int32 max_body_size = 3;
-}
-
-message Subscription {
- // Subscription settings below here is appointed by client, thus it is
- // unnecessary for server to push at present.
- //
- // Consumer group.
- optional Resource group = 1;
-
- // Subscription for consumer.
- repeated SubscriptionEntry subscriptions = 2;
-
- // Subscription settings below here are from server, it is essential for
- // server to push.
- //
- // When FIFO flag is `true`, messages of the same message group are processed
- // in first-in-first-out manner.
- //
- // Brokers will not deliver further messages of the same group utill prior
- // ones are completely acknowledged.
- optional bool fifo = 3;
-
- // Message receive batch size here is essential for push consumer.
- optional int32 receive_batch_size = 4;
-
- // Long-polling timeout for `ReceiveMessageRequest`, which is essential for
- // push consumer.
- optional google.protobuf.Duration long_polling_timeout = 5;
-}
-
-message Metric {
- // Indicates that if client should export local metrics to server.
- bool on = 1;
-
- // The endpoint that client metrics should be exported to, which is required if the switch is on.
- optional Endpoints endpoints = 2;
-}
-
-message Settings {
- // Configurations for all clients.
- optional ClientType client_type = 1;
-
- optional Endpoints access_point = 2;
-
- // If publishing of messages encounters throttling or server internal errors,
- // publishers should implement automatic retries after progressive longer
- // back-offs for consecutive errors.
- //
- // When processing message fails, `backoff_policy` describes an interval
- // after which the message should be available to consume again.
- //
- // For FIFO messages, the interval should be relatively small because
- // messages of the same message group would not be readily available utill
- // the prior one depletes its lifecycle.
- optional RetryPolicy backoff_policy = 3;
-
- // Request timeout for RPCs excluding long-polling.
- optional google.protobuf.Duration request_timeout = 4;
-
- oneof pub_sub {
- Publishing publishing = 5;
-
- Subscription subscription = 6;
- }
-
- // User agent details
- UA user_agent = 7;
-
- Metric metric = 8;
+ Message message = 1;
+ string transaction_id = 2;
}
message TelemetryCommand {