You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by ur...@apache.org on 2022/08/23 06:09:20 UTC
[pulsar-site] branch main updated: Docs sync done from apache/pulsar(#423ab75)
This is an automated email from the ASF dual-hosted git repository.
urfree pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
The following commit(s) were added to refs/heads/main by this push:
new c90dc2019ce Docs sync done from apache/pulsar(#423ab75)
c90dc2019ce is described below
commit c90dc2019ce9b771a733f717e7c98c4bed148d59
Author: Pulsar Site Updater <de...@pulsar.apache.org>
AuthorDate: Tue Aug 23 06:09:14 2022 +0000
Docs sync done from apache/pulsar(#423ab75)
---
site2/website-next/docs/admin-api-topics.md | 16 +++++-----
site2/website-next/docs/functions-concepts.md | 3 +-
site2/website-next/docs/functions-develop-api.md | 30 +++++++++++++++++++
site2/website-next/docs/io-debug.md | 4 +--
site2/website-next/docs/io-develop.md | 20 ++++++-------
site2/website-next/docs/reference-metrics.md | 2 +-
.../static/swagger/restApiVersions.json | 34 +++++++++++-----------
7 files changed, 70 insertions(+), 39 deletions(-)
diff --git a/site2/website-next/docs/admin-api-topics.md b/site2/website-next/docs/admin-api-topics.md
index 2bdab4a4e60..470f387906b 100644
--- a/site2/website-next/docs/admin-api-topics.md
+++ b/site2/website-next/docs/admin-api-topics.md
@@ -417,13 +417,13 @@ You can check the following statistics of a given non-partitioned topic.
- **nonContiguousDeletedMessagesRangesSerializedSize**: The serialized size of non-contiguous deleted messages ranges.
- - **throughEntryFilterMsgs**: The number of messages passes through `EntryFilter`.
+ - **filterProcessedMsgCount**: The number of messages processed by `EntryFilter`.
- - **entryFilterAccepted**: The number of messages accepted by `EntryFilter`.
+ - **filterAcceptedMsgCount**: The number of messages accepted by `EntryFilter`.
- - **entryFilterRejected**: The number of messages rejected by `EntryFilter`.
+ - **filterRejectedMsgCount**: The number of messages rejected by `EntryFilter`.
- - **entryFilterRescheduled**: The number of messages rescheduled by `EntryFilter`.
+ - **filterRescheduledMsgCount**: The number of messages rescheduled by `EntryFilter`.
- **consumers**: The list of connected consumers for this subscription.
@@ -549,10 +549,10 @@ The following is an example of a topic status.
"lastConsumedTimestamp" : 1623230583946,
"lastAckedTimestamp" : 1623230584033,
"lastMarkDeleteAdvancedTimestamp" : 1623230584033,
- "throughEntryFilterMsgs": 100,
- "entryFilterAccepted": 100,
- "entryFilterRejected": 0,
- "entryFilterRescheduled": 0,
+ "filterProcessedMsgCount": 100,
+ "filterAcceptedMsgCount": 100,
+ "filterRejectedMsgCount": 0,
+ "filterRescheduledMsgCount": 0,
"consumers" : [ {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
diff --git a/site2/website-next/docs/functions-concepts.md b/site2/website-next/docs/functions-concepts.md
index 4f40f8ec7be..8b49b62edc8 100644
--- a/site2/website-next/docs/functions-concepts.md
+++ b/site2/website-next/docs/functions-concepts.md
@@ -130,7 +130,8 @@ Java, Python, and Go SDKs provide access to a **context object** that can be use
* An interface for storing and retrieving state in [state storage](functions-develop-state.md).
* A function to publish new messages onto arbitrary topics.
* A function to acknowledge the message being processed (if auto-ack is disabled).
-* (Java) get Pulsar admin client.
+* (Java) A function to get the Pulsar admin client.
+* (Java) A function to create a Record to return with default values taken from the Context and the input Record.
:::tip
diff --git a/site2/website-next/docs/functions-develop-api.md b/site2/website-next/docs/functions-develop-api.md
index baa21f5fbe6..0a1f4a408fb 100644
--- a/site2/website-next/docs/functions-develop-api.md
+++ b/site2/website-next/docs/functions-develop-api.md
@@ -102,6 +102,36 @@ public class ExclamationFunction implements Function<String, String> {
For more details, see [code example](https://github.com/apache/pulsar/blob/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/ExclamationFunction.java).
+The return type of the function can be wrapped in a `Record` generic which gives you more control over the output messages, such as topics, schemas, properties, and so on.
+Use the `Context::newOutputRecordBuilder` method to build this `Record` output.
+
+```java
+
+import java.util.HashMap;
+import java.util.Map;
+import org.apache.pulsar.functions.api.Context;
+import org.apache.pulsar.functions.api.Function;
+import org.apache.pulsar.functions.api.Record;
+
+public class RecordFunction implements Function<String, Record<String>> {
+
+ @Override
+ public Record<String> process(String input, Context context) throws Exception {
+ String output = String.format("%s!", input);
+ Map<String, String> properties = new HashMap<>(context.getCurrentRecord().getProperties());
+ context.getCurrentRecord().getTopicName().ifPresent(topic -> properties.put("input_topic", topic));
+
+ return context.newOutputRecordBuilder(Schema.STRING)
+ .value(output)
+ .properties(properties)
+ .build();
+ }
+}
+
+```
+
+For more details, see [code example](https://github.com/apache/pulsar/blob/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/RecordFunction.java).
+
</TabItem>
<TabItem value="Python">
diff --git a/site2/website-next/docs/io-debug.md b/site2/website-next/docs/io-debug.md
index dbde5ab82d4..7fa071bc486 100644
--- a/site2/website-next/docs/io-debug.md
+++ b/site2/website-next/docs/io-debug.md
@@ -4,7 +4,7 @@ title: How to debug Pulsar connectors
sidebar_label: "Debug"
---
This guide explains how to debug connectors in localrun or cluster mode and gives a debugging checklist.
-To better demonstrate how to debug Pulsar connectors, here takes a Mongo sink connector as an example.
+To better demonstrate how to debug Pulsar connectors, take the Mongo sink connector as an example.
**Deploy a Mongo sink environment**
1. Start a Mongo service.
@@ -106,7 +106,7 @@ Use one of the following methods to get a connector log in localrun mode:
```
-To clearly explain the log information, here breaks down the large block of information into small blocks and add descriptions for each block.
+To clearly explain the log information, the following is a breakdown into smaller blocks with added descriptions.
* This piece of log information shows the storage path of the nar package after decompression.
```
diff --git a/site2/website-next/docs/io-develop.md b/site2/website-next/docs/io-develop.md
index d361a302b25..1c814a85f6a 100644
--- a/site2/website-next/docs/io-develop.md
+++ b/site2/website-next/docs/io-develop.md
@@ -70,7 +70,7 @@ interface, which means you need to implement the {@inject: github:open:/pulsar-i
```
- If nothing to return, the implementation should be blocking rather than returning `null`.
+ If there is nothing to return, the implementation should be blocking rather than returning `null`.
The returned {@inject: github:Record:/pulsar-functions/api-java/src/main/java/org/apache/pulsar/functions/api/Record.java} should encapsulate the following information, which is needed by Pulsar IO runtime.
@@ -95,10 +95,10 @@ interface, which means you need to implement the {@inject: github:open:/pulsar-i
`ack` |Acknowledge that the record is fully processed.
`fail`|Indicate that the record fails to be processed.
-## Handle schema information
+#### Handle schema information
Pulsar IO automatically handles the schema and provides a strongly typed API based on Java generics.
-If you know the schema type that you are producing, you can declare the Java class relative to that type in your sink declaration.
+If you know the schema type that you are producing, you can declare the Java class relative to that type in your source declaration.
```
@@ -136,8 +136,8 @@ To handle the `KeyValue` type properly, follow the guidelines for your record im
- It must return a `KeyValue` object as `Record.getValue()`
- It may return null in `Record.getSchema()`
-When Pulsar IO runtime encounters a `KVRecord`, it brings the following changes automatically:
-- Set properly the `KeyValueSchema`
+When Pulsar IO runtime encounters a `KVRecord`, it does the following changes automatically:
+- Set the proper `KeyValueSchema`
- Encode the Message Key and the Message Value according to the `KeyValueEncoding` (SEPARATED or INLINE)
:::tip
@@ -179,14 +179,14 @@ Developing a sink connector **is similar to** developing a source connector, tha
```
During the implementation, you can decide how to write the `Value` and
- the `Key` to the actual source, and leverage all the provided information such as
+ the `Key` to the actual sink, and leverage all the provided information such as
`PartitionId` and `RecordSequence` to achieve different processing guarantees.
You also need to ack records (if messages are sent successfully) or fail records (if messages fail to send).
-## Handling Schema information
+#### Handle schema information
-Pulsar IO handles automatically the Schema and provides a strongly typed API based on Java generics.
+Pulsar IO automatically handles the Schema and provides a strongly typed API based on Java generics.
If you know the Schema type that you are consuming from you can declare the Java class relative to that type in your Sink declaration.
```
@@ -282,11 +282,11 @@ work with Pulsar Functions' runtime, that is, [NAR](#nar) and [uber JAR](#uber-j
:::note
If you plan to package and distribute your connector for others to use, you are obligated to
+license and copyright your own code properly. Remember to add the license and copyright to
+all libraries your code uses and to your distribution.
:::
-license and copyright your own code properly. Remember to add the license and copyright to
-all libraries your code uses and to your distribution.
>
> If you use the [NAR](#nar) method, the NAR plugin
automatically creates a `DEPENDENCIES` file in the generated NAR package, including the proper
diff --git a/site2/website-next/docs/reference-metrics.md b/site2/website-next/docs/reference-metrics.md
index 31ed3baf776..aa59c02fee8 100644
--- a/site2/website-next/docs/reference-metrics.md
+++ b/site2/website-next/docs/reference-metrics.md
@@ -444,7 +444,7 @@ All the subscription metrics are labelled with the following labels:
| pulsar_subscription_total_msg_expired | Gauge | The total number of messages expired on this subscription. |
| pulsar_subscription_msg_drop_rate | Gauge | The rate of messages dropped on this subscription (message per second). |
| pulsar_subscription_consumers_count | Gauge | The number of connected consumers on this subscription. |
-| pulsar_subscription_through_filter_msg_count | Counter | The number of messages passes through `EntryFilter`. |
+| pulsar_subscription_filter_processed_msg_count | Counter | The number of messages processed by `EntryFilter`. |
| pulsar_subscription_filter_accepted_msg_count | Counter | The number of messages accepted by `EntryFilter`. |
| pulsar_subscription_filter_rejected_msg_count | Counter | The number of messages rejected by `EntryFilter`. |
| pulsar_subscription_filter_rescheduled_msg_count | Counter | The number of messages rescheduled by `EntryFilter`. |
diff --git a/site2/website-next/static/swagger/restApiVersions.json b/site2/website-next/static/swagger/restApiVersions.json
index 34ad9254ef4..8d35bd4d0b0 100644
--- a/site2/website-next/static/swagger/restApiVersions.json
+++ b/site2/website-next/static/swagger/restApiVersions.json
@@ -1,4 +1,21 @@
{
+ "2.10.0": [
+ {
+ "version": "v2",
+ "fileName": [
+ "swagger"
+ ]
+ },
+ {
+ "version": "v3",
+ "fileName": [
+ "swaggerfunctions",
+ "swaggerpackages",
+ "swaggersink",
+ "swaggersource"
+ ]
+ }
+ ],
"2.10.1": [
{
"version": "v2",
@@ -339,23 +356,6 @@
]
}
],
- "2.10.0": [
- {
- "version": "v2",
- "fileName": [
- "swagger"
- ]
- },
- {
- "version": "v3",
- "fileName": [
- "swaggerfunctions",
- "swaggerpackages",
- "swaggersink",
- "swaggersource"
- ]
- }
- ],
"2.8.3": [
{
"version": "v2",