You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/11/23 15:43:38 UTC
[GitHub] [pulsar] cbornet opened a new pull request, #18582: [improve][doc] Add doc for the HTTP Sink
cbornet opened a new pull request, #18582:
URL: https://github.com/apache/pulsar/pull/18582
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [x] `doc` <!-- Your PR contains doc changes. Please attach the local preview screenshots (run `sh start.sh` at `pulsar/site2/website`) to your PR description, or else your PR might not get merged. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
### Matching PR in forked repository
PR in forked repository: <!-- ENTER URL HERE -->
<!--
After opening this PR, the build in apache/pulsar will fail and instructions will
be provided for opening a PR in the PR author's forked repository.
apache/pulsar pull requests should be first tested in your own fork since the
apache/pulsar CI based on GitHub Actions has constrained resources and quota.
GitHub Actions provides separate quota for pull requests that are executed in
a forked repository.
The tests will be run in the forked repository until all PR review comments have
been handled, the tests pass and the PR is approved by a reviewer.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on a diff in pull request #18582: [feat][doc] Add doc for the HTTP Sink
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on code in PR #18582:
URL: https://github.com/apache/pulsar/pull/18582#discussion_r1035461432
##########
site2/docs/io-http-sink.md:
##########
@@ -0,0 +1,56 @@
+---
+id: io-http-sink
+title: HTTTP sink connector
+sidebar_label: "HTTP sink connector"
+---
+
+The HTTP sink connector pulls the records from Pulsar topics and makes a POST request to a configurable HTTP URL (webhook).
+
+The body of the HTTP request is the JSON representation of the record value. The header `Content-Type: application/json` is added to the HTTP request.
+
+Some other HTTP headers are added to the HTTP request:
+
+* `PulsarTopic`: the topic of the record
+* `PulsarKey`: the key of the record
+* `PulsarEventTime`: the event time of the record
+* `PulsarPublishTime`: the publish time of the record
+* `PulsarMessageId`: the ID of the message contained in the record
+* `PulsarProperties`-*: each record property is passed with the property name prefixed by `PulsarProperties-`
Review Comment:
```suggestion
* `PulsarProperties-`*: each record property is passed with the property name prefixed by `PulsarProperties-`
```
why this contains *? Does it mean `required` or something else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun merged pull request #18582: [feat][doc] Add doc for the HTTP Sink
Posted by GitBox <gi...@apache.org>.
tisonkun merged PR #18582:
URL: https://github.com/apache/pulsar/pull/18582
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] cbornet commented on a diff in pull request #18582: [feat][doc] Add doc for the HTTP Sink
Posted by GitBox <gi...@apache.org>.
cbornet commented on code in PR #18582:
URL: https://github.com/apache/pulsar/pull/18582#discussion_r1035782415
##########
site2/docs/io-http-sink.md:
##########
@@ -0,0 +1,56 @@
+---
+id: io-http-sink
+title: HTTTP sink connector
+sidebar_label: "HTTP sink connector"
+---
+
+The HTTP sink connector pulls the records from Pulsar topics and makes a POST request to a configurable HTTP URL (webhook).
+
+The body of the HTTP request is the JSON representation of the record value. The header `Content-Type: application/json` is added to the HTTP request.
+
+Some other HTTP headers are added to the HTTP request:
+
+* `PulsarTopic`: the topic of the record
+* `PulsarKey`: the key of the record
+* `PulsarEventTime`: the event time of the record
+* `PulsarPublishTime`: the publish time of the record
+* `PulsarMessageId`: the ID of the message contained in the record
+* `PulsarProperties`-*: each record property is passed with the property name prefixed by `PulsarProperties-`
Review Comment:
Oups, I committed the wrong suggestion. I did another commit to adjust.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun commented on a diff in pull request #18582: [feat][doc] Add doc for the HTTP Sink
Posted by GitBox <gi...@apache.org>.
tisonkun commented on code in PR #18582:
URL: https://github.com/apache/pulsar/pull/18582#discussion_r1035462682
##########
site2/docs/io-http-sink.md:
##########
@@ -0,0 +1,56 @@
+---
+id: io-http-sink
+title: HTTTP sink connector
+sidebar_label: "HTTP sink connector"
+---
+
+The HTTP sink connector pulls the records from Pulsar topics and makes a POST request to a configurable HTTP URL (webhook).
+
+The body of the HTTP request is the JSON representation of the record value. The header `Content-Type: application/json` is added to the HTTP request.
+
+Some other HTTP headers are added to the HTTP request:
+
+* `PulsarTopic`: the topic of the record
+* `PulsarKey`: the key of the record
+* `PulsarEventTime`: the event time of the record
+* `PulsarPublishTime`: the publish time of the record
+* `PulsarMessageId`: the ID of the message contained in the record
+* `PulsarProperties`-*: each record property is passed with the property name prefixed by `PulsarProperties-`
Review Comment:
```suggestion
* `PulsarProperties-*`: each record property is passed with the property name prefixed by `PulsarProperties-`
```
I'd prefer to quote all `PulsarProperties-*`. Here the `*` comes from regexp concepts that "all properties with prefix 'PulsarProperties-'"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a diff in pull request #18582: [improve][doc] Add doc for the HTTP Sink
Posted by GitBox <gi...@apache.org>.
momo-jun commented on code in PR #18582:
URL: https://github.com/apache/pulsar/pull/18582#discussion_r1031958418
##########
site2/docs/io-http-sink.md:
##########
@@ -0,0 +1,56 @@
+---
+id: io-http-sink
+title: HTTTP sink connector
+sidebar_label: "HTTP sink connector"
+---
+
+The HTTP sink connector pulls the records from Pulsar topics and makes a POST request to a configurable HTTP url (webhook).
Review Comment:
```suggestion
The HTTP sink connector pulls the records from Pulsar topics and makes a POST request to a configurable HTTP URL (webhook).
```
##########
site2/docs/io-http-sink.md:
##########
@@ -0,0 +1,56 @@
+---
+id: io-http-sink
+title: HTTTP sink connector
+sidebar_label: "HTTP sink connector"
+---
+
+The HTTP sink connector pulls the records from Pulsar topics and makes a POST request to a configurable HTTP url (webhook).
+
+The body of the HTTP request is the JSON representation of the record value. The header `Content-Type: application/json` is added to the HTTP request.
+
+Some other HTTP headers are added to the HTTP request:
+
+* `PulsarTopic`: the topic of the record
+* `PulsarKey`: the key of the record
+* `PulsarEventTime`: the event time of the record
+* `PulsarPublishTime`: the publish time of the record
+* `PulsarMessageId`: the ID of the message contained in the record
+* `PulsarProperties`-*: each record property is passed with the property name prefixed by `PulsarProperties-`
+
+## Configuration
+
+The configuration of the HTTP sink connector has the following properties.
+
+### Property
+
+| Name | Type | Required | Default | Description |
+|-----------|--------|----------|------------------|-------------|
+| `url` | String | false | http://localhost | The URL of the HTTP server |
+| `headers` | Map | false | empty map |The HDFS directory where files read from or written to. |
Review Comment:
```suggestion
| `headers` | Map | false | empty map |The HDFS directory where files are read from or written to. |
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a diff in pull request #18582: [feat][doc] Add doc for the HTTP Sink
Posted by GitBox <gi...@apache.org>.
momo-jun commented on code in PR #18582:
URL: https://github.com/apache/pulsar/pull/18582#discussion_r1035472196
##########
site2/docs/io-http-sink.md:
##########
@@ -0,0 +1,56 @@
+---
+id: io-http-sink
+title: HTTTP sink connector
+sidebar_label: "HTTP sink connector"
+---
+
+The HTTP sink connector pulls the records from Pulsar topics and makes a POST request to a configurable HTTP URL (webhook).
+
+The body of the HTTP request is the JSON representation of the record value. The header `Content-Type: application/json` is added to the HTTP request.
+
+Some other HTTP headers are added to the HTTP request:
+
+* `PulsarTopic`: the topic of the record
+* `PulsarKey`: the key of the record
+* `PulsarEventTime`: the event time of the record
+* `PulsarPublishTime`: the publish time of the record
+* `PulsarMessageId`: the ID of the message contained in the record
+* `PulsarProperties`-*: each record property is passed with the property name prefixed by `PulsarProperties-`
Review Comment:
`PulsarProperties-*` +1. Better to include the wildcard in the quotes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun commented on pull request #18582: [feat][doc] Add doc for the HTTP Sink
Posted by GitBox <gi...@apache.org>.
tisonkun commented on PR #18582:
URL: https://github.com/apache/pulsar/pull/18582#issuecomment-1337532920
Merging...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] cbornet commented on a diff in pull request #18582: [improve][doc] Add doc for the HTTP Sink
Posted by GitBox <gi...@apache.org>.
cbornet commented on code in PR #18582:
URL: https://github.com/apache/pulsar/pull/18582#discussion_r1032513369
##########
site2/docs/io-http-sink.md:
##########
@@ -0,0 +1,56 @@
+---
+id: io-http-sink
+title: HTTTP sink connector
+sidebar_label: "HTTP sink connector"
+---
+
+The HTTP sink connector pulls the records from Pulsar topics and makes a POST request to a configurable HTTP url (webhook).
+
+The body of the HTTP request is the JSON representation of the record value. The header `Content-Type: application/json` is added to the HTTP request.
+
+Some other HTTP headers are added to the HTTP request:
+
+* `PulsarTopic`: the topic of the record
+* `PulsarKey`: the key of the record
+* `PulsarEventTime`: the event time of the record
+* `PulsarPublishTime`: the publish time of the record
+* `PulsarMessageId`: the ID of the message contained in the record
+* `PulsarProperties`-*: each record property is passed with the property name prefixed by `PulsarProperties-`
+
+## Configuration
+
+The configuration of the HTTP sink connector has the following properties.
+
+### Property
+
+| Name | Type | Required | Default | Description |
+|-----------|--------|----------|------------------|-------------|
+| `url` | String | false | http://localhost | The URL of the HTTP server |
+| `headers` | Map | false | empty map |The HDFS directory where files read from or written to. |
Review Comment:
The description was an incorrect copy/paste. I fixed it, PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on pull request #18582: [improve][doc] Add doc for the HTTP Sink
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on PR #18582:
URL: https://github.com/apache/pulsar/pull/18582#issuecomment-1326929855
Thanks for your contribution! Writers will review this PR from technical writing pespectives after it gets technical reviews from engineers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on pull request #18582: [improve][doc] Add doc for the HTTP Sink
Posted by GitBox <gi...@apache.org>.
momo-jun commented on PR #18582:
URL: https://github.com/apache/pulsar/pull/18582#issuecomment-1326962825
@cbornet thanks for adding the docs. It deserves a `feature` in the PR title.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun commented on a diff in pull request #18582: [feat][doc] Add doc for the HTTP Sink
Posted by GitBox <gi...@apache.org>.
tisonkun commented on code in PR #18582:
URL: https://github.com/apache/pulsar/pull/18582#discussion_r1034628199
##########
site2/docs/io-http-sink.md:
##########
@@ -0,0 +1,56 @@
+---
+id: io-http-sink
+title: HTTTP sink connector
+sidebar_label: "HTTP sink connector"
Review Comment:
I think we don't show this page on the sidebar so this metadata is unnecessary. But it's not a blocker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a diff in pull request #18582: [improve][doc] Add doc for the HTTP Sink
Posted by GitBox <gi...@apache.org>.
momo-jun commented on code in PR #18582:
URL: https://github.com/apache/pulsar/pull/18582#discussion_r1031959039
##########
site2/docs/io-http-sink.md:
##########
@@ -0,0 +1,56 @@
+---
+id: io-http-sink
+title: HTTTP sink connector
+sidebar_label: "HTTP sink connector"
+---
+
+The HTTP sink connector pulls the records from Pulsar topics and makes a POST request to a configurable HTTP url (webhook).
+
+The body of the HTTP request is the JSON representation of the record value. The header `Content-Type: application/json` is added to the HTTP request.
+
+Some other HTTP headers are added to the HTTP request:
+
+* `PulsarTopic`: the topic of the record
+* `PulsarKey`: the key of the record
+* `PulsarEventTime`: the event time of the record
+* `PulsarPublishTime`: the publish time of the record
+* `PulsarMessageId`: the ID of the message contained in the record
+* `PulsarProperties`-*: each record property is passed with the property name prefixed by `PulsarProperties-`
+
+## Configuration
+
+The configuration of the HTTP sink connector has the following properties.
+
+### Property
+
+| Name | Type | Required | Default | Description |
+|-----------|--------|----------|------------------|-------------|
+| `url` | String | false | http://localhost | The URL of the HTTP server |
+| `headers` | Map | false | empty map |The HDFS directory where files read from or written to. |
Review Comment:
```suggestion
| `headers` | Map | false | empty map |The HDFS directory where files are read from or written to. |
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] cbornet commented on a diff in pull request #18582: [feat][doc] Add doc for the HTTP Sink
Posted by GitBox <gi...@apache.org>.
cbornet commented on code in PR #18582:
URL: https://github.com/apache/pulsar/pull/18582#discussion_r1034643592
##########
site2/docs/io-http-sink.md:
##########
@@ -0,0 +1,56 @@
+---
+id: io-http-sink
+title: HTTTP sink connector
+sidebar_label: "HTTP sink connector"
Review Comment:
Then it should be removed from all the connector pages I guess ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun commented on a diff in pull request #18582: [feat][doc] Add doc for the HTTP Sink
Posted by GitBox <gi...@apache.org>.
tisonkun commented on code in PR #18582:
URL: https://github.com/apache/pulsar/pull/18582#discussion_r1034688647
##########
site2/docs/io-http-sink.md:
##########
@@ -0,0 +1,56 @@
+---
+id: io-http-sink
+title: HTTTP sink connector
+sidebar_label: "HTTP sink connector"
Review Comment:
Yep. I comment here, and possibly we file an issue to track such a movement :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org