You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/12/15 11:18:01 UTC

[GitHub] [pulsar] Anonymitaet opened a new pull request #13336: [Doc] add explanations for broker entry metadata

Anonymitaet opened a new pull request #13336:
URL: https://github.com/apache/pulsar/pull/13336


   Add docs for https://github.com/apache/pulsar/pull/11553 and https://apache-pulsar.slack.com/archives/C5Z4T36F7/p1639493174300500


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] Anonymitaet commented on a change in pull request #13336: [Doc] add explanations for broker entry metadata

Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on a change in pull request #13336:
URL: https://github.com/apache/pulsar/pull/13336#discussion_r770311619



##########
File path: site2/docs/reference-configuration.md
##########
@@ -354,6 +354,8 @@ brokerServiceCompactionThresholdInBytes|If the estimated backlog size is greater
 | managedLedgerInfoCompressionType | Compression type of managed ledger information. <br><br>Available options are `NONE`, `LZ4`, `ZLIB`, `ZSTD`, and `SNAPPY`). <br><br>If this value is `NONE` or invalid, the `managedLedgerInfo` is not compressed. <br><br>**Note** that after enabling this configuration, if you want to degrade a broker, you need to change the value to `NONE` and make sure all ledger metadata is saved without compression. | None |
 | additionalServlets | Additional servlet name. <br><br>If you have multiple additional servlets, separate them by commas. <br><br>For example, additionalServlet_1, additionalServlet_2 | N/A |
 | additionalServletDirectory | Location of broker additional servlet NAR directory | ./brokerAdditionalServlet |
+brokerEntryMetadataInterceptors | Set broker entry metadata interceptors.<br /><br />Multiple interceptors should be separated by commas. <br /><br />Available values:<li>org.apache.pulsar.common.intercept.AppendBrokerTimestampMetadataInterceptor</li><li>org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor</li> <br /><br />Example<br />brokerEntryMetadataInterceptors=org.apache.pulsar.common.intercept.AppendBrokerTimestampMetadataInterceptor, org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor|N/A
+enableExposingBrokerEntryMetadataToClient|Whether to expose broker entry metadata to client or not.<br /><br />Available values:<li>true</li><li>false</li><br />Example<br />enableExposingBrokerEntryMetadataToClient=true  | false

Review comment:
       OK, I've updated to keep consistency, PTAL




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] Huanli-Meng commented on a change in pull request #13336: [Doc] add explanations for broker entry metadata

Posted by GitBox <gi...@apache.org>.
Huanli-Meng commented on a change in pull request #13336:
URL: https://github.com/apache/pulsar/pull/13336#discussion_r770154039



##########
File path: site2/docs/developing-binary-protocol.md
##########
@@ -63,6 +63,14 @@ It is created by the broker when the message arrived at the broker and passed wi
 | `broker_timestamp` | Optional        | The timestamp when a message arrived at the broker (`id est` as the number of milliseconds since January 1st, 1970 in UTC)      |
 | `index`            | Optional        | The index of the message. It is assigned by the broker.
 
+If you want use broker entry metadata for **brokers**, configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter in the `broker.conf` file.
+
+If you want use broker entry metadata for **consumers**:
+
+1. Use the client protocol version [no later than 18](https://github.com/apache/pulsar/blob/ca37e67211feda4f7e0984e6414e707f1c1dfd07/pulsar-common/src/main/proto/PulsarApi.proto#L259).
+   

Review comment:
       could we change `no later than 18` to `18 or higher`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] Anonymitaet commented on pull request #13336: [Doc] add explanations for broker entry metadata

Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on pull request #13336:
URL: https://github.com/apache/pulsar/pull/13336#issuecomment-994693233


   @BewareMyPower @codelipenghui @Huanli-Meng can you help review? Thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] BewareMyPower commented on a change in pull request #13336: [Doc] add explanations for broker entry metadata

Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on a change in pull request #13336:
URL: https://github.com/apache/pulsar/pull/13336#discussion_r770267038



##########
File path: site2/docs/reference-configuration.md
##########
@@ -354,6 +354,8 @@ brokerServiceCompactionThresholdInBytes|If the estimated backlog size is greater
 | managedLedgerInfoCompressionType | Compression type of managed ledger information. <br><br>Available options are `NONE`, `LZ4`, `ZLIB`, `ZSTD`, and `SNAPPY`). <br><br>If this value is `NONE` or invalid, the `managedLedgerInfo` is not compressed. <br><br>**Note** that after enabling this configuration, if you want to degrade a broker, you need to change the value to `NONE` and make sure all ledger metadata is saved without compression. | None |
 | additionalServlets | Additional servlet name. <br><br>If you have multiple additional servlets, separate them by commas. <br><br>For example, additionalServlet_1, additionalServlet_2 | N/A |
 | additionalServletDirectory | Location of broker additional servlet NAR directory | ./brokerAdditionalServlet |
+brokerEntryMetadataInterceptors | Set broker entry metadata interceptors.<br /><br />Multiple interceptors should be separated by commas. <br /><br />Available values:<li>org.apache.pulsar.common.intercept.AppendBrokerTimestampMetadataInterceptor</li><li>org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor</li> <br /><br />Example<br />brokerEntryMetadataInterceptors=org.apache.pulsar.common.intercept.AppendBrokerTimestampMetadataInterceptor, org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor|N/A
+enableExposingBrokerEntryMetadataToClient|Whether to expose broker entry metadata to client or not.<br /><br />Available values:<li>true</li><li>false</li><br />Example<br />enableExposingBrokerEntryMetadataToClient=true  | false

Review comment:
       Should we add the `|` at the head and tail though the current markdown also works? Because other configurations also use this style.
   
   ```
   | additionalServlets | ... |
   brokerEntryMetadataInterceptors | ...
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] Anonymitaet commented on a change in pull request #13336: [Doc] add explanations for broker entry metadata

Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on a change in pull request #13336:
URL: https://github.com/apache/pulsar/pull/13336#discussion_r770174526



##########
File path: site2/docs/developing-binary-protocol.md
##########
@@ -63,6 +63,14 @@ It is created by the broker when the message arrived at the broker and passed wi
 | `broker_timestamp` | Optional        | The timestamp when a message arrived at the broker (`id est` as the number of milliseconds since January 1st, 1970 in UTC)      |
 | `index`            | Optional        | The index of the message. It is assigned by the broker.
 
+If you want use broker entry metadata for **brokers**, configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter in the `broker.conf` file.
+
+If you want use broker entry metadata for **consumers**:
+
+1. Use the client protocol version [no later than 18](https://github.com/apache/pulsar/blob/ca37e67211feda4f7e0984e6414e707f1c1dfd07/pulsar-common/src/main/proto/PulsarApi.proto#L259).
+   

Review comment:
       make sense. I prefer `18 or later`, reason: https://docs.google.com/document/d/1lc5j4RtuLIzlEYCBo97AC8-U_3Erzs_lxpkDuseU0n4/edit#bookmark=id.tdt6lehtxogm




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] Shoothzj commented on a change in pull request #13336: [Doc] add explanations for broker entry metadata

Posted by GitBox <gi...@apache.org>.
Shoothzj commented on a change in pull request #13336:
URL: https://github.com/apache/pulsar/pull/13336#discussion_r770139434



##########
File path: site2/docs/developing-binary-protocol.md
##########
@@ -63,6 +63,14 @@ It is created by the broker when the message arrived at the broker and passed wi
 | `broker_timestamp` | Optional        | The timestamp when a message arrived at the broker (`id est` as the number of milliseconds since January 1st, 1970 in UTC)      |
 | `index`            | Optional        | The index of the message. It is assigned by the broker.
 
+If you want use broker entry metadata for **brokers**, configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter in the `broker.conf` file.
+
+If you want use broker entry metadata for **consumers**:

Review comment:
       want to

##########
File path: site2/docs/developing-binary-protocol.md
##########
@@ -63,6 +63,14 @@ It is created by the broker when the message arrived at the broker and passed wi
 | `broker_timestamp` | Optional        | The timestamp when a message arrived at the broker (`id est` as the number of milliseconds since January 1st, 1970 in UTC)      |
 | `index`            | Optional        | The index of the message. It is assigned by the broker.
 
+If you want use broker entry metadata for **brokers**, configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter in the `broker.conf` file.

Review comment:
       want to use




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] Anonymitaet commented on a change in pull request #13336: [Doc] add explanations for broker entry metadata

Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on a change in pull request #13336:
URL: https://github.com/apache/pulsar/pull/13336#discussion_r770311619



##########
File path: site2/docs/reference-configuration.md
##########
@@ -354,6 +354,8 @@ brokerServiceCompactionThresholdInBytes|If the estimated backlog size is greater
 | managedLedgerInfoCompressionType | Compression type of managed ledger information. <br><br>Available options are `NONE`, `LZ4`, `ZLIB`, `ZSTD`, and `SNAPPY`). <br><br>If this value is `NONE` or invalid, the `managedLedgerInfo` is not compressed. <br><br>**Note** that after enabling this configuration, if you want to degrade a broker, you need to change the value to `NONE` and make sure all ledger metadata is saved without compression. | None |
 | additionalServlets | Additional servlet name. <br><br>If you have multiple additional servlets, separate them by commas. <br><br>For example, additionalServlet_1, additionalServlet_2 | N/A |
 | additionalServletDirectory | Location of broker additional servlet NAR directory | ./brokerAdditionalServlet |
+brokerEntryMetadataInterceptors | Set broker entry metadata interceptors.<br /><br />Multiple interceptors should be separated by commas. <br /><br />Available values:<li>org.apache.pulsar.common.intercept.AppendBrokerTimestampMetadataInterceptor</li><li>org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor</li> <br /><br />Example<br />brokerEntryMetadataInterceptors=org.apache.pulsar.common.intercept.AppendBrokerTimestampMetadataInterceptor, org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor|N/A
+enableExposingBrokerEntryMetadataToClient|Whether to expose broker entry metadata to client or not.<br /><br />Available values:<li>true</li><li>false</li><br />Example<br />enableExposingBrokerEntryMetadataToClient=true  | false

Review comment:
       OK, I've updated, PTAL




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] Anonymitaet commented on a change in pull request #13336: [Doc] add explanations for broker entry metadata

Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on a change in pull request #13336:
URL: https://github.com/apache/pulsar/pull/13336#discussion_r770174148



##########
File path: site2/docs/developing-binary-protocol.md
##########
@@ -63,6 +63,14 @@ It is created by the broker when the message arrived at the broker and passed wi
 | `broker_timestamp` | Optional        | The timestamp when a message arrived at the broker (`id est` as the number of milliseconds since January 1st, 1970 in UTC)      |
 | `index`            | Optional        | The index of the message. It is assigned by the broker.
 
+If you want use broker entry metadata for **brokers**, configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter in the `broker.conf` file.

Review comment:
       @Shoothzj 
   oh, sorry my bad, a careless mistake. have updated. PTAL




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] Anonymitaet merged pull request #13336: [Doc] add explanations for broker entry metadata

Posted by GitBox <gi...@apache.org>.
Anonymitaet merged pull request #13336:
URL: https://github.com/apache/pulsar/pull/13336


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org