You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/06/27 12:18:14 UTC
[GitHub] [pulsar] ElationAndElation opened a new issue #7375: An option in Pulsar functions to define routing for output messages to partitions
ElationAndElation opened a new issue #7375:
URL: https://github.com/apache/pulsar/issues/7375
**Is your feature request related to a problem? Please describe.**
An option in Pulsar functions to define routing for output messages to partitions.
**Describe the solution you'd like**
In producer builder there is hashing scheme, expecting similar solution.
**Describe alternatives you've considered**
NA
**Additional context**
NA
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] ElationAndElation commented on issue #7375: An option in Pulsar functions to define routing for output messages to partitions
Posted by GitBox <gi...@apache.org>.
ElationAndElation commented on issue #7375:
URL: https://github.com/apache/pulsar/issues/7375#issuecomment-650924130
> @ElationAndElation
>
> You can use `Context.newOutputMessage()` to specify the topic (partition) to produce messages to. Does it meet your requirements?
In pulsar producer has hashing schema, using these schema pulsar topic partition are chosen for a particular message. Please refer below code. Lets say ptest has 4 partitions and hashing as Murmur3_32Hash, then total 5 messages are sent, for each message partition is chosen based on Hashing Scheme murmur. There is no need exclusive logic written by here to chose partition.
Whereas for functions in Context.newOutputMessage(), we have to write a exclusive logic to chose a partition or topic. So, looking for the feature as in producer.
PulsarClient client = PulsarClient.builder().serviceUrl("pulsar://localhost:6650").build();
Producer producer = client.newProducer()
.hashingScheme(HashingScheme.Murmur3_32Hash)
.topic("ptest")
.create();
for (int i = 1; i <= 5; i++) {
String key = "key-1"+System.currentTimeMillis();
if( i%2 == 0)
key = "key-2"+System.currentTimeMillis();
try {
MessageId message = producer.send(key);
} catch (PulsarClientException e ) {
e.printStackTrace();
}
}
producer.close();
client.close();
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] ElationAndElation edited a comment on issue #7375: An option in Pulsar functions to define routing for output messages to partitions
Posted by GitBox <gi...@apache.org>.
ElationAndElation edited a comment on issue #7375:
URL: https://github.com/apache/pulsar/issues/7375#issuecomment-650924130
> @ElationAndElation
>
> You can use `Context.newOutputMessage()` to specify the topic (partition) to produce messages to. Does it meet your requirements?
In pulsar producer has hashing schema, using these schema pulsar topic partition are chosen for a particular message. Please refer below code. Lets say ptest has 4 partitions and hashing as Murmur3_32Hash, then total 5 messages are sent, for each message partition is chosen based on Hashing Scheme murmur. There is no need exclusive logic written by here to chose partition.
Whereas for functions in Context.newOutputMessage(), we have to write a exclusive logic to chose a partition or topic. So, looking for the feature as in producer.
```
PulsarClient client = PulsarClient.builder().serviceUrl("pulsar://localhost:6650").build();
Producer producer = client.newProducer()
.hashingScheme(HashingScheme.Murmur3_32Hash)
.topic("ptest")
.create();
for (int i = 1; i <= 5; i++) {
String key = "key-1"+System.currentTimeMillis();
if( i%2 == 0)
key = "key-2"+System.currentTimeMillis();
try {
MessageId message = producer.send(key);
} catch (PulsarClientException e ) {
e.printStackTrace();
}
}
producer.close();
client.close();
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7375: Provide option to specify the hashing scheme for output topic producers in Pulsar functions
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #7375:
URL: https://github.com/apache/pulsar/issues/7375#issuecomment-652839009
@ElationAndElation I see. Thank you for the explanation! Will incorporate your comment into adding this feature.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7375: An option in Pulsar functions to define routing for output messages to partitions
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #7375:
URL: https://github.com/apache/pulsar/issues/7375#issuecomment-650861316
@ElationAndElation
You can use `Context.newOutputMessage()` to specify the topic (partition) to produce messages to. Does it meet your requirements?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] ElationAndElation edited a comment on issue #7375: An option in Pulsar functions to define routing for output messages to partitions
Posted by GitBox <gi...@apache.org>.
ElationAndElation edited a comment on issue #7375:
URL: https://github.com/apache/pulsar/issues/7375#issuecomment-650924130
> @ElationAndElation
>
> You can use `Context.newOutputMessage()` to specify the topic (partition) to produce messages to. Does it meet your requirements?
In pulsar producer has hashing schema, using these schema pulsar topic partition are chosen for a particular message. Please refer below code. Lets say ptest has 4 partitions and hashing as Murmur3_32Hash, then total 5 messages are sent, for each message partition is chosen based on Hashing Scheme murmur. There is no exclusive logic written by here to chose partition.
Whereas for functions in Context.newOutputMessage(), we have to write a exclusive logic to chose a partition or topic.
So, looking for the feature as in producer.
```
PulsarClient client = PulsarClient.builder().serviceUrl("pulsar://localhost:6650").build();
Producer producer = client.newProducer()
.hashingScheme(HashingScheme.Murmur3_32Hash)
.topic("ptest")
.create();
for (int i = 1; i <= 5; i++) {
String key = "key-1"+System.currentTimeMillis();
if( i%2 == 0)
key = "key-2"+System.currentTimeMillis();
try {
MessageId message = producer.send(key);
} catch (PulsarClientException e ) {
e.printStackTrace();
}
}
producer.close();
client.close();
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org