You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by anonymitaet _ <an...@hotmail.com> on 2019/11/04 14:24:11 UTC
[Community Weekly Update] 2019-10-26 ~ 2019-11-01
Dear Pulsar enthusiast,
This is the weekly community update for 2019-10-26 ~ 2019-11-01, with updates on new PIPs kicking in discussions of revisiting admin API and permissions, introducing package management in Pulsar Functions, a new Pulsar admin CLI `pulsarctl`, first apache release of Pulsar Manager, and more.
This Pulsar weekly update is also available at https://streamnative.io/weekly/2019/2019-11/2019-11-01-pulsar-weekly/.
===================
Pulsar Development
===================
- [Pulsar Manager] The release plan was settled down. Guangning is going to kick off the first release plan.
https://lists.apache.org/thread.html/b3f05f0f50a6b8c32536c5a29f8e1ce5c6efe42b2a6ccf91f28d33a0@%3Cdev.pulsar.apache.org%3E
- [PIP-47][Release] Time-Based Release Plan
PIP-47 was submitted for proposing moving towards a time-based release plan. So Pulsar development can enter a faster feedback cycle and users can benefit from features shipped quicker.
- https://lists.apache.org/thread.html/6509f012d52241090dd987609ed7eb27ab29af224217f6934cfee64c@%3Cdev.pulsar.apache.org%3E
- https://github.com/apache/pulsar/wiki/PIP-47%3A-Time-Based-Release-Plan
- [PIP-48][admin] Hierarchical admin API
Steven and Florentin from OVH kicked off a discussion for improving current admin API.
- https://lists.apache.org/thread.html/742ff9ffcc1f7212acf5e0eaae4bbd45c4754a43b7a36f22bd57578a@%3Cdev.pulsar.apache.org%3E
- https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api
- [PIP-49][admin] Permission levels and inheritance
Xiaolong started a proposal for revisiting the permission levels and inheritance in the Pulsar admin API.
- https://lists.apache.org/thread.html/d56048b007d713d7b927c37e4cdf23a353852b2f7d6491840f1e3e0f@%3Cdev.pulsar.apache.org%3E
- https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
- [PIP-50][Functions] Package Management
Yong proposed introducing a package management system for managing different versions of functions and connectors.
- https://lists.apache.org/thread.html/61a8138b0a304298d0de6178c5b870b38bd32bd762374486777f2c8c@%3Cdev.pulsar.apache.org%3E
- https://github.com/apache/pulsar/wiki/PIP-50%3A-Package-Management
- [Document] The broken links in the website have been fixed. Kudos to Guangning and Jennifer.
- [Document] A new discussion thread started to introduce versioning in managing Pulsar client API documentation.
- [Functions] Distributed the CA for KubernetesSecretsTokenAuthProvider (Release: 2.5.0)
- https://github.com/apache/pulsar/pull/5469
- https://lists.apache.org/thread.html/a6a91990907cf8d2458ff8b7e1e916a9bfb17ec0cd216d57c66ab96c@%3Cdev.pulsar.apache.org%3E
- [BookKeeper] Apache BookKeeper is voting its 4.10 release, which can unblock Pulsar 2.4.2 release. Thank you to Enrico from the BookKeeper community!
https://lists.apache.org/thread.html/d57f500fe8383903922fc4be3c3901fb0c00fda777672f32653c979e@%3Cdev.bookkeeper.apache.org%3E
===================
Notable Features
===================
- [Functions] Added deletion of state for Functions (Release: 2.5.0)
https://github.com/apache/pulsar/pull/5469
- [Functions] Distributed the CA for KubernetesSecretsTokenAuthProvider (Release: 2.5.0)
https://github.com/apache/pulsar/pull/5398
- [ZooKeeper] Bumped ZooKeeper to version 3.5.6
https://github.com/apache/pulsar/pull/5043
- [Functions] Function runtime pluggable
https://github.com/apache/pulsar/pull/5463
===================
Notable Bug Fix
===================
- [Client][Java] Wrongly report "3600 messages have timed-out" (Fixed, Release: 2.4.2 / 2.5.0)
https://github.com/apache/pulsar/pull/5477
- [Functions] Fixed Pulsar cannot load the customized SerDe (Fixed, Release: 2.4.2 / 2.5.0)
https://github.com/apache/pulsar/pull/5357
- [Authentication] Fixed broken custom auth-provider that uses authenticationData (Fixed, Release: 2.4.2 / 2.5.0)
https://github.com/apache/pulsar/pull/5462
- [Tiered Storage] Only seek when reading unexpected entry (Fixed, Release: 2.4.2 / 2.5.0)
https://github.com/apache/pulsar/pull/5356
- [Broker] Trim messages which less than mark delete position for message redelivery (Fixed, Release: 2.4.2 / 2.5.0)
https://github.com/apache/pulsar/pull/5378
- [Admin] Fix list non-persistent topics shows the persistent topics (Fixed, Release: 2.4.2 / 2.5.0)
https://github.com/apache/pulsar/pull/5502
===================
Ecosystem
===================
- [CLI] StreamNative open sourced a Pulsar Go admin client and a new CLI tool `pulsarctl` (built on the Pulsar Go admin client).
https://github.com/streamnative/pulsarctl
- [Hadoop-unit](https://github.com/jetoile/hadoop-unit) added the support for Pulsar and BookKeeper.
https://github.com/jetoile/hadoop-unit/blob/master/CHANGELOG.md#v36-20191101-1603-0000
- Pulsar.Client 0.8.0 was released with Reader support.
https://www.nuget.org/packages/Pulsar.Client/
===================
Event / News
===================
- [Meetup] A new Apache Pulsar meetup is coming in Shanghai (China) on November 16, which will feature adoption stories from China Telecom, Zhaopin, and TuyaSmart.
Signup link: https://www.eventbrite.com/e/apache-pulsar-meetup-shanghai-tickets-79293658467
===================
Blog / Article
===================
- [Video] Presentation "Apache Pulsar 101: architecture, concepts et comparaison" given by Quentin Adam & Steven Le Roux at DevFest Nantes 2019 was alive.
https://www.youtube.com/watch?v=5fqhT82wghY&list=PLuZ_sYdawLiUjPGPsOvBcgBxC6yP_HSA6&index=55&t=0s
If we miss anything, welcome to reply to this thread, thank you.
Cheers,
Sijie Guo, Yu Liu (@Anonymitaet)
From: anonymitaet _ <an...@hotmail.com>
Date: Monday, October 28, 2019 at 21:40
To: "users@pulsar.apache.org" <us...@pulsar.apache.org>, "dev@pulsar.apache.org" <de...@pulsar.apache.org>
Subject: [Community Weekly Update] 2019-10-19 ~ 2019-10-25
Dear Pulsar enthusiast,
This is the community weekly update for 2019-10-19 ~ 2019-10-25, which helps you quickly capture Pulsar's highlights and spot trends over last week, meanwhile strengthen the communication and connection within the Pulsar family.
All Pulsar weekly updates are available at https://streamnative.io/weekly/.
===================
Pulsar Development
===================
* [CI] ASF Jenkins is back to 'normal' after reverting 'Add default loader for latest pyyaml (#4974)' [1] in #5432 [2]. The problem was from the usage of pyyaml in python 2.7 causing function workers failing to start in integration tests. The committers started merging the pull requests.
[1] https://github.com/apache/pulsar/pull/4974
[2] https://github.com/apache/pulsar/pull/5432
* [PIP-43] The main logic for supporting producers to send message with different schemas was merged. With this change, Pulsar provides the capability of supporting event sourcing applications with different schemas. Kudos to Yi Tang!
https://github.com/apache/pulsar/pull/5443
* [Functions] Jerry Peng started the effort on refactoring functions runtime to make it pluggale. It will make the future development of adding a new runtime easier and more smoothly.
https://github.com/apache/pulsar/pull/5463
* [PIP-45] The second pull request for pluggable metadata interface is out to change the implementation of ManagedLedger to use MetadataStorre interface.
https://github.com/apache/pulsar/pull/5358
* [Key-Shared] Penghui kicked off the implementation of supporting sticky consumers in key_shared subscription. This provides a capability for consuming sub-streams from a given topic (partition) in order. It can be used in Flink integration for supporting flexiblle scaling up-and-down.
https://github.com/apache/pulsar/pull/5388
* [Transaction] The development of TC continues with adding topic ownership listener for bootstraping coordinator when a coordination topic is owned.
https://github.com/apache/pulsar/pull/5457
===================
Notable Features
===================
* [Client][Java] Add support for partitioned topic consumer seek by time. (Release: 2.5.0)
https://github.com/apache/pulsar/pull/5435
* [Functions] Make Function authentication provider pluggable. (Release: 2.5.0)
https://github.com/apache/pulsar/pull/5404
* [Client][Java] Support set read-position based on timestamp. (Release: 2.5.0)
https://github.com/apache/pulsar/pull/5075
===================
Notable Bug Fix
===================
* [Client][CGo] Return message ID for produced messages. (Fixed, Release: 2.5.0)
https://github.com/apache/pulsar/pull/4811
* [Broker] Fix potential deadlock that can occur in addConsumer. (Fixed, Release: 2.4.2 / 2.5.0)
https://github.com/apache/pulsar/pull/5371
* [Client][Java] Avoid leak on publish failure on batch message. (Fixed, Release: 2.4.2 / 2.5.0)
https://github.com/apache/pulsar/pull/5442
* [Broker] Fix: race condition: failed to read-more entries on dispatcher. (Fixed, Release: 2.4.2 / 2.5.0)
https://github.com/apache/pulsar/pull/5391
* [Client][Java] Fix message corruption on OOM for batch messages. (Fixed, Release: 2.4.2 / 2.5.0)
https://github.com/apache/pulsar/pull/5443
===================
Ecosystem
===================
* Pulsar.Client 0.7.0 was released with TLS and token authentication support.
https://www.nuget.org/packages/Pulsar.Client/
* More Pulsar tools, integrations, and resources can also be found at https://github.com/streamnative/awesome-pulsar.
===================
Event / News
===================
* Apache Pulsar gets more attention by giants like Splunk. On Oct 21, Splunk announced to acquire Streamlio to accelerate efforts in real-time stream processing and containerized multi-tenant cloud platform applications. Streamlio is powered by Apache Pulsar, specializing in designing and operating streaming data solutions at scale in demanding enterprise environments.
https://www.splunk.com/blog/2019/10/21/splunk-to-expand-streaming-expertise-announces-intent-to-acquire-streamlio-open-source-distributed-messaging-leader.html
* Paris Data Engineers (meetup)
Paris Data Engineers was held on Oct 22 in France. Quentin Adam talked about how CleverCloud is using Pulsar for scalable logs processing.
https://www.meetup.com/fr-FR/Paris-Data-Engineers/events/264819837/
===================
Blog / Article
===================
* Powering Tencent Billing Platform with Apache Pulsar (by Dezhi Liu)
https://streamnative.io/blog/tech/2019-10-22-powering-tencent-billing-platform-with-apache-pulsar/
* How to use Apache Pulsar Manager with HerdDB (by Enrico Olivelli)
https://medium.com/streamnative/how-to-use-apache-pulsar-manager-with-herddb-dd265c955ca4
* Why Nutanix Beam went ahead with Apache Pulsar instead of Apache Kafka? (by Yuvaraj Loganathan)
https://medium.com/@yuvarajl/why-nutanix-beam-went-ahead-with-apache-pulsar-instead-of-apache-kafka-1415f592dbbb
* Basic Pulsar producer and consumer (by Thomas Memenga)
https://www.syscrest.com/2019/10/basic-pulsar-producer-and-consumer-json-helm-kubernetes/
If we miss anything, welcome to reply to this thread, thank you.
Cheers,
Sijie Guo, Yu Liu (@Anonymitaet)
Re: [Community Weekly Update] 2019-10-26 ~ 2019-11-01
Posted by Yuva raj <uv...@gmail.com>.
Thank You anonymitaet
On Mon, 4 Nov 2019 at 18:00, anonymitaet _ <an...@hotmail.com> wrote:
>
> Dear Pulsar enthusiast,
>
>
>
> This is the weekly community update for 2019-10-26 ~ 2019-11-01, with updates on new PIPs kicking in discussions of revisiting admin API and permissions, introducing package management in Pulsar Functions, a new Pulsar admin CLI `pulsarctl`, first apache release of Pulsar Manager, and more.
>
>
>
> This Pulsar weekly update is also available at https://streamnative.io/weekly/2019/2019-11/2019-11-01-pulsar-weekly/.
>
>
>
> ===================
>
> Pulsar Development
>
> ===================
>
>
>
> - [Pulsar Manager] The release plan was settled down. Guangning is going to kick off the first release plan.
>
>
>
> https://lists.apache.org/thread.html/b3f05f0f50a6b8c32536c5a29f8e1ce5c6efe42b2a6ccf91f28d33a0@%3Cdev.pulsar.apache.org%3E
>
>
>
> - [PIP-47][Release] Time-Based Release Plan
>
>
>
> PIP-47 was submitted for proposing moving towards a time-based release plan. So Pulsar development can enter a faster feedback cycle and users can benefit from features shipped quicker.
>
>
>
> - https://lists.apache.org/thread.html/6509f012d52241090dd987609ed7eb27ab29af224217f6934cfee64c@%3Cdev.pulsar.apache.org%3E
>
>
>
> - https://github.com/apache/pulsar/wiki/PIP-47%3A-Time-Based-Release-Plan
>
>
>
> - [PIP-48][admin] Hierarchical admin API
>
>
>
> Steven and Florentin from OVH kicked off a discussion for improving current admin API.
>
>
>
> - https://lists.apache.org/thread.html/742ff9ffcc1f7212acf5e0eaae4bbd45c4754a43b7a36f22bd57578a@%3Cdev.pulsar.apache.org%3E
>
>
>
> - https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api
>
>
>
> - [PIP-49][admin] Permission levels and inheritance
>
>
>
> Xiaolong started a proposal for revisiting the permission levels and inheritance in the Pulsar admin API.
>
>
>
> - https://lists.apache.org/thread.html/d56048b007d713d7b927c37e4cdf23a353852b2f7d6491840f1e3e0f@%3Cdev.pulsar.apache.org%3E
>
>
>
> - https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
>
>
>
> - [PIP-50][Functions] Package Management
>
>
>
> Yong proposed introducing a package management system for managing different versions of functions and connectors.
>
>
>
> - https://lists.apache.org/thread.html/61a8138b0a304298d0de6178c5b870b38bd32bd762374486777f2c8c@%3Cdev.pulsar.apache.org%3E
>
>
>
> - https://github.com/apache/pulsar/wiki/PIP-50%3A-Package-Management
>
>
>
> - [Document] The broken links in the website have been fixed. Kudos to Guangning and Jennifer.
>
>
>
> - [Document] A new discussion thread started to introduce versioning in managing Pulsar client API documentation.
>
>
>
> - [Functions] Distributed the CA for KubernetesSecretsTokenAuthProvider (Release: 2.5.0)
>
>
>
> - https://github.com/apache/pulsar/pull/5469
>
>
>
> - https://lists.apache.org/thread.html/a6a91990907cf8d2458ff8b7e1e916a9bfb17ec0cd216d57c66ab96c@%3Cdev.pulsar.apache.org%3E
>
>
>
> - [BookKeeper] Apache BookKeeper is voting its 4.10 release, which can unblock Pulsar 2.4.2 release. Thank you to Enrico from the BookKeeper community!
>
>
>
> https://lists.apache.org/thread.html/d57f500fe8383903922fc4be3c3901fb0c00fda777672f32653c979e@%3Cdev.bookkeeper.apache.org%3E
>
>
>
> ===================
>
> Notable Features
>
> ===================
>
>
>
> - [Functions] Added deletion of state for Functions (Release: 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5469
>
>
>
> - [Functions] Distributed the CA for KubernetesSecretsTokenAuthProvider (Release: 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5398
>
>
>
> - [ZooKeeper] Bumped ZooKeeper to version 3.5.6
>
>
>
> https://github.com/apache/pulsar/pull/5043
>
>
>
> - [Functions] Function runtime pluggable
>
>
>
> https://github.com/apache/pulsar/pull/5463
>
>
>
> ===================
>
> Notable Bug Fix
>
> ===================
>
>
>
> - [Client][Java] Wrongly report "3600 messages have timed-out" (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5477
>
>
>
> - [Functions] Fixed Pulsar cannot load the customized SerDe (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5357
>
>
>
> - [Authentication] Fixed broken custom auth-provider that uses authenticationData (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5462
>
>
>
> - [Tiered Storage] Only seek when reading unexpected entry (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5356
>
>
>
> - [Broker] Trim messages which less than mark delete position for message redelivery (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5378
>
>
>
> - [Admin] Fix list non-persistent topics shows the persistent topics (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5502
>
>
>
> ===================
>
> Ecosystem
>
> ===================
>
>
>
> - [CLI] StreamNative open sourced a Pulsar Go admin client and a new CLI tool `pulsarctl` (built on the Pulsar Go admin client).
>
>
>
> https://github.com/streamnative/pulsarctl
>
>
>
> - [Hadoop-unit](https://github.com/jetoile/hadoop-unit) added the support for Pulsar and BookKeeper.
>
>
>
> https://github.com/jetoile/hadoop-unit/blob/master/CHANGELOG.md#v36-20191101-1603-0000
>
>
>
> - Pulsar.Client 0.8.0 was released with Reader support.
>
>
>
> https://www.nuget.org/packages/Pulsar.Client/
>
>
>
> ===================
>
> Event / News
>
> ===================
>
>
>
> - [Meetup] A new Apache Pulsar meetup is coming in Shanghai (China) on November 16, which will feature adoption stories from China Telecom, Zhaopin, and TuyaSmart.
>
>
>
> Signup link: https://www.eventbrite.com/e/apache-pulsar-meetup-shanghai-tickets-79293658467
>
>
>
> ===================
>
> Blog / Article
>
> ===================
>
>
>
> - [Video] Presentation "Apache Pulsar 101: architecture, concepts et comparaison" given by Quentin Adam & Steven Le Roux at DevFest Nantes 2019 was alive.
>
>
>
> https://www.youtube.com/watch?v=5fqhT82wghY&list=PLuZ_sYdawLiUjPGPsOvBcgBxC6yP_HSA6&index=55&t=0s
>
>
>
> If we miss anything, welcome to reply to this thread, thank you.
>
>
>
> Cheers,
>
>
>
> Sijie Guo, Yu Liu (@Anonymitaet)
>
>
>
> From: anonymitaet _ <an...@hotmail.com>
> Date: Monday, October 28, 2019 at 21:40
> To: "users@pulsar.apache.org" <us...@pulsar.apache.org>, "dev@pulsar.apache.org" <de...@pulsar.apache.org>
> Subject: [Community Weekly Update] 2019-10-19 ~ 2019-10-25
>
>
>
> Dear Pulsar enthusiast,
>
>
>
> This is the community weekly update for 2019-10-19 ~ 2019-10-25, which helps you quickly capture Pulsar's highlights and spot trends over last week, meanwhile strengthen the communication and connection within the Pulsar family.
>
>
>
> All Pulsar weekly updates are available at https://streamnative.io/weekly/.
>
>
>
> ===================
>
> Pulsar Development
>
> ===================
>
>
>
> * [CI] ASF Jenkins is back to 'normal' after reverting 'Add default loader for latest pyyaml (#4974)' [1] in #5432 [2]. The problem was from the usage of pyyaml in python 2.7 causing function workers failing to start in integration tests. The committers started merging the pull requests.
>
>
>
> [1] https://github.com/apache/pulsar/pull/4974
>
>
>
> [2] https://github.com/apache/pulsar/pull/5432
>
>
>
> * [PIP-43] The main logic for supporting producers to send message with different schemas was merged. With this change, Pulsar provides the capability of supporting event sourcing applications with different schemas. Kudos to Yi Tang!
>
>
>
> https://github.com/apache/pulsar/pull/5443
>
>
>
> * [Functions] Jerry Peng started the effort on refactoring functions runtime to make it pluggale. It will make the future development of adding a new runtime easier and more smoothly.
>
>
>
> https://github.com/apache/pulsar/pull/5463
>
>
>
> * [PIP-45] The second pull request for pluggable metadata interface is out to change the implementation of ManagedLedger to use MetadataStorre interface.
>
>
>
> https://github.com/apache/pulsar/pull/5358
>
>
>
> * [Key-Shared] Penghui kicked off the implementation of supporting sticky consumers in key_shared subscription. This provides a capability for consuming sub-streams from a given topic (partition) in order. It can be used in Flink integration for supporting flexiblle scaling up-and-down.
>
>
>
> https://github.com/apache/pulsar/pull/5388
>
>
>
> * [Transaction] The development of TC continues with adding topic ownership listener for bootstraping coordinator when a coordination topic is owned.
>
>
>
> https://github.com/apache/pulsar/pull/5457
>
>
>
> ===================
>
> Notable Features
>
> ===================
>
>
>
> * [Client][Java] Add support for partitioned topic consumer seek by time. (Release: 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5435
>
>
>
> * [Functions] Make Function authentication provider pluggable. (Release: 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5404
>
>
>
> * [Client][Java] Support set read-position based on timestamp. (Release: 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5075
>
>
>
> ===================
>
> Notable Bug Fix
>
> ===================
>
>
>
> * [Client][CGo] Return message ID for produced messages. (Fixed, Release: 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/4811
>
>
>
> * [Broker] Fix potential deadlock that can occur in addConsumer. (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5371
>
>
>
> * [Client][Java] Avoid leak on publish failure on batch message. (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5442
>
>
>
> * [Broker] Fix: race condition: failed to read-more entries on dispatcher. (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5391
>
>
>
> * [Client][Java] Fix message corruption on OOM for batch messages. (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5443
>
>
>
> ===================
>
> Ecosystem
>
> ===================
>
>
>
> * Pulsar.Client 0.7.0 was released with TLS and token authentication support.
>
>
>
> https://www.nuget.org/packages/Pulsar.Client/
>
>
>
> * More Pulsar tools, integrations, and resources can also be found at https://github.com/streamnative/awesome-pulsar.
>
>
>
> ===================
>
> Event / News
>
> ===================
>
>
>
> * Apache Pulsar gets more attention by giants like Splunk. On Oct 21, Splunk announced to acquire Streamlio to accelerate efforts in real-time stream processing and containerized multi-tenant cloud platform applications. Streamlio is powered by Apache Pulsar, specializing in designing and operating streaming data solutions at scale in demanding enterprise environments.
>
>
>
> https://www.splunk.com/blog/2019/10/21/splunk-to-expand-streaming-expertise-announces-intent-to-acquire-streamlio-open-source-distributed-messaging-leader.html
>
>
>
> * Paris Data Engineers (meetup)
>
>
>
> Paris Data Engineers was held on Oct 22 in France. Quentin Adam talked about how CleverCloud is using Pulsar for scalable logs processing.
>
>
>
> https://www.meetup.com/fr-FR/Paris-Data-Engineers/events/264819837/
>
>
>
> ===================
>
> Blog / Article
>
> ===================
>
>
>
> * Powering Tencent Billing Platform with Apache Pulsar (by Dezhi Liu)
>
>
>
> https://streamnative.io/blog/tech/2019-10-22-powering-tencent-billing-platform-with-apache-pulsar/
>
>
>
> * How to use Apache Pulsar Manager with HerdDB (by Enrico Olivelli)
>
>
>
> https://medium.com/streamnative/how-to-use-apache-pulsar-manager-with-herddb-dd265c955ca4
>
>
>
> * Why Nutanix Beam went ahead with Apache Pulsar instead of Apache Kafka? (by Yuvaraj Loganathan)
>
>
>
> https://medium.com/@yuvarajl/why-nutanix-beam-went-ahead-with-apache-pulsar-instead-of-apache-kafka-1415f592dbbb
>
>
>
> * Basic Pulsar producer and consumer (by Thomas Memenga)
>
>
>
> https://www.syscrest.com/2019/10/basic-pulsar-producer-and-consumer-json-helm-kubernetes/
>
>
>
> If we miss anything, welcome to reply to this thread, thank you.
>
>
>
> Cheers,
>
>
>
> Sijie Guo, Yu Liu (@Anonymitaet)
>
>
>
>
--
Thanks
Yuvaraj L
Re: [Community Weekly Update] 2019-10-26 ~ 2019-11-01
Posted by Yuva raj <uv...@gmail.com>.
Thank You anonymitaet
On Mon, 4 Nov 2019 at 18:00, anonymitaet _ <an...@hotmail.com> wrote:
>
> Dear Pulsar enthusiast,
>
>
>
> This is the weekly community update for 2019-10-26 ~ 2019-11-01, with updates on new PIPs kicking in discussions of revisiting admin API and permissions, introducing package management in Pulsar Functions, a new Pulsar admin CLI `pulsarctl`, first apache release of Pulsar Manager, and more.
>
>
>
> This Pulsar weekly update is also available at https://streamnative.io/weekly/2019/2019-11/2019-11-01-pulsar-weekly/.
>
>
>
> ===================
>
> Pulsar Development
>
> ===================
>
>
>
> - [Pulsar Manager] The release plan was settled down. Guangning is going to kick off the first release plan.
>
>
>
> https://lists.apache.org/thread.html/b3f05f0f50a6b8c32536c5a29f8e1ce5c6efe42b2a6ccf91f28d33a0@%3Cdev.pulsar.apache.org%3E
>
>
>
> - [PIP-47][Release] Time-Based Release Plan
>
>
>
> PIP-47 was submitted for proposing moving towards a time-based release plan. So Pulsar development can enter a faster feedback cycle and users can benefit from features shipped quicker.
>
>
>
> - https://lists.apache.org/thread.html/6509f012d52241090dd987609ed7eb27ab29af224217f6934cfee64c@%3Cdev.pulsar.apache.org%3E
>
>
>
> - https://github.com/apache/pulsar/wiki/PIP-47%3A-Time-Based-Release-Plan
>
>
>
> - [PIP-48][admin] Hierarchical admin API
>
>
>
> Steven and Florentin from OVH kicked off a discussion for improving current admin API.
>
>
>
> - https://lists.apache.org/thread.html/742ff9ffcc1f7212acf5e0eaae4bbd45c4754a43b7a36f22bd57578a@%3Cdev.pulsar.apache.org%3E
>
>
>
> - https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api
>
>
>
> - [PIP-49][admin] Permission levels and inheritance
>
>
>
> Xiaolong started a proposal for revisiting the permission levels and inheritance in the Pulsar admin API.
>
>
>
> - https://lists.apache.org/thread.html/d56048b007d713d7b927c37e4cdf23a353852b2f7d6491840f1e3e0f@%3Cdev.pulsar.apache.org%3E
>
>
>
> - https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
>
>
>
> - [PIP-50][Functions] Package Management
>
>
>
> Yong proposed introducing a package management system for managing different versions of functions and connectors.
>
>
>
> - https://lists.apache.org/thread.html/61a8138b0a304298d0de6178c5b870b38bd32bd762374486777f2c8c@%3Cdev.pulsar.apache.org%3E
>
>
>
> - https://github.com/apache/pulsar/wiki/PIP-50%3A-Package-Management
>
>
>
> - [Document] The broken links in the website have been fixed. Kudos to Guangning and Jennifer.
>
>
>
> - [Document] A new discussion thread started to introduce versioning in managing Pulsar client API documentation.
>
>
>
> - [Functions] Distributed the CA for KubernetesSecretsTokenAuthProvider (Release: 2.5.0)
>
>
>
> - https://github.com/apache/pulsar/pull/5469
>
>
>
> - https://lists.apache.org/thread.html/a6a91990907cf8d2458ff8b7e1e916a9bfb17ec0cd216d57c66ab96c@%3Cdev.pulsar.apache.org%3E
>
>
>
> - [BookKeeper] Apache BookKeeper is voting its 4.10 release, which can unblock Pulsar 2.4.2 release. Thank you to Enrico from the BookKeeper community!
>
>
>
> https://lists.apache.org/thread.html/d57f500fe8383903922fc4be3c3901fb0c00fda777672f32653c979e@%3Cdev.bookkeeper.apache.org%3E
>
>
>
> ===================
>
> Notable Features
>
> ===================
>
>
>
> - [Functions] Added deletion of state for Functions (Release: 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5469
>
>
>
> - [Functions] Distributed the CA for KubernetesSecretsTokenAuthProvider (Release: 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5398
>
>
>
> - [ZooKeeper] Bumped ZooKeeper to version 3.5.6
>
>
>
> https://github.com/apache/pulsar/pull/5043
>
>
>
> - [Functions] Function runtime pluggable
>
>
>
> https://github.com/apache/pulsar/pull/5463
>
>
>
> ===================
>
> Notable Bug Fix
>
> ===================
>
>
>
> - [Client][Java] Wrongly report "3600 messages have timed-out" (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5477
>
>
>
> - [Functions] Fixed Pulsar cannot load the customized SerDe (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5357
>
>
>
> - [Authentication] Fixed broken custom auth-provider that uses authenticationData (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5462
>
>
>
> - [Tiered Storage] Only seek when reading unexpected entry (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5356
>
>
>
> - [Broker] Trim messages which less than mark delete position for message redelivery (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5378
>
>
>
> - [Admin] Fix list non-persistent topics shows the persistent topics (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5502
>
>
>
> ===================
>
> Ecosystem
>
> ===================
>
>
>
> - [CLI] StreamNative open sourced a Pulsar Go admin client and a new CLI tool `pulsarctl` (built on the Pulsar Go admin client).
>
>
>
> https://github.com/streamnative/pulsarctl
>
>
>
> - [Hadoop-unit](https://github.com/jetoile/hadoop-unit) added the support for Pulsar and BookKeeper.
>
>
>
> https://github.com/jetoile/hadoop-unit/blob/master/CHANGELOG.md#v36-20191101-1603-0000
>
>
>
> - Pulsar.Client 0.8.0 was released with Reader support.
>
>
>
> https://www.nuget.org/packages/Pulsar.Client/
>
>
>
> ===================
>
> Event / News
>
> ===================
>
>
>
> - [Meetup] A new Apache Pulsar meetup is coming in Shanghai (China) on November 16, which will feature adoption stories from China Telecom, Zhaopin, and TuyaSmart.
>
>
>
> Signup link: https://www.eventbrite.com/e/apache-pulsar-meetup-shanghai-tickets-79293658467
>
>
>
> ===================
>
> Blog / Article
>
> ===================
>
>
>
> - [Video] Presentation "Apache Pulsar 101: architecture, concepts et comparaison" given by Quentin Adam & Steven Le Roux at DevFest Nantes 2019 was alive.
>
>
>
> https://www.youtube.com/watch?v=5fqhT82wghY&list=PLuZ_sYdawLiUjPGPsOvBcgBxC6yP_HSA6&index=55&t=0s
>
>
>
> If we miss anything, welcome to reply to this thread, thank you.
>
>
>
> Cheers,
>
>
>
> Sijie Guo, Yu Liu (@Anonymitaet)
>
>
>
> From: anonymitaet _ <an...@hotmail.com>
> Date: Monday, October 28, 2019 at 21:40
> To: "users@pulsar.apache.org" <us...@pulsar.apache.org>, "dev@pulsar.apache.org" <de...@pulsar.apache.org>
> Subject: [Community Weekly Update] 2019-10-19 ~ 2019-10-25
>
>
>
> Dear Pulsar enthusiast,
>
>
>
> This is the community weekly update for 2019-10-19 ~ 2019-10-25, which helps you quickly capture Pulsar's highlights and spot trends over last week, meanwhile strengthen the communication and connection within the Pulsar family.
>
>
>
> All Pulsar weekly updates are available at https://streamnative.io/weekly/.
>
>
>
> ===================
>
> Pulsar Development
>
> ===================
>
>
>
> * [CI] ASF Jenkins is back to 'normal' after reverting 'Add default loader for latest pyyaml (#4974)' [1] in #5432 [2]. The problem was from the usage of pyyaml in python 2.7 causing function workers failing to start in integration tests. The committers started merging the pull requests.
>
>
>
> [1] https://github.com/apache/pulsar/pull/4974
>
>
>
> [2] https://github.com/apache/pulsar/pull/5432
>
>
>
> * [PIP-43] The main logic for supporting producers to send message with different schemas was merged. With this change, Pulsar provides the capability of supporting event sourcing applications with different schemas. Kudos to Yi Tang!
>
>
>
> https://github.com/apache/pulsar/pull/5443
>
>
>
> * [Functions] Jerry Peng started the effort on refactoring functions runtime to make it pluggale. It will make the future development of adding a new runtime easier and more smoothly.
>
>
>
> https://github.com/apache/pulsar/pull/5463
>
>
>
> * [PIP-45] The second pull request for pluggable metadata interface is out to change the implementation of ManagedLedger to use MetadataStorre interface.
>
>
>
> https://github.com/apache/pulsar/pull/5358
>
>
>
> * [Key-Shared] Penghui kicked off the implementation of supporting sticky consumers in key_shared subscription. This provides a capability for consuming sub-streams from a given topic (partition) in order. It can be used in Flink integration for supporting flexiblle scaling up-and-down.
>
>
>
> https://github.com/apache/pulsar/pull/5388
>
>
>
> * [Transaction] The development of TC continues with adding topic ownership listener for bootstraping coordinator when a coordination topic is owned.
>
>
>
> https://github.com/apache/pulsar/pull/5457
>
>
>
> ===================
>
> Notable Features
>
> ===================
>
>
>
> * [Client][Java] Add support for partitioned topic consumer seek by time. (Release: 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5435
>
>
>
> * [Functions] Make Function authentication provider pluggable. (Release: 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5404
>
>
>
> * [Client][Java] Support set read-position based on timestamp. (Release: 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5075
>
>
>
> ===================
>
> Notable Bug Fix
>
> ===================
>
>
>
> * [Client][CGo] Return message ID for produced messages. (Fixed, Release: 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/4811
>
>
>
> * [Broker] Fix potential deadlock that can occur in addConsumer. (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5371
>
>
>
> * [Client][Java] Avoid leak on publish failure on batch message. (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5442
>
>
>
> * [Broker] Fix: race condition: failed to read-more entries on dispatcher. (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5391
>
>
>
> * [Client][Java] Fix message corruption on OOM for batch messages. (Fixed, Release: 2.4.2 / 2.5.0)
>
>
>
> https://github.com/apache/pulsar/pull/5443
>
>
>
> ===================
>
> Ecosystem
>
> ===================
>
>
>
> * Pulsar.Client 0.7.0 was released with TLS and token authentication support.
>
>
>
> https://www.nuget.org/packages/Pulsar.Client/
>
>
>
> * More Pulsar tools, integrations, and resources can also be found at https://github.com/streamnative/awesome-pulsar.
>
>
>
> ===================
>
> Event / News
>
> ===================
>
>
>
> * Apache Pulsar gets more attention by giants like Splunk. On Oct 21, Splunk announced to acquire Streamlio to accelerate efforts in real-time stream processing and containerized multi-tenant cloud platform applications. Streamlio is powered by Apache Pulsar, specializing in designing and operating streaming data solutions at scale in demanding enterprise environments.
>
>
>
> https://www.splunk.com/blog/2019/10/21/splunk-to-expand-streaming-expertise-announces-intent-to-acquire-streamlio-open-source-distributed-messaging-leader.html
>
>
>
> * Paris Data Engineers (meetup)
>
>
>
> Paris Data Engineers was held on Oct 22 in France. Quentin Adam talked about how CleverCloud is using Pulsar for scalable logs processing.
>
>
>
> https://www.meetup.com/fr-FR/Paris-Data-Engineers/events/264819837/
>
>
>
> ===================
>
> Blog / Article
>
> ===================
>
>
>
> * Powering Tencent Billing Platform with Apache Pulsar (by Dezhi Liu)
>
>
>
> https://streamnative.io/blog/tech/2019-10-22-powering-tencent-billing-platform-with-apache-pulsar/
>
>
>
> * How to use Apache Pulsar Manager with HerdDB (by Enrico Olivelli)
>
>
>
> https://medium.com/streamnative/how-to-use-apache-pulsar-manager-with-herddb-dd265c955ca4
>
>
>
> * Why Nutanix Beam went ahead with Apache Pulsar instead of Apache Kafka? (by Yuvaraj Loganathan)
>
>
>
> https://medium.com/@yuvarajl/why-nutanix-beam-went-ahead-with-apache-pulsar-instead-of-apache-kafka-1415f592dbbb
>
>
>
> * Basic Pulsar producer and consumer (by Thomas Memenga)
>
>
>
> https://www.syscrest.com/2019/10/basic-pulsar-producer-and-consumer-json-helm-kubernetes/
>
>
>
> If we miss anything, welcome to reply to this thread, thank you.
>
>
>
> Cheers,
>
>
>
> Sijie Guo, Yu Liu (@Anonymitaet)
>
>
>
>
--
Thanks
Yuvaraj L