You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "asafm (via GitHub)" <gi...@apache.org> on 2024/04/10 13:42:38 UTC
Re: [I] PIP-261: Restructure Getting Started section [pulsar]
asafm commented on issue #19912:
URL: https://github.com/apache/pulsar/issues/19912#issuecomment-2047582554
My plan - TOC - for the documentation site:
### Quick-start Guides
<aside>
💡 The basic premise in a quick-start guide is to cater for those who want to learn by “hand”, thus we’ll learn by running a ready-made application, or building it piece by piece. We’ll have a couple of applications to showcase several features (e.g. streaming, job-queue)
</aside>
- “Feel Pulsar” using CLI. Audience: both dev & ops
- Run pulsar using Docker; produce and consume messages using the command line
- Developer Quick-start.
- Each section will be an app showcasing pulsar features, starting from:
- a basic job queue - messaging use-case
- we’ll build 2 microservices: one that produces messages into a work queue, the other consuming them
- we’ll explore the producer & consumer basic API
- We’ll show the Shared & Failover subscription modes:
- what happens when I add a new consumer instance or when I kill one.
- then a basic streaming app (like Kafka)
- we’ll explore the key_shared & failover subscription
- we’ll explore the reader API
- we’ll observe that the messages will be consumed in the right order
- then switch to the async API - completable future, MessageListener
- we’ll first focus on Java, as this is the most adopted language
- Operator Quick-start.
- “Feel Pulsar” using CLI
- Run pulsar using K8s; produce and consume messages using command line
- Pulsar Show-case
- Deploy
- Deploy Pulsar to K8s - step-by-step instructions
- Deploy Grafana and Prometheus using helm charts to showcase its ready-made dashboard
- Deploy demo-data-generator app (docker image) and it’s matching Grafana dashboard
- Deploy demo-data-consumer-app
- Steps
- Go to dashboards to see it’s working
- Scale Pulsar pods from 3 to 6 to showcase load-balancer
- Downscale Pulsar and BK to 2 to show all still works
- …
### Developer Guide
- Overview
- What is Pulsar in high level, and it is capable of doing
- High level description of Pulsar
- Short typical use case description
- Exhaustive list of Pulsar features, featuring 1-2 paragraph for each
- Architecture
- High level description of Pulsar’s architecture - 1 pager
- Pulsar Concepts
- Topics, Namespace and Tenant
- Normal topic and Partitioned Topic
- Quick reference of persistent vs non-persistent, and link to Pulsar Advanced features where it will be explained more thoroughly
- Ack quorum, Write quorum (important config)
- Namespaces - quick explanation of what it can do (like a feature list), with links to more detailed explanation in Pulsar Advanced page
- Tenant - quick explanation of what it is, and it can do with link to a more detailed page.
- Message retention and expiry
- Producer
- It’s role;
- Different access modes
- Routing modes for partitioned topics
- Quick listing of advanced features : 1-2 sentence per each one with links to a more detailed page in other sections
- Batching
- Chunking
- Working with Schema
- Producing Guarantees
- deduplicate? short explanation
- Consumer & Subscription
- How message consumption works in Pulsar
- What is a subscription and what is a consumer and the links between them
- Guarantees - like at-least once consumption
- Subscription Types
- Streaming-based consumption
- Cumulative acknowledgment
- Exclusive Subscription type
- Failover Subscription type
- Individual-based consumption
- Individual acknowledgment
- Shared subscription type
- Key-shared subscription type
- quick explanation about the default hashing scheme, and link to `Hashing Schemes for Key Shared Subscription` in Advanced Subscriptions
- Reader
- What is a reader and it is used for
- Quick explanation that it uses non-durable subscription and link to it in advanced
- Multi-topic subscription
- Processing Failures
- Negative Acknowledgment
- including config for getting this message with fixed delay
- Retries
- Exponential back-off retry;
- dead letter queue
- retry topic
- Acknowledgment Timeout
- Delayed Message Delivery
- Pulsar Advanced Features
- Performance-related
- Batching
- Chunking
- Non-persistent Topics
- Namespaces
- Multi-tenancy
- Advanced Consumption features
- Hashing Schemes for Key Shared Subscription
- Non-durable subscription
- System Topics
- Message Deduplication
- Message Schema (Pulsar Schema)
- Tiered Storage
- Topic Compaction
- Transactions
- Pulsar Eco system
- Pulsar Functions
- Pulsar I/O
- Pulsar SQL
- Pulsar Proxy
- 3rd party adaptors
- Client Libraries
- Pulsar Client…
- Pulsar Admin Client
-
### Operator Guide
- Operating Pulsar
- Features
- Tiered Storage
- Geo Replication
- Load Balancing
- Security
- Authentication and Authorization
- Configuring
- Multiple Advertised Listeners
- Deployment
- Tools
- Pulsar Shell
- Pulsar Manager
- Monitoring
### Contributor Guide
- Development Environment
- Setup IDE
- Running tests
- Development Processes
- PIP
- Discussions
- The Apache way
- Under the hood - how things work internally
- …
- Performance
- Simulation tools
- Pulsar Plugins
- References
- Binary Protocol
- Under the hood
- How Acknowledgment works?
- …
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org