You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@paimon.apache.org by lz...@apache.org on 2023/03/20 01:35:55 UTC
[incubator-paimon] branch master updated: [doc][github] Update README and create issue/pr template (#631)
This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-paimon.git
The following commit(s) were added to refs/heads/master by this push:
new 6430b6f02 [doc][github] Update README and create issue/pr template (#631)
6430b6f02 is described below
commit 6430b6f027d3de67a3ad83d5c76a27d2f8e81571
Author: Jingsong Lee <ji...@gmail.com>
AuthorDate: Mon Mar 20 09:35:50 2023 +0800
[doc][github] Update README and create issue/pr template (#631)
---
.github/ISSUE_TEMPLATE/bug-report.yml | 56 ++++++++++++++++++++++++++++++++++
.github/ISSUE_TEMPLATE/config.yml | 8 +++++
.github/ISSUE_TEMPLATE/feature.yml | 40 ++++++++++++++++++++++++
.github/PULL_REQUEST_TEMPLATE.md | 17 +++++++++++
README.md | 25 +++++++--------
docs/content/_index.md | 23 ++++++--------
docs/content/concepts/overview.md | 9 ++----
docs/static/img/architecture.png | Bin 261342 -> 580814 bytes
8 files changed, 145 insertions(+), 33 deletions(-)
diff --git a/.github/ISSUE_TEMPLATE/bug-report.yml b/.github/ISSUE_TEMPLATE/bug-report.yml
new file mode 100644
index 000000000..34444a376
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/bug-report.yml
@@ -0,0 +1,56 @@
+name: Bug report
+description: Problems with the software
+title: "[Bug] "
+labels: ["bug"]
+body:
+ - type: markdown
+ attributes:
+ value: |
+ Thank you very much for your feedback!
+ - type: checkboxes
+ attributes:
+ label: Search before asking
+ description: >
+ Please search [issues](https://github.com/apache/incubator-paimon/issues) to check if your issue has already been reported.
+ options:
+ - label: >
+ I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
+ required: true
+ - type: textarea
+ attributes:
+ label: Paimon version
+ description: >
+ Please provide the version of Paimon you are using. If you are using the master branch, please provide the commit id.
+ validations:
+ required: true
+ - type: textarea
+ attributes:
+ label: Compute Engine
+ description: >
+ Please provide the compute engine you use and its version. Flink or Spark or Hive or Trino or Presto or JavaAPI.
+ validations:
+ required: true
+ - type: textarea
+ attributes:
+ label: Minimal reproduce step
+ description: Please try to give reproducing steps to facilitate quick location of the problem.
+ validations:
+ required: true
+ - type: textarea
+ attributes:
+ label: What doesn't meet your expectations?
+ validations:
+ required: true
+ - type: textarea
+ attributes:
+ label: Anything else?
+ - type: checkboxes
+ attributes:
+ label: Are you willing to submit a PR?
+ description: >
+ We look forward to the community of developers or users helping solve Paimon problems together. If you are willing to submit a PR to fix this problem, please check the box.
+ options:
+ - label: I'm willing to submit a PR!
+ - type: markdown
+ attributes:
+ value: "Thanks for completing our form!"
diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml
new file mode 100644
index 000000000..688179141
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/config.yml
@@ -0,0 +1,8 @@
+blank_issues_enabled: false
+contact_links:
+ - name: Subscribe Mailing List
+ url: mailto:user-subscribe@paimon.apache.org
+ about: Subscribe User Mailing List
+ - name: DingTalk Group (Chinese)
+ url: https://qr.dingtalk.com/action/joingroup?code=v1,k1,+LzfVP12Bl5pdCtcy+G9YIDZjnXsKjhN/y3yLCQAffY=&_dt_no_comment=1&origin=11
+ about: Join DingTalk Group
diff --git a/.github/ISSUE_TEMPLATE/feature.yml b/.github/ISSUE_TEMPLATE/feature.yml
new file mode 100644
index 000000000..393f6f583
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/feature.yml
@@ -0,0 +1,40 @@
+name: Feature
+description: Add new feature, improve code, and more
+labels: [ "enhancement" ]
+body:
+ - type: markdown
+ attributes:
+ value: |
+ Thank you very much for your feature proposal!
+ - type: checkboxes
+ attributes:
+ label: Search before asking
+ description: >
+ Please search [issues](https://github.com/apache/incubator-paimon/issues) to check if your issue has already been reported.
+ options:
+ - label: >
+ I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
+ required: true
+ - type: textarea
+ attributes:
+ label: Motivation
+ description: Describe the motivations for this feature, like how it fixes the problem you meet.
+ validations:
+ required: true
+ - type: textarea
+ attributes:
+ label: Solution
+ description: Describe the proposed solution and add related materials like links if any.
+ - type: textarea
+ attributes:
+ label: Anything else?
+ - type: checkboxes
+ attributes:
+ label: Are you willing to submit a PR?
+ description: >
+ We look forward to the community of developers or users helping develop Paimon features together. If you are willing to submit a PR to implement the feature, please check the box.
+ options:
+ - label: I'm willing to submit a PR!
+ - type: markdown
+ attributes:
+ value: "Thanks for completing our form!"
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
new file mode 100644
index 000000000..6f814b93b
--- /dev/null
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,17 @@
+*(Please specify the module before the PR name: [core] ... or [flink] ...)*
+
+### Purpose
+
+*(What is the purpose of the change, or the associated issue)*
+
+### Tests
+
+*(List UT and IT cases to verify this change)*
+
+### API and Format
+
+*(Does this change affect API or storage format)*
+
+### Documentation
+
+*(Does this change introduce a new feature)*
diff --git a/README.md b/README.md
index 3defa001b..4cdcdd925 100644
--- a/README.md
+++ b/README.md
@@ -1,25 +1,22 @@
-# Paimon
+# Paimon (Incubating)
-Paimon is a data lake storage for streaming updates/deletes changelog ingestion and high-performance queries in real time.
+Paimon is a streaming data lake platform that supports high-speed data ingestion, change data tracking and efficient real-time analytics.
-Paimon is developed under the umbrella of [Apache Flink](https://flink.apache.org/).
+Background and documentation is available at https://paimon.apache.org
-## Documentation & Getting Started
+Paimon's former name was Flink Table Store, developed from the Flink community. The architecture refers to some design concepts of Iceberg.
+Thanks to Apache Flink and Apache Iceberg.
-Please check out the full [documentation](https://nightlies.apache.org/flink/paimon-docs-master/), hosted by the
-[ASF](https://www.apache.org/), for detailed information and user guides.
+## Collaboration
-Check our [quick-start](https://nightlies.apache.org/flink/paimon-docs-master/docs/try-paimon/quick-start/) guide for simple setup instructions to get you started with the paimon.
+Paimon tracks issues in GitHub and prefers to receive contributions as pull requests.
-## Building
-
-Run the `mvn clean package -DskipTests` command to build the project.
+Community discussions happen primarily on the [dev mailing list](mailto:dev@paimon.apache.org) or on specific issues.
-Then you will find a JAR file for Flink engine with all shaded dependencies: `paimon-flink/paimon-flink-**/target/paimon-flink-**-<version>.jar`.
-
-## Contributing
+## Building
-You can learn more about how to contribute on the [Apache Flink website](https://flink.apache.org/contributing/how-to-contribute.html). For code contributions, please read carefully the [Contributing Code](https://flink.apache.org/contributing/contribute-code.html) section for an overview of ongoing community work.
+- Run the `mvn clean package -DskipTests` command to build the project.
+- Run the `mvn spotless:apply` to format the project (both Java and Scala).
## License
diff --git a/docs/content/_index.md b/docs/content/_index.md
index 79999f577..f86466258 100644
--- a/docs/content/_index.md
+++ b/docs/content/_index.md
@@ -24,13 +24,15 @@ under the License.
# Apache Paimon
-Paimon is a unified storage to build dynamic tables for both streaming and
-batch processing in Flink, supporting high-speed data ingestion and timely data query.
+Apache Paimon(incubating) is a streaming data lake platform that supports high-speed data ingestion, change data tracking and efficient real-time analytics.
+
Paimon offers the following core capabilities:
-- Support storage of large datasets and allow read/write in both batch and streaming mode.
-- Support streaming queries with minimum latency down to milliseconds.
-- Support Batch/OLAP queries with minimum latency down to the second level.
-- Support incremental snapshots for stream consumption by default. So users do not need to combine different pipelines by themself.
+
+- Unified Batch & Streaming: Paimon supports batch write and batch read, as well as streaming write changes and streaming read table changelogs.
+- Data Lake: As a data lake storage, Paimon has the following advantages: low cost, high reliability, and scalable metadata.
+- Merge Engines: Paimon supports rich Merge Engines. By default, the last entry of the primary key is reserved. You can also use the "partial-update" or "aggregation" engine.
+- Changelog producer: Paimon supports rich Changelog producers, such as "lookup" and "full-compaction". The correct changelog can simplify the construction of a streaming pipeline.
+- Append Only Tables: Paimon supports Append Only tables, automatically compact small files, and provides orderly stream reading. You can use this to replace message queues.
{{< columns >}}
## Try Paimon
@@ -43,12 +45,7 @@ step introduction to the APIs and guides you through real applications.
## Get Help with Paimon
-If you get stuck, check out our [community support
-resources](https://flink.apache.org/community.html). In particular, Apache
-Flink’s user mailing list is consistently ranked as one of the most active of
-any Apache project, and is a great way to get help quickly.
+If you get stuck, you can subscribe User Mailing List (user-subscribe@paimon.apache.org),
+Paimon tracks issues in GitHub and prefers to receive contributions as pull requests. You can also create an issue.
{{< /columns >}}
-
-Paimon is developed under the umbrella of
-[Apache Flink](https://flink.apache.org/).
diff --git a/docs/content/concepts/overview.md b/docs/content/concepts/overview.md
index 3a1d8857d..0cfd9e291 100644
--- a/docs/content/concepts/overview.md
+++ b/docs/content/concepts/overview.md
@@ -26,8 +26,7 @@ under the License.
# Overview
-Paimon is a unified storage to build dynamic tables for both streaming and
-batch processing in Flink, supporting high-speed data ingestion and timely data query.
+Apache Paimon(incubating) is a streaming data lake platform that supports high-speed data ingestion, change data tracking and efficient real-time analytics.
## Architecture
@@ -46,10 +45,8 @@ As shown in the architecture above:
**Ecosystem:** In addition to Apache Flink, Paimon also supports read by other computation
engines like Apache Hive, Apache Spark and Trino.
-**Internal:** Under the hood, Paimon uses a hybrid storage architecture with a lake format to store
-historical data and a queue system to store incremental data. The former stores the columnar files on
-the filesystem/object-store and uses the LSM tree structure to support a large volume of data updates
-and high-performance queries. The latter uses Apache Kafka to capture data in real-time.
+**Internal:** Under the hood, Paimon stores the columnar files on the filesystem/object-store and uses
+the LSM tree structure to support a large volume of data updates and high-performance queries.
## Unified Storage
diff --git a/docs/static/img/architecture.png b/docs/static/img/architecture.png
index a102dd518..639447ecc 100644
Binary files a/docs/static/img/architecture.png and b/docs/static/img/architecture.png differ