You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by ji...@apache.org on 2022/09/21 01:23:28 UTC
[doris-website] branch master updated: Community documentation is produced daily
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new a99af7f6cd9 Community documentation is produced daily
a99af7f6cd9 is described below
commit a99af7f6cd9f3e467dd3c6e72223ea489223f5c9
Author: jiafeng.zhang <zh...@gmail.com>
AuthorDate: Wed Sep 21 09:23:19 2022 +0800
Community documentation is produced daily
---
community/design/Flink-doris-connector-Design.md | 250 ---------
community/design/doris_storage_optimization.md | 235 --------
community/design/grouping_sets_design.md | 501 -----------------
community/design/metadata-design.md | 127 -----
community/design/spark_load.md | 29 -
community/developer-guide/be-vscode-dev.md | 296 -----------
community/developer-guide/benchmark-tool.md | 151 ------
.../developer-guide/bitmap-hll-file-format.md | 64 ---
community/developer-guide/cpp-diagnostic-code.md | 54 --
community/developer-guide/cpp-format-code.md | 104 ----
community/developer-guide/debug-tool.md | 333 ------------
community/developer-guide/docker-dev.md | 161 ------
community/developer-guide/fe-eclipse-dev.md | 226 --------
community/developer-guide/fe-idea-dev.md | 191 -------
community/developer-guide/fe-vscode-dev.md | 80 ---
community/developer-guide/github-checks.md | 31 --
community/developer-guide/how-to-share-blogs.md | 39 --
community/developer-guide/java-format-code.md | 99 ----
community/developer-guide/minidump.md | 177 ------
community/developer-guide/regression-testing.md | 30 --
community/feedback.md | 45 --
community/gitter.md | 59 --
.../commit-format-specification.md | 164 ------
community/how-to-contribute/contribute-doc.md | 313 -----------
community/how-to-contribute/contributor-guide.md | 92 ----
.../how-to-contribute/how-to-be-a-committer.md | 37 --
community/how-to-contribute/how-to-contribute.md | 85 ---
community/how-to-contribute/pull-request.md | 259 ---------
community/release-and-verify/release-complete.md | 183 -------
.../release-and-verify/release-doris-connectors.md | 231 --------
community/release-and-verify/release-doris-core.md | 324 -----------
.../release-and-verify/release-doris-manager.md | 302 -----------
community/release-and-verify/release-prepare.md | 346 ------------
community/release-and-verify/release-verify.md | 100 ----
community/security.md | 38 --
community/subscribe-mail-list.md | 69 ---
community/team.md | 90 ----
.../current/design/doris_storage_optimization.md | 234 --------
.../current/design/flink_doris_connector_design.md | 272 ----------
.../current/design/grouping_sets_design.md | 517 ------------------
.../current/design/metadata-design.md | 126 -----
.../current/design/spark_load.md | 212 --------
.../current/developer-guide/be-vscode-dev.md | 294 ----------
.../current/developer-guide/benchmark-tool.md | 150 ------
.../developer-guide/bitmap-hll-file-format.md | 64 ---
.../current/developer-guide/cpp-diagnostic-code.md | 54 --
.../current/developer-guide/cpp-format-code.md | 105 ----
.../current/developer-guide/debug-tool.md | 334 ------------
.../current/developer-guide/docker-dev.md | 162 ------
.../current/developer-guide/fe-eclipse-dev.md | 227 --------
.../current/developer-guide/fe-idea-dev.md | 186 -------
.../current/developer-guide/fe-vscode-dev.md | 80 ---
.../current/developer-guide/github-checks.md | 31 --
.../current/developer-guide/how-to-share-blogs.md | 39 --
.../current/developer-guide/java-format-code.md | 96 ----
.../current/developer-guide/minidump.md | 185 -------
.../current/developer-guide/regression-testing.md | 592 ---------------------
.../current/feedback.md | 45 --
.../current/gitter.md | 59 --
.../commit-format-specification.md | 162 ------
.../current/how-to-contribute/contribute-doc.md | 312 -----------
.../current/how-to-contribute/contributor-guide.md | 94 ----
.../how-to-contribute/how-to-be-a-committer.md | 37 --
.../current/how-to-contribute/how-to-contribute.md | 84 ---
.../current/how-to-contribute/pull-request.md | 259 ---------
.../current/release-and-verify/release-complete.md | 180 -------
.../release-and-verify/release-doris-connectors.md | 231 --------
.../release-and-verify/release-doris-core.md | 323 -----------
.../release-and-verify/release-doris-manager.md | 302 -----------
.../current/release-and-verify/release-prepare.md | 347 ------------
.../current/release-and-verify/release-verify.md | 102 ----
.../current/security.md | 34 --
.../current/subscribe-mail-list.md | 69 ---
.../current/team.md | 90 ----
sidebarsCommunity.json | 68 ---
75 files changed, 12673 deletions(-)
diff --git a/community/design/Flink-doris-connector-Design.md b/community/design/Flink-doris-connector-Design.md
deleted file mode 100644
index b52cd8535b0..00000000000
--- a/community/design/Flink-doris-connector-Design.md
+++ /dev/null
@@ -1,250 +0,0 @@
----
-{ 'title': 'Flink doris connector Design', 'language': 'en' }
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Flink doris connector Design
-
-First of all, thanks to the author of the community Spark Doris Connector
-
-From the perspective of Doris, by introducing its data into Flink, Flink can use a series of rich ecological products, which broadens the imagination of the product and also makes it possible to query Doris and other data sources jointly.
-
-Starting from our business architecture and business needs, we chose Flink as part of our architecture, the ETL and real-time computing framework for data. The community currently supports Spark doris connector, so we designed and developed Flink doris Connector with reference to Spark doris connector.
-
-##Technical Choice
-
-When the model was originally selected, it was the same as the Spark Doris connector, so we started to consider the JDBC method, but, as described in the Spark doris connector article, this method has advantages, but the disadvantages are more obvious. Later, we read and tested the Spark code and decided to implement it on the shoulders of giants (note: copy the code and modify it directly).
-
-The following content is from the Spark Doris Connector blog, directly copied
-
-```
-Therefore, we developed a new data source Spark-Doris-Connector for Doris. Under this scheme, Doris can publish Doris data and distribute it to Spark. The Spark driver accesses Doris's FE to obtain the Doris table architecture and basic data distribution. After that, according to this data distribution, the data query task is reasonably allocated to the executors. Finally, Spark's execution program accesses different BEs for querying. Greatly improve query efficiency
-```
-
-## 1. Instructions
-
-Compile and generate doris-flink-1.0.0-SNAPSHOT.jar in the extension/flink-doris-connector/ directory of the Doris code base, add this jar package to the ClausPath of flink, and then you can use Flink-on -Doris function
-
-## 2. how to use
-
-Compile and generate doris-flink-1.0.0-SNAPSHOT.jar in the extension/flink-doris-connector/ directory of the Doris code library, add this jar package to the ClassPath of flink, and then use the Flink-on-Doris function
-
-#### 2.1 SQL way
-
-Support function:
-
-1. Supports reading data in Doris data warehouse tables through Flink SQL to Flink for calculations
-2. Support inserting data into the corresponding table of the data warehouse through Flink SQL. The back-end implementation is to communicate directly with BE through Stream Load to complete the data insertion operation
-3. You can use Flink to associate non-Doris external data source tables for association analysis
-
-example:
-
-```java
- final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
- env.setParallelism(1);
- final StreamTableEnvironment tEnv = StreamTableEnvironment.create(env);
- tEnv.executeSql(
- "CREATE TABLE test_aggregation01 (" +
- "user_id STRING," +
- "user_city STRING," +
- "age INT," +
- "last_visit_date STRING" +
- ") " +
- "WITH (\n" +
- " 'connector' = 'doris',\n" +
- " 'fenodes' = 'doris01:8030',\n" +
- " 'table.identifier' = 'demo.test_aggregation',\n" +
- " 'username' = 'root',\n" +
- " 'password' = ''\n" +
- ")");
- tEnv.executeSql(
- "CREATE TABLE test_aggregation02 (" +
- "user_id STRING," +
- "user_city STRING," +
- "age INT," +
- "last_visit_date STRING" +
- ") " +
- "WITH (\n" +
- " 'connector' = 'doris',\n" +
- " 'fenodes' = 'doris01:8030',\n" +
- " 'table.identifier' = 'demo.test_aggregation_01',\n" +
- " 'username' = 'root',\n" +
- " 'password' = ''\n" +
- ")");
-
- tEnv.executeSql("INSERT INTO test_aggregation02 select * from test_aggregation01");
- tEnv.executeSql("select count(1) from test_aggregation01");
-```
-
-#### 2.2 DataStream way:
-
-```java
-DorisOptions.Builder options = DorisOptions.builder()
- .setFenodes("$YOUR_DORIS_FE_HOSTNAME:$YOUR_DORIS_FE_RESFUL_PORT")
- .setUsername("$YOUR_DORIS_USERNAME")
- .setPassword("$YOUR_DORIS_PASSWORD")
- .setTableIdentifier("$YOUR_DORIS_DATABASE_NAME.$YOUR_DORIS_TABLE_NAME");
-env.addSource(new DorisSourceFunction<>(options.build(),new SimpleListDeserializationSchema())).print();
-```
-
-## 3. Applicable scene
-
-
-
-#### 3.1. Use Flink to perform joint analysis on data in Doris and other data sources
-
-Many business departments place their data on different storage systems, such as some online analysis and report data in Doris, some structured retrieval data in Elasticsearch, and some data used for transaction processing in MySQL, and so on. It is often necessary to analyze the business across multiple storage sources. After connecting Flink and Doris through the Flink Doris connector, companies can directly use Flink to perform joint query calculations on the data in Doris and multipl [...]
-
-#### 3.2 Real-time data access
-
-Before Flink Doris Connector: For business irregular data, it is usually necessary to perform standardized processing on messages, and write null value filtering into new topics, and then start regular loading to write Doris.
-
-
-
-After Flink Doris Connector: flink reads kafka and writes doris directly.
-
-
-
-## 4.Technical realization
-
-### 4.1 Architecture diagram
-
-
-
-### 4.2 Doris provides more external capabilities
-
-#### 4.2.1 Doris FE
-
-The interface for obtaining metadata information of internal tables, single-table query planning and some statistical information has been opened to the outside world.
-
-All Rest API interfaces require HttpBasic authentication. The user name and password are the user name and password for logging in to the database. Pay attention to the correct assignment of permissions.
-
-```
-// Get table schema meta information
-GET api/{database}/{table}/_schema
-
-// Get the query plan template for a single table
-POST api/{database}/{table}/_query_plan
-{
-"sql": "select k1, k2 from {database}.{table}"
-}
-
-// Get the table size
-GET api/{database}/{table}/_count
-```
-
-#### 4.2.2 Doris BE
-
-Through the Thrift protocol, data filtering, scanning and cropping capabilities are directly provided to the outside world.
-
-```
-service TDorisExternalService {
- // Initialize the query executor
-TScanOpenResult open_scanner(1: TScanOpenParams params);
-
-// Streaming batch to get data, Apache Arrow data format
- TScanBatchResult get_next(1: TScanNextBatchParams params);
-
-// end scan
- TScanCloseResult close_scanner(1: TScanCloseParams params);
-}
-```
-
-For definitions of Thrift related structures, please refer to:
-
-https://github.com/apache/incubator-doris/blob/master/gensrc/thrift/DorisExternalService.thrift
-
-### 4.3 Implement DataStream
-
-Inherit org.apache.flink.streaming.api.functions.source.RichSourceFunction and customize DorisSourceFunction. During initialization, get the execution plan of the related table and get the corresponding partition.
-
-Rewrite the run method to read data from the partition in a loop.
-
-```
-public void run(SourceContext sourceContext){
- //Cycle through the partitions
- for(PartitionDefinition partitions : dorisPartitions){
- scalaValueReader = new ScalaValueReader(partitions, settings);
- while (scalaValueReader.hasNext()){
- Object next = scalaValueReader.next();
- sourceContext.collect(next);
- }
- }
-}
-```
-
-### 4.4 Implement Flink SQL on Doris
-
-Refer to [Flink Custom Source&Sink](https://ci.apache.org/projects/flink/flink-docs-stable/zh/dev/table/sourceSinks.md) and Flink-jdbc-connector to implement the following As a result, Flink SQL can be used to directly manipulate Doris tables, including reading and writing.
-
-#### 4.4.1 Implementation details
-
-1. Realize DynamicTableSourceFactory and DynamicTableSinkFactory register doris connector
-2. Customize DynamicTableSource and DynamicTableSink to generate logical plans
-3. After DorisRowDataInputFormat and DorisDynamicOutputFormat obtain the logical plan, start execution
-
-
-
-The most important implementation is DorisRowDataInputFormat and DorisDynamicOutputFormat customized based on RichInputFormat and RichOutputFormat.
-
-In DorisRowDataInputFormat, the obtained dorisPartitions are divided into multiple shards in createInputSplits for parallel computing.
-
-```java
-public DorisTableInputSplit[] createInputSplits(int minNumSplits) {
- List<DorisTableInputSplit> dorisSplits = new ArrayList<>();
- int splitNum = 0;
- for (PartitionDefinition partition : dorisPartitions) {
- dorisSplits.add(new DorisTableInputSplit(splitNum++,partition));
- }
- return dorisSplits.toArray(new DorisTableInputSplit[0]);
-}
-
-public RowData nextRecord(RowData reuse) {
- if (!hasNext) {
- //After reading the data, return null
- return null;
- }
- List next = (List)scalaValueReader.next();
- GenericRowData genericRowData = new GenericRowData(next.size());
- for(int i =0;i<next.size();i++){
- genericRowData.setField(i, next.get(i));
- }
- //Determine if there is still data
- hasNext = scalaValueReader.hasNext();
- return genericRowData;
-}
-```
-
-In DorisRowDataOutputFormat, write data to doris through streamload. Refer to org.apache.doris.plugin.audit.DorisStreamLoader for streamload program
-
-```java
-public void writeRecord(RowData row) throws IOException {
- //streamload Default delimiter \t
- StringJoiner value = new StringJoiner("\t");
- GenericRowData rowData = (GenericRowData) row;
- for(int i = 0; i < row.getArity(); ++i) {
- value.add(rowData.getField(i).toString());
- }
- //streamload write data
- DorisStreamLoad.LoadResponse loadResponse = dorisStreamLoad.loadBatch(value.toString());
- System.out.println(loadResponse);
-}
-```
diff --git a/community/design/doris_storage_optimization.md b/community/design/doris_storage_optimization.md
deleted file mode 100644
index 4cfb5866278..00000000000
--- a/community/design/doris_storage_optimization.md
+++ /dev/null
@@ -1,235 +0,0 @@
----
-{
- "title": "Doris Storage File Format Optimization",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-
-# Doris Storage File Format Optimization #
-
-## File format ##
-
-
-<center>1. doris segment</center>
-
-Documents include:
-- The file starts with an 8-byte magic code to identify the file format and version
-- Data Region: Used to store data information for each column, where the data is loaded on demand by pages.
-- Index Region: Doris stores the index data of each column in Index Region, where the data is loaded according to column granularity, so the data information of the following column is stored separately.
-- Footer
- - FileFooterPB: Metadata Information for Definition Files
- - Checksum of 4 bytes of footer Pb content
- - Four bytes FileFooterPB message length for reading FileFooterPB
- - The 8 byte MAGIC CODE is stored in the last bit to facilitate the identification of file types in different scenarios.
-
-The data in the file is organized in the form of page, which is the basic unit of coding and compression. Current page types include the following:
-
-### DataPage ###
-
-Data Page is divided into two types: nullable and non-nullable data pages.
-
-Nullable's data page includes:
-```
-
- +----------------+
- | value count |
- |----------------|
- | first row id |
- |----------------|
- | bitmap length |
- |----------------|
- | null bitmap |
- |----------------|
- | data |
- |----------------|
- | checksum |
- +----------------+
-```
-
-non -zero data page32467;- 26500;- 229140;-
-
-```
- |----------------|
- | value count |
- |----------------|
- | first row id |
- |----------------|
- | data |
- |----------------|
- | checksum |
- +----------------+
-```
-
-The meanings of each field are as follows:
-
-- value count
- - Represents the number of rows in a page
-- First row id
- - Line number of the first line in page
-- bitmap length
- - Represents the number of bytes in the next bitmap
-- null bitmap
- - bitmap representing null information
-- Data
- - Store data after encoding and compress
- - You need to write in the header information of the data: is_compressed
- - Various kinds of data encoded by different codes need to write some field information in the header information in order to achieve data parsing.
- - TODO: Add header information for various encodings
-- Checksum
- - Store page granularity checksum, including page header and subsequent actual data
-
-
-### Bloom Filter Pages ###
-
-For each bloom filter column, a page of the bloom filter is generated corresponding to the granularity of the page and saved in the bloom filter pages area.
-
-### Ordinal Index Page ###
-
-For each column, a sparse index of row numbers is established according to page granularity. The content is a pointer to the block (including offset and length) for the line number of the start line of the page
-
-### Short Key Index page ###
-
-We generate a sparse index of short key every N rows (configurable) with the contents of short key - > line number (ordinal)
-
-### Column's other indexes ###
-
-The format design supports the subsequent expansion of other index information, such as bitmap index, spatial index, etc. It only needs to write the required data to the existing column data, and add the corresponding metadata fields to FileFooterPB.
-
-### Metadata Definition ###
-SegmentFooterPB is defined as:
-
-```
-message ColumnPB {
- required int32 unique_id = 1; // The column id is used here, and the column name is not used
- optional string name = 2; // Column name, when name equals __DORIS_DELETE_SIGN__, this column is a hidden delete column
- required string type = 3; // Column type
- optional bool is_key = 4; // Whether column is a primary key column
- optional string aggregation = 5; // Aggregate type
- optional bool is_nullable = 6; // Whether column is allowed to assgin null
- optional bytes default_value = 7; // Defalut value
- optional int32 precision = 8; // Precision of column
- optional int32 frac = 9;
- optional int32 length = 10; // Length of column
- optional int32 index_length = 11; // Length of column index
- optional bool is_bf_column = 12; // Whether column has bloom filter index
- optional bool has_bitmap_index = 15 [default=false]; // Whether column has bitmap index
-}
-
-// page offset
-message PagePointerPB {
- required uint64 offset; // offset of page in segment file
- required uint32 length; // length of page
-}
-
-message MetadataPairPB {
- optional string key = 1;
- optional bytes value = 2;
-}
-
-message ColumnMetaPB {
- optional ColumnMessage encoding; // Encoding of column
-
- optional PagePointerPB dict_page // Dictionary page
- repeated PagePointerPB bloom_filter_pages; // Bloom filter pages
- optional PagePointerPB ordinal_index_page; // Ordinal index page
- optional PagePointerPB page_zone_map_page; // Page level of statistics index data
-
- optional PagePointerPB bitmap_index_page; // Bitmap index page
-
- optional uint64 data_footprint; // The size of the index in the column
- optional uint64 index_footprint; // The size of the data in the column
- optional uint64 raw_data_footprint; // Original column data size
-
- optional CompressKind compress_kind; // Column compression type
-
- optional ZoneMapPB column_zone_map; // Segment level of statistics index data
- repeated MetadataPairPB column_meta_datas;
-}
-
-message SegmentFooterPB {
- optional uint32 version = 2 [default = 1]; // For version compatibility and upgrade use
- repeated ColumnPB schema = 5; // Schema of columns
- optional uint64 num_values = 4; // Number of lines saved in the file
- optional uint64 index_footprint = 7; // Index size
- optional uint64 data_footprint = 8; // Data size
- optional uint64 raw_data_footprint = 8; // Original data size
-
- optional CompressKind compress_kind = 9 [default = COMPRESS_LZO]; // Compression type
- repeated ColumnMetaPB column_metas = 10; // Column metadata
- optional PagePointerPB key_index_page = 11; // short key index page
-}
-
-```
-
-## Read-write logic ##
-
-### Write ###
-
-The general writing process is as follows:
-1. Write magic
-2. Generate corresponding Column Writer according to schema information. Each Column Writer obtains corresponding encoding information (configurable) according to different types, and generates corresponding encoder according to encoding.
-3. Call encoder - > add (value) for data writing. Each K line generates a short key index entry, and if the current page satisfies certain conditions (the size exceeds 1M or the number of rows is K), a new page is generated and cached in memory.
-4. Continuous cycle step 3 until data writing is completed. Brush the data of each column into the file in sequence
-5. Generate FileFooterPB information and write it to the file.
-
-Relevant issues:
-
-- How does the index of short key be generated?
- - Now we still generate a short key sparse index according to how many rows are sparse, and keep a short sparse index generated every 1024 rows. The specific content is: short key - > ordinal
-
-- What should be stored in the ordinal index?
- - Store the first ordinal to page pointer mapping information for pages
-- What are stored in pages of different encoding types?
- - Dictionary Compression
- - plain
- - rle
- - bshuf
-
-### Read ###
-
-1. Read the magic of the file and judge the type and version of the file.
-2. Read FileFooterPB and check sum
-3. Read short key index and data ordinal index information of corresponding columns according to required columns
-4. Use start key and end key, locate the row number to be read through short key index, then determine the row ranges to be read through ordinal index, and filter the row ranges to be read through statistics, bitmap index and so on.
-5. Then read row data through ordinal index according to row ranges
-
-Relevant issues:
-1. How to quickly locate a row within the page?
-
- The data inside the page is encoding, so it cannot locate the row-level data quickly. Different encoding methods have different schemes for fast line number positioning in-house, which need to be analyzed concretely:
- - If it is rle-coded, skip is performed by resolving the head of RLE until the RLE block containing the row is reached, and then the reverse solution is performed.
- - binary plain encoding: offset information will be stored in the page, and offset information will be specified in the page header. When reading, offset information will be parsed into the array first, so that you can quickly locate the data of a row of block through offset data information of each row.
-2. How to achieve efficient block reading? Consider merging adjacent blocks while they are being read, one-time reading?
-This requires judging whether the block is continuous at the time of reading, and if it is continuous, reading it once.
-
-## Coding ##
-
-In the existing Doris storage, plain encoding is adopted for string type encoding, which is inefficient. After comparison, it is found that in Baidu statistics scenario, data will expand more than twice because of string type coding. Therefore, it is planned to introduce dictionary-based coding compression.
-
-## Compression ##
-
-It implements a scalable compression framework, supports a variety of compression algorithms, facilitates the subsequent addition of new compression algorithms, and plans to introduce zstd compression.
-
-## TODO ##
-1. How to implement nested types? How to locate line numbers in nested types?
-2. How to optimize the downstream bitmap and column statistics caused by ScanRange splitting?
diff --git a/community/design/grouping_sets_design.md b/community/design/grouping_sets_design.md
deleted file mode 100644
index 16acc33997c..00000000000
--- a/community/design/grouping_sets_design.md
+++ /dev/null
@@ -1,501 +0,0 @@
----
-{
- "title": "GROUPING SETS DESIGN",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-# GROUPING SETS DESIGN
-
-## 1. GROUPING SETS Background
-
-The `CUBE`, `ROLLUP`, and `GROUPING` `SETS` extensions to SQL make querying and reporting easier and faster. `CUBE`, `ROLLUP`, and grouping sets produce a single result set that is equivalent to a `UNION` `ALL` of differently grouped rows. `ROLLUP` calculates aggregations such as `SUM`, `COUNT`, `MAX`, `MIN`, and `AVG` at increasing levels of aggregation, from the most detailed up to a grand total. `CUBE` is an extension similar to `ROLLUP`, enabling a single statement to calculate all p [...]
-To enhance performance, `CUBE`, `ROLLUP`, and `GROUPING SETS` can be parallelized: multiple processes can simultaneously execute all of these statements. These capabilities make aggregate calculations more efficient, thereby enhancing database performance, and scalability.
-
-The three `GROUPING` functions help you identify the group each row belongs to and enable sorting subtotal rows and filtering results.
-
-### 1.1 GROUPING SETS Syntax
-
-`GROUPING SETS` syntax lets you define multiple groupings in the same query. `GROUP BY` computes all the groupings specified and combines them with `UNION ALL`. For example, consider the following statement:
-
-```
-SELECT k1, k2, SUM( k3 ) FROM t GROUP BY GROUPING SETS ( (k1, k2), (k1), (k2), ( ) );
-```
-
-
-This statement is equivalent to:
-
-```
-SELECT k1, k2, SUM( k3 ) FROM t GROUP BY k1, k2
-UNION
-SELECT k1, null, SUM( k3 ) FROM t GROUP BY k1
-UNION
-SELECT null, k2, SUM( k3 ) FROM t GROUP BY k2
-UNION
-SELECT null, null, SUM( k3 ) FROM t
-```
-
-This is an example of real query:
-
-```
-mysql> SELECT * FROM t;
-+------+------+------+
-| k1 | k2 | k3 |
-+------+------+------+
-| a | A | 1 |
-| a | A | 2 |
-| a | B | 1 |
-| a | B | 3 |
-| b | A | 1 |
-| b | A | 4 |
-| b | B | 1 |
-| b | B | 5 |
-+------+------+------+
-8 rows in set (0.01 sec)
-
-mysql> SELECT k1, k2, SUM(k3) FROM t GROUP BY GROUPING SETS ( (k1, k2), (k2), (k1), ( ) );
-+------+------+-----------+
-| k1 | k2 | sum(`k3`) |
-+------+------+-----------+
-| b | B | 6 |
-| a | B | 4 |
-| a | A | 3 |
-| b | A | 5 |
-| NULL | B | 10 |
-| NULL | A | 8 |
-| a | NULL | 7 |
-| b | NULL | 11 |
-| NULL | NULL | 18 |
-+------+------+-----------+
-9 rows in set (0.06 sec)
-```
-
-### 1.2 ROLLUP Syntax
-
-`ROLLUP` enables a `SELECT` statement to calculate multiple levels of subtotals across a specified group of dimensions. It also calculates a grand total. `ROLLUP` is a simple extension to the `GROUP` `BY` clause, so its syntax is extremely easy to use. The `ROLLUP` extension is highly efficient, adding minimal overhead to a query.
-
-`ROLLUP` appears in the `GROUP` `BY` clause in a `SELECT` statement. Its form is:
-
-```
-SELECT a, b,c, SUM( d ) FROM tab1 GROUP BY ROLLUP(a,b,c)
-```
-
-This statement is equivalent to GROUPING SETS as followed:
-
-```
-GROUPING SETS (
-(a,b,c),
-( a, b ),
-( a),
-( )
-)
-```
-
-### 1.3 CUBE Syntax
-
-Like `ROLLUP` `CUBE` generates all the subtotals that could be calculated for a data cube with the specified dimensions.
-
-```
-SELECT a, b,c, SUM( d ) FROM tab1 GROUP BY CUBE(a,b,c)
-```
-
-e.g. CUBE ( a, b, c ) is equivalent to GROUPING SETS as followed:
-
-```
-GROUPING SETS (
-( a, b, c ),
-( a, b ),
-( a, c ),
-( a ),
-( b, c ),
-( b ),
-( c ),
-( )
-)
-```
-
-### 1.4 GROUPING and GROUPING_ID Function
-
-Indicates whether a specified column expression in a `GROUP BY` list is aggregated or not. `GROUPING `returns 1 for aggregated or 0 for not aggregated in the result set. `GROUPING` can be used only in the `SELECT` list, `HAVING`, and `ORDER BY` clauses when `GROUP BY` is specified.
-
-`GROUPING_ID` describes which of a list of expressions are grouped in a row produced by a `GROUP BY` query. The `GROUPING_ID` function simply returns the decimal equivalent of the binary value formed as a result of the concatenation of the values returned by the `GROUPING` functions.
-
-Each `GROUPING_ID` argument must be an element of the `GROUP BY` list. `GROUPING_ID ()` returns an **integer** bitmap whose lowest N bits may be lit. A lit **bit** indicates the corresponding argument is not a grouping column for the given output row. The lowest-order **bit** corresponds to argument N, and the N-1th lowest-order **bit** corresponds to argument 1. If the column is a grouping column the bit is 0 else is 1.
-
-For example:
-
-```
-mysql> select * from t;
-+------+------+------+
-| k1 | k2 | k3 |
-+------+------+------+
-| a | A | 1 |
-| a | A | 2 |
-| a | B | 1 |
-| a | B | 3 |
-| b | A | 1 |
-| b | A | 4 |
-| b | B | 1 |
-| b | B | 5 |
-+------+------+------+
-```
-
-grouping sets result:
-
-```
-mysql> SELECT k1, k2, GROUPING(k1), GROUPING(k2), SUM(k3) FROM t GROUP BY GROUPING SETS ( (k1, k2), (k2), (k1), ( ) );
-+------+------+----------------+----------------+-----------+
-| k1 | k2 | grouping(`k1`) | grouping(`k2`) | sum(`k3`) |
-+------+------+----------------+----------------+-----------+
-| a | A | 0 | 0 | 3 |
-| a | B | 0 | 0 | 4 |
-| a | NULL | 0 | 1 | 7 |
-| b | A | 0 | 0 | 5 |
-| b | B | 0 | 0 | 6 |
-| b | NULL | 0 | 1 | 11 |
-| NULL | A | 1 | 0 | 8 |
-| NULL | B | 1 | 0 | 10 |
-| NULL | NULL | 1 | 1 | 18 |
-+------+------+----------------+----------------+-----------+
-9 rows in set (0.02 sec)
-
-mysql> SELECT k1, k2, GROUPING_ID(k1,k2), SUM(k3) FROM t GROUP BY GROUPING SETS ( (k1, k2), (k2), (k1), ( ) );
-+------+------+-------------------------+-----------+
-| k1 | k2 | grouping_id(`k1`, `k2`) | sum(`k3`) |
-+------+------+-------------------------+-----------+
-| a | A | 0 | 3 |
-| a | B | 0 | 4 |
-| a | NULL | 1 | 7 |
-| b | A | 0 | 5 |
-| b | B | 0 | 6 |
-| b | NULL | 1 | 11 |
-| NULL | A | 2 | 8 |
-| NULL | B | 2 | 10 |
-| NULL | NULL | 3 | 18 |
-+------+------+-------------------------+-----------+
-9 rows in set (0.02 sec)
-
-mysql> SELECT k1, k2, grouping(k1), grouping(k2), GROUPING_ID(k1,k2), SUM(k4) FROM t GROUP BY GROUPING SETS ( (k1, k2), (k2), (k1), ( ) ) order by k1, k2;
-+------+------+----------------+----------------+-------------------------+-----------+
-| k1 | k2 | grouping(`k1`) | grouping(`k2`) | grouping_id(`k1`, `k2`) | sum(`k4`) |
-+------+------+----------------+----------------+-------------------------+-----------+
-| a | A | 0 | 0 | 0 | 3 |
-| a | B | 0 | 0 | 0 | 4 |
-| a | NULL | 0 | 1 | 1 | 7 |
-| b | A | 0 | 0 | 0 | 5 |
-| b | B | 0 | 0 | 0 | 6 |
-| b | NULL | 0 | 1 | 1 | 11 |
-| NULL | A | 1 | 0 | 2 | 8 |
-| NULL | B | 1 | 0 | 2 | 10 |
-| NULL | NULL | 1 | 1 | 3 | 18 |
-+------+------+----------------+----------------+-------------------------+-----------+
-9 rows in set (0.02 sec)
-
-```
-### 1.5 Composition and nesting of GROUPING SETS
-
-First of all, a GROUP BY clause is essentially a special case of GROUPING SETS, for example:
-
-```
- GROUP BY a
-is equivalent to:
- GROUP BY GROUPING SETS((a))
-also,
- GROUP BY a,b,c
-is equivalent to:
- GROUP BY GROUPING SETS((a,b,c))
-```
-
-Similarly, CUBE and ROLLUP can be expanded into GROUPING SETS, so the various combinations and nesting of GROUP BY, CUBE, ROLLUP, GROUPING SETS are essentially the combination and nesting of GROUPING SETS.
-
-For GROUPING SETS nesting, it is semantically equivalent to writing the statements inside the nest directly outside. (ref:<https://www.brytlyt.com/documentation/data-manipulation-dml/grouping-sets-rollup-cube/>) mentions:
-
-```
-The CUBE and ROLLUP constructs can be used either directly in the GROUP BY clause, or nested inside a GROUPING SETS clause. If one GROUPING SETS clause is nested inside another, the effect is the same as if all the elements of the inner clause had been written directly in the outer clause.
-```
-
-For a combined list of multiple GROUPING SETS, many databases consider it a cross product relationship.
-
-for example:
-
-```
-GROUP BY a, CUBE (b, c), GROUPING SETS ((d), (e))
-
-is equivalent to:
-
-GROUP BY GROUPING SETS (
-(a, b, c, d), (a, b, c, e),
-(a, b, d), (a, b, e),
-(a, c, d), (a, c, e),
-(a, d), (a, e)
-)
-```
-
-For the combination and nesting of GROUPING SETS, each database support is not the same. For example snowflake does not support any combination and nesting.
-（<https://docs.snowflake.net/manuals/sql-reference/constructs/group-by.html>）
-
-Oracle supports both composition and nesting.
-（<https://docs.oracle.com/cd/B19306_01/server.102/b14223/aggreg.htm#i1006842>）
-
-Presto supports composition, but not nesting.
-（<https://prestodb.github.io/docs/current/sql/select.html>）
-
-## 2. Object
-
-Support `GROUPING SETS`, `ROLLUP` and `CUBE ` syntax, implements 1.1, 1.2, 1.3 1.4, 1.5, not support the combination
- and nesting of GROUPING SETS in current version.
-
-### 2.1 GROUPING SETS Syntax
-
-```
-SELECT ...
-FROM ...
-[ ... ]
-GROUP BY GROUPING SETS ( groupSet [ , groupSet [ , ... ] ] )
-[ ... ]
-
-groupSet ::= { ( expr [ , expr [ , ... ] ] )}
-
-<expr>
-Expression, column name.
-```
-
-### 2.2 ROLLUP Syntax
-
-```
-SELECT ...
-FROM ...
-[ ... ]
-GROUP BY ROLLUP ( expr [ , expr [ , ... ] ] )
-[ ... ]
-
-<expr>
-Expression, column name.
-```
-
-### 2.3 CUBE Syntax
-
-```
-SELECT ...
-FROM ...
-[ ... ]
-GROUP BY CUBE ( expr [ , expr [ , ... ] ] )
-[ ... ]
-
-<expr>
-Expression, column name.
-```
-
-## 3. Implementation
-
-### 3.1 Overall Design Approaches
-
-For `GROUPING SET` is equivalent to the `UNION` of `GROUP BY` . So we can expand input rows, and run an GROUP BY on these rows.
-
-For example:
-
-```
-SELECT a, b FROM src GROUP BY a, b GROUPING SETS ((a, b), (a), (b), ());
-```
-
-Data in table src:
-
-```
-1, 2
-3, 4
-```
-
-Base on GROUPING SETS , we can expend the input to:
-
-```
-1, 2 (GROUPING_ID: a, b -> 00 -> 0)
-1, null (GROUPING_ID: a, null -> 01 -> 1)
-null, 2 (GROUPING_ID: null, b -> 10 -> 2)
-null, null (GROUPING_ID: null, null -> 11 -> 3)
-
-3, 4 (GROUPING_ID: a, b -> 00 -> 0)
-3, null (GROUPING_ID: a, null -> 01 -> 1)
-null, 4 (GROUPING_ID: null, b -> 10 -> 2)
-null, null (GROUPING_ID: null, null -> 11 -> 3)
-```
-
-And then use those row as input, then GROUP BY a, b, GROUPING_ID
-
-### 3.2 Example
-
-Table t:
-
-```
-mysql> select * from t;
-+------+------+------+
-| k1 | k2 | k3 |
-+------+------+------+
-| a | A | 1 |
-| a | A | 2 |
-| a | B | 1 |
-| a | B | 3 |
-| b | A | 1 |
-| b | A | 4 |
-| b | B | 1 |
-| b | B | 5 |
-+------+------+------+
-8 rows in set (0.01 sec)
-```
-
-for the query:
-
-```
-SELECT k1, k2, GROUPING_ID(k1,k2), SUM(k3) FROM t GROUP BY GROUPING SETS ((k1, k2), (k1), (k2), ());
-```
-
-First, expand the input, every row expand into 4 rows ( the size of GROUPING SETS), and insert GROUPING_ID column
-
-e.g. a, A, 1 expanded to:
-
-```
-+------+------+------+-------------------------+
-| k1 | k2 | k3 | GROUPING_ID(`k1`, `k2`) |
-+------+------+------+-------------------------+
-| a | A | 1 | 0 |
-| a | NULL | 1 | 1 |
-| NULL | A | 1 | 2 |
-| NULL | NULL | 1 | 3 |
-+------+------+------+-------------------------+
-```
-
-Finally, all rows expended as follows (32 rows):
-
-```
-+------+------+------+-------------------------+
-| k1 | k2 | k3 | GROUPING_ID(`k1`, `k2`) |
-+------+------+------+-------------------------+
-| a | A | 1 | 0 |
-| a | A | 2 | 0 |
-| a | B | 1 | 0 |
-| a | B | 3 | 0 |
-| b | A | 1 | 0 |
-| b | A | 4 | 0 |
-| b | B | 1 | 0 |
-| b | B | 5 | 0 |
-| a | NULL | 1 | 1 |
-| a | NULL | 1 | 1 |
-| a | NULL | 2 | 1 |
-| a | NULL | 3 | 1 |
-| b | NULL | 1 | 1 |
-| b | NULL | 1 | 1 |
-| b | NULL | 4 | 1 |
-| b | NULL | 5 | 1 |
-| NULL | A | 1 | 2 |
-| NULL | A | 1 | 2 |
-| NULL | A | 2 | 2 |
-| NULL | A | 4 | 2 |
-| NULL | B | 1 | 2 |
-| NULL | B | 1 | 2 |
-| NULL | B | 3 | 2 |
-| NULL | B | 5 | 2 |
-| NULL | NULL | 1 | 3 |
-| NULL | NULL | 1 | 3 |
-| NULL | NULL | 1 | 3 |
-| NULL | NULL | 1 | 3 |
-| NULL | NULL | 2 | 3 |
-| NULL | NULL | 3 | 3 |
-| NULL | NULL | 4 | 3 |
-| NULL | NULL | 5 | 3 |
-+------+------+------+-------------------------+
-32 rows in set.
-```
-
-now GROUP BY k1, k2, GROUPING_ID(k1,k2):
-
-```
-+------+------+-------------------------+-----------+
-| k1 | k2 | grouping_id(`k1`, `k2`) | sum(`k3`) |
-+------+------+-------------------------+-----------+
-| a | A | 0 | 3 |
-| a | B | 0 | 4 |
-| a | NULL | 1 | 7 |
-| b | A | 0 | 5 |
-| b | B | 0 | 6 |
-| b | NULL | 1 | 11 |
-| NULL | A | 2 | 8 |
-| NULL | B | 2 | 10 |
-| NULL | NULL | 3 | 18 |
-+------+------+-------------------------+-----------+
-9 rows in set (0.02 sec)
-```
-
-The result is equivalent to the UNION ALL
-
-```
-select k1, k2, sum(k3) from t group by k1, k2
-UNION ALL
-select NULL, k2, sum(k3) from t group by k2
-UNION ALL
-select k1, NULL, sum(k3) from t group by k1
-UNION ALL
-select NULL, NULL, sum(k3) from t;
-
-+------+------+-----------+
-| k1 | k2 | sum(`k3`) |
-+------+------+-----------+
-| b | B | 6 |
-| b | A | 5 |
-| a | A | 3 |
-| a | B | 4 |
-| a | NULL | 7 |
-| b | NULL | 11 |
-| NULL | B | 10 |
-| NULL | A | 8 |
-| NULL | NULL | 18 |
-+------+------+-----------+
-9 rows in set (0.06 sec)
-```
-
-### 3.3 FE
-
-#### 3.3.1 Tasks
-
-1. Add GroupByClause, replace groupingExprs.
-2. Add Grouping Sets, Cube and RollUp syntax.
-3. Add GroupByClause in SelectStmt.
-4. Add GroupingFunctionCallExpr, implements grouping grouping_id function call
-5. Add VirtualSlot, generate the map of virtual slots and real slots
-6. add virtual column GROUPING_ID and other virtual columns generated by grouping and grouping_id, insert into groupingExprs,
-7. Add a PlanNode, name as RepeatNode. For GroupingSets aggregation insert RepeatNode to the plan.
-
-#### 3.3.2 Tuple
-
-In order to add GROUPING_ID to groupingExprs in GroupByClause, need to create virtual SlotRef, also, need tot create a tuple for this slot, named GROUPING\_\_ID Tuple.
-
-For the plannode RepeatNode, its input are all the tuples of its children and its output tuple are the repeat data and GROUPING_ID.
-
-
-#### 3.3.3 Expression and Function Substitution
-
-expr -> if(bitand(pos, grouping_id)=0, expr, null) for expr in extension grouping clause
-grouping_id() -> grouping_id(grouping_id) for grouping_id function
-
-### 3.4 BE
-
-#### 3.4.1 Tasks
-
-1. Add RepeatNode executor, expend the input data and append GROUPING_ID to every row
-2. Implements grouping_id() and grouping() function.
diff --git a/community/design/metadata-design.md b/community/design/metadata-design.md
deleted file mode 100644
index d291c8ef6eb..00000000000
--- a/community/design/metadata-design.md
+++ /dev/null
@@ -1,127 +0,0 @@
----
-{
- "title": "Metadata Design Document",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-
-# Metadata Design Document
-
-## Noun Interpretation
-
-* FE: Frontend, the front-end node of Doris. Mainly responsible for receiving and returning client requests, metadata, cluster management, query plan generation and so on.
-* BE: Backend, the back-end node of Doris. Mainly responsible for data storage and management, query plan execution and other work.
-* bdbje: [Oracle Berkeley DB Java Edition](http://www.oracle.com/technetwork/database/berkeleydb/overview/index-093405.html). In Doris, we use bdbje to persist metadata operation logs and high availability of FE.
-
-## Overall architecture
-
-
-As shown above, Doris's overall architecture is divided into two layers. Multiple FEs form the first tier, providing lateral expansion and high availability of FE. Multiple BEs form the second layer, which is responsible for data storage and management. This paper mainly introduces the design and implementation of metadata in FE layer.
-
-1. There are two different kinds of FE nodes: follower and observer. Leader election and data synchronization are taken among FE nodes by bdbje ([BerkeleyDB Java Edition](http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/overview/index-093405.html)).
-
-2. The follower node is elected, and one of the followers becomes the leader node, which is responsible for the writing of metadata. When the leader node goes down, other follower nodes re-elect a leader to ensure high availability of services.
-
-3. The observer node only synchronizes metadata from the leader node and does not participate in the election. It can be scaled horizontally to provide the extensibility of metadata reading services.
-
-> Note: The concepts of follower and observer corresponding to bdbje are replica and observer. You may use both names below.
-
-## Metadata structure
-
-Doris's metadata is in full memory. A complete metadata image is maintained in each FE memory. Within Baidu, a cluster of 2,500 tables and 1 million fragments (3 million copies) occupies only about 2GB of metadata in memory. (Of course, the memory overhead for querying intermediate objects and various job information needs to be estimated according to the actual situation. However, it still maintains a low memory overhead.
-
-At the same time, metadata is stored in the memory as a whole in a tree-like hierarchical structure. By adding auxiliary structure, metadata information at all levels can be accessed quickly.
-
-The following figure shows the contents stored in Doris meta-information.
-
-
-
-As shown above, Doris's metadata mainly stores four types of data:
-
-1. User data information. Including database, table Schema, fragmentation information, etc.
-2. All kinds of job information. For example, import jobs, Clone jobs, SchemaChange jobs, etc.
-3. User and permission information.
-4. Cluster and node information.
-
-## Data stream
-
-
-
-The data flow of metadata is as follows:
-
-1. Only leader FE can write metadata. After modifying leader's memory, the write operation serializes into a log and writes to bdbje in the form of key-value. The key is a continuous integer, and as log id, value is the serialized operation log.
-
-2. After the log is written to bdbje, bdbje copies the log to other non-leader FE nodes according to the policy (write most/write all). The non-leader FE node modifies its metadata memory image by playback of the log, and completes the synchronization with the metadata of the leader node.
-
-3. The number of log entries of the leader node reaches the threshold (default 10w) and meets the checkpoint thread execution cycle (default 60 seconds). Checkpoint reads existing image files and subsequent logs and replays a new mirror copy of metadata in memory. The copy is then written to disk to form a new image. The reason for this is to regenerate a mirror copy instead of writing an existing image to an image, mainly considering that the write operation will be blocked during writi [...]
-
-4. After the image file is generated, the leader node notifies other non-leader nodes that a new image has been generated. Non-leader actively pulls the latest image files through HTTP to replace the old local files.
-
-5. The logs in bdbje will be deleted regularly after the image is completed.
-
-## Implementation details
-
-### Metadata catalogue
-
-1. The metadata directory is specified by the FE configuration item `meta_dir'.
-
-2. Data storage directory for bdbje under `bdb/` directory.
-
-3. The storage directory for image files under the `image/` directory.
-
-* `Image.[logid]`is the latest image file. The suffix `logid` indicates the ID of the last log contained in the image.
-* `Image.ckpt` is the image file being written. If it is successfully written, it will be renamed `image.[logid]` and replaced with the original image file.
-* The`cluster_id` is recorded in the `VERSION` file. `Cluster_id` uniquely identifies a Doris cluster. It is a 32-bit integer randomly generated at the first startup of leader. You can also specify a cluster ID through the Fe configuration item `cluster_id'.
-* The role of FE itself recorded in the `ROLE` file. There are only `FOLLOWER` and `OBSERVER`. Where `FOLLOWER` denotes FE as an optional node. (Note: Even the leader node has a role of `FOLLOWER`)
-
-### Start-up process
-
-1. FE starts for the first time. If the startup script does not add any parameters, it will try to start as leader. You will eventually see `transfer from UNKNOWN to MASTER` in the FE startup log.
-
-2. FE starts for the first time. If the `-helper` parameter is specified in the startup script and points to the correct leader FE node, the FE first asks the leader node about its role (ROLE) and cluster_id through http. Then pull up the latest image file. After reading image file and generating metadata image, start bdbje and start bdbje log synchronization. After synchronization is completed, the log after image file in bdbje is replayed, and the final metadata image generation is completed.
-
- > Note 1: When starting with the `-helper` parameter, you need to first add the FE through the leader through the MySQL command, otherwise, the start will report an error.
-
- > Note 2: `-helper` can point to any follower node, even if it is not leader.
-
- > Note 3: In the process of synchronization log, the Fe log will show `xxx detached`. At this time, the log pull is in progress, which is a normal phenomenon.
-
-3. FE is not the first startup. If the startup script does not add any parameters, it will determine its identity according to the ROLE information stored locally. At the same time, according to the cluster information stored in the local bdbje, the leader information is obtained. Then read the local image file and the log in bdbje to complete the metadata image generation. (If the roles recorded in the local ROLE are inconsistent with those recorded in bdbje, an error will be reported.)
-
-4. FE is not the first boot, and the `-helper` parameter is specified in the boot script. Just like the first process started, the leader role is asked first. But it will be compared with the ROLE stored by itself. If they are inconsistent, they will report errors.
-
-#### Metadata Read-Write and Synchronization
-
-1. Users can use Mysql to connect any FE node to read and write metadata. If the connection is a non-leader node, the node forwards the write operation to the leader node. When the leader is successfully written, it returns a current and up-to-date log ID of the leader. Later, the non-leader node waits for the log ID it replays to be larger than the log ID it returns to the client before returning the message that the command succeeds. This approach guarantees Read-Your-Write semantics f [...]
-
- > Note: Some non-write operations are also forwarded to leader for execution. For example, `SHOW LOAD` operation. Because these commands usually need to read the intermediate states of some jobs, which are not written to bdbje, there are no such intermediate states in the memory of the non-leader node. (FE's direct metadata synchronization depends entirely on bdbje's log playback. If a metadata modification operation does not write bdbje's log, the result of the modification of the oper [...]
-
-2. The leader node starts a TimePrinter thread. This thread periodically writes a key-value entry for the current time to bdbje. The remaining non-leader nodes read the recorded time in the log by playback and compare it with the local time. If the lag between the local time and the local time is found to be greater than the specified threshold (configuration item: `meta_delay_toleration_second`). If the write interval is half of the configuration item, the node will be in the **unreadab [...]
-
-3. The metadata of each FE only guarantees the final consistency. Normally, inconsistent window periods are only milliseconds. We guarantee the monotonous consistency of metadata access in the same session. But if the same client connects different FEs, metadata regression may occur. (But for batch update systems, this problem has little impact.)
-
-### Downtime recovery
-
-1. When the leader node goes down, the rest of the followers will immediately elect a new leader node to provide services.
-2. Metadata cannot be written when most follower nodes are down. When metadata is not writable, if a write operation request occurs, the current process is that the **FE process exits**. This logic will be optimized in the future, and read services will still be provided in the non-writable state.
-3. The downtime of observer node will not affect the state of any other node. It also does not affect metadata reading and writing at other nodes.
diff --git a/community/design/spark_load.md b/community/design/spark_load.md
deleted file mode 100644
index 9e630f4393a..00000000000
--- a/community/design/spark_load.md
+++ /dev/null
@@ -1,29 +0,0 @@
----
-{
- "title": "Spark Load",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Spark load
-
-TODO
diff --git a/community/developer-guide/be-vscode-dev.md b/community/developer-guide/be-vscode-dev.md
deleted file mode 100644
index 0c71d6f5757..00000000000
--- a/community/developer-guide/be-vscode-dev.md
+++ /dev/null
@@ -1,296 +0,0 @@
----
-{
- "title": "BE development and debugging environment under Linux",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Apache Doris Be development and debugging
-
-## initial preparation work
-
-**This tutorial was conducted under Ubuntu 20.04**
-
-**The name of the BE binary that appears in this doc is `doris_be`, which was `palo_be` in previous versions.**
-
-1. Download the doris source code
-
- URL：[apache/doris: Apache Doris (github.com)](https://github.com/apache/doris)
-
-2. Install GCC 8.3.1+, Oracle JDK 1.8+, Python 2.7+, confirm that the gcc, java, python commands point to the correct version, and set the JAVA_HOME environment variable
-
-3. Install other dependent packages
-
-```
-sudo apt install build-essential openjdk-8-jdk maven cmake byacc flex automake libtool-bin bison binutils-dev libiberty-dev zip unzip libncurses5-dev curl git ninja-build python brotli
-sudo add-apt-repository ppa:ubuntu-toolchain-r/ppa
-sudo apt update
-sudo apt install gcc-10 g++-10
-sudo apt-get install autoconf automake libtool autopoint
-```
-
-4. install : openssl libssl-dev
-```
-sudo apt install -y openssl libssl-dev
-```
-
-## Compile
-
-The following steps are carried out in the /home/workspace directory
-
-1. dowload source
-
-```
-git clone https://github.com/apache/doris.git
-```
-
-2. Compile third-party dependency packages
-
-```
- cd /home/workspace/doris/thirdparty
- ./build-thirdparty.sh
-```
-
-3. Compile doris product code
-
-```
-cd /home/workspace/doris
-./build.sh
-```
-
-Note: This compilation has the following instructions:
-
-```shell
-./build.sh #Compile be and fe at the same time
-./build.sh --be #Only compile be
-./build.sh --fe #Only compilefe
-./build.sh --fe --be --clean#Delete and compile be fe at the same time
-./build.sh --fe --clean#Delete and compile fe
-./build.sh --be --clean#Delete and compile be
-./build.sh --be --fe --clean#Delete and compile be fe at the same time
-```
-
-If nothing happens, the compilation should be successful, and the final deployment file will be output to the /home/workspace/doris/output/ directory. If you still encounter other problems, you can refer to the doris installation document http://doris.apache.org.
-
-## Deployment and debugging(GDB)
-
-1. Authorize be compilation result files
-
-```
-chmod /home/workspace/doris/output/be/lib/doris_be
-```
-
-Note: /home/workspace/doris/output/be/lib/doris_be is the executable file of be.
-
-2. Create a data storage directory
-
-By viewing /home/workspace/doris/output/be/conf/be.conf
-
-```
-# INFO, WARNING, ERROR, FATAL
-sys_log_level = INFO
-be_port = 9060
-be_rpc_port = 9070
-webserver_port = 8040
-heartbeat_service_port = 9050
-brpc_port = 8060
-
-# Note that there should at most one ip match this list.
-# If no ip match this rule, will choose one randomly.
-# use CIDR format, e.g. 10.10.10.0/
-# Default value is empty.
-priority_networks = 192.168.59.0/24 # data root path, separate by ';'
-storage_root_path = /soft/be/storage
-# sys_log_dir = ${PALO_HOME}/log
-# sys_log_roll_mode = SIZE-MB-
-# sys_log_roll_num =
-# sys_log_verbose_modules =
-# log_buffer_level = -
-# palo_cgroups
-```
-
-Need to create this folder, this is where the be data is stored
-
-```
-mkdir -p /soft/be/storage
-```
-
-3. Open vscode, and open the directory where the be source code is located. In this case, open the directory as **/home/workspace/doris/**，For details on how to vscode, refer to the online tutorial
-
-4. Install the vscode ms c++ debugging plug-in, the plug-in identified by the red box in the figure below
-
-
-
-5. Create a launch.json file, the content of the file is as follows:
-
-```
-{
- "version": "0.2.0",
- "configurations": [
- {
- "name": "(gdb) Launch",
- "type": "cppdbg",
- "request": "launch",
- "program": "/home/workspace/doris/output/be/lib/doris_be",
- "args": [],
- "stopAtEntry": false,
- "cwd": "/home/workspace/doris/",
- "environment": [{"name":"PALO_HOME","value":"/home/workspace/doris/output/be/"},
- {"name":"UDF_RUNTIME_DIR","value":"/home/workspace/doris/output/be/lib/udf-runtime"},
- {"name":"LOG_DIR","value":"/home/workspace/doris/output/be/log"},
- {"name":"PID_DIR","value":"/home/workspace/doris/output/be/bin"}
- ],
- "externalConsole": true,
- "MIMode": "gdb",
- "setupCommands": [
- {
- "description": "Enable pretty-printing for gdb",
- "text": "-enable-pretty-printing",
- "ignoreFailures": true
- }
- ]
- }
- ]
-}
-```
-
-Among them, environment defines several environment variables DORIS_HOME UDF_RUNTIME_DIR LOG_DIR PID_DIR, which are the environment variables needed when doris_be is running. If it is not set, the startup will fail
-
-**Note: If you want attach (additional process) debugging, the configuration code is as follows:**
-
-```
-{
- "version": "0.2.0",
- "configurations": [
- {
- "name": "(gdb) Launch",
- "type": "cppdbg",
- "request": "attach",
- "program": "/home/workspace/doris/output/lib/doris_be",
- "processId":,
- "MIMode": "gdb",
- "internalConsoleOptions":"openOnSessionStart",
- "setupCommands": [
- {
- "description": "Enable pretty-printing for gdb",
- "text": "-enable-pretty-printing",
- "ignoreFailures": true
- }
- ]
- }
- ]
-}
-```
-
-In the configuration **"request": "attach", "processId": PID**, these two configurations are the key points: set the debug mode of gdb to attach and attach the processId of the process, otherwise it will fail. To find the process id, you can enter the following command in the command line:
-
-```
-ps -ef | grep palo*
-```
-
-As shown in the figure:
-
-
-
-Among them, 15200 is the process id of the currently running be.
-
-An example of a complete launch.json is as follows:
-
-```
- {
- "version": "0.2.0",
- "configurations": [
- {
- "name": "(gdb) Attach",
- "type": "cppdbg",
- "request": "attach",
- "program": "/home/workspace/doris/output/be/lib/doris_be",
- "processId": 17016,
- "MIMode": "gdb",
- "setupCommands": [
- {
- "description": "Enable pretty-printing for gdb",
- "text": "-enable-pretty-printing",
- "ignoreFailures": true
- }
- ]
- },
- {
- "name": "(gdb) Launch",
- "type": "cppdbg",
- "request": "launch",
- "program": "/home/workspace/doris/output/be/lib/doris_be",
- "args": [],
- "stopAtEntry": false,
- "cwd": "/home/workspace/doris/output/be",
- "environment": [
- {
- "name": "DORIS_HOME",
- "value": "/home/workspace/doris/output/be"
- },
- {
- "name": "UDF_RUNTIME_DIR",
- "value": "/home/workspace/doris/output/be/lib/udf-runtime"
- },
- {
- "name": "LOG_DIR",
- "value": "/home/workspace/doris/output/be/log"
- },
- {
- "name": "PID_DIR",
- "value": "/home/workspace/doris/output/be/bin"
- }
- ],
- "externalConsole": false,
- "MIMode": "gdb",
- "setupCommands": [
- {
- "description": "Enable pretty-printing for gdb",
- "text": "-enable-pretty-printing",
- "ignoreFailures": true
- }
- ]
- }
- ]
-}
-```
-
-6. Click to debug
-
- You can start your debugging journey with the rest,
-
-
-
-## Debugging(LLDB)
-
-lldb's attach mode is fast than gdb，and the usage is similar to gdb. we should install plugin `CodeLLDB`, then add config to launch:
-```json
-{
- "name": "CodeLLDB attach",
- "type": "lldb",
- "request": "attach",
- "program": "${workspaceFolder}/output/be/lib/doris_be",
- "pid":"${command:pickMyProcess}"
-}
-```
-
-It should be noted that this method requires the system `glibc` version to be `2.18+`. you can refer [Get VSCode CodeLLDB plugin work on CentOS 7](https://gist.github.com/JaySon-Huang/63dcc6c011feb5bd6deb1ef0cf1a9b96) to make plugin work。
\ No newline at end of file
diff --git a/community/developer-guide/benchmark-tool.md b/community/developer-guide/benchmark-tool.md
deleted file mode 100644
index a6ed29e12a5..00000000000
--- a/community/developer-guide/benchmark-tool.md
+++ /dev/null
@@ -1,151 +0,0 @@
----
-{
- "title": "Doris BE Storage Layer Benchmark Tool",
- "language": "en"
-}
-
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Doris BE Storage Layer Benchmark Tool
-
-## usage
-
-It can be used to test the performance of some parts of the BE storage layer (for example, segment, page). According to the input data, the designated object is constructed, and the google benchmark is used for performance testing.
-
-## Compilation
-
-1. To ensure that the environment has been able to successfully compile the Doris ontology, you can refer to [Installation and deployment](/docs/install/source-install/compilation)。
-
-2. Execute`run-be-ut.sh`
-
-3. The compiled executable file is located in `./be/ut_build_ASAN/test/tools/benchmark_tool`
-
-## operator
-
-#### Use randomly generated data set for segment read test
-
-The data set will be used to write a `segment` file first, and then the time-consuming scan of the entire `segment` will be counted.
-
-> ./benchmark_tool --operation=SegmentScan --column_type=int,varchar --rows_number=10000 --iterations=0
-
-The `column_type` here can set the schema, the column type of the `segment` layer currently supports `int, char, varchar, string`, the length of the `char` type is `8`, and both `varchar` and `string` types have length restrictions Is the maximum value. The default value is `int,varchar`.
-
-The data set is generated according to the following rules.
->int: Random in [1,1000000].
-
-The data character set of string type is uppercase and lowercase English letters, and the length varies according to the type.
-> char: Length random in [1,8]。
-> varchar: Length random in [1,128]。
-> string: Length random in [1,100000]。
-
-`rows_number` indicates the number of rows of data, the default value is `10000`.
-
-`iterations` indicates the number of iterations, the benchmark will repeat the test, and then calculate the average time. If `iterations` is `0`, it means that the number of iterations is automatically selected by the benchmark. The default value is `10`.
-
-#### Use randomly generated data set for segment write test
-
-Perform time-consuming statistics on the process of adding data sets to segments and writing them to disk.
-
-> ./benchmark_tool --operation=SegmentWrite
-
-#### Use the data set imported from the file for segment read test
-
-> ./benchmark_tool --operation=SegmentScanByFile --input_file=./sample.dat
-
-The `input_file` here is the imported data set file.
-The first row of the data set file defines the schema, and each row corresponds to a row of data, and each data is separated by `,`.
-
-Example:
-```
-int,char,varchar
-123,hello,world
-321,good,bye
-```
-
-The type support is also `int`, `char`, `varchar`, `string`. Note that the data length of the `char` type cannot exceed 8.
-
-#### Use the data set imported from the file for segment write test
-
-> ./benchmark_tool --operation=SegmentWriteByFile --input_file=./sample.dat
-
-#### Use randomly generated data set for page dictionary encoding test
-
-> ./benchmark_tool --operation=BinaryDictPageEncode --rows_number=10000 --iterations=0
-
-Randomly generate varchar with a length between [1,8], and perform time-consuming statistics on encoding.
-
-#### Use randomly generated data set for page dictionary decoding test
-
-> ./benchmark_tool --operation=BinaryDictPageDecode
-
-Randomly generate varchar with a length between [1,8] and encode, and perform time-consuming statistics on decoding.
-
-## Custom test
-
-Here, users are supported to use their own functions for performance testing, which can be implemented in `/be/test/tools/benchmark_tool.cpp`.
-
-For example:
-```cpp
-void custom_run_plus() {
- int p = 100000;
- int q = 0;
- while (p--) {
- q++;
- if (UNLIKELY(q == 1024)) q = 0;
- }
-}
-void custom_run_mod() {
- int p = 100000;
- int q = 0;
- while (p--) {
- q++;
- if (q %= 1024) q = 0;
- }
-}
-```
-You can join the test by registering `CustomBenchmark`.
-```cpp
-benchmarks.emplace_back(
- new doris::CustomBenchmark("custom_run_plus", 0,
- custom_init, custom_run_plus));
-benchmarks.emplace_back(
- new doris::CustomBenchmark("custom_run_mod", 0,
- custom_init, custom_run_mod));
-```
-The `custom_init` here is the initialization step of each round of testing (not counted as time-consuming). If the user has an object that needs to be initialized, it can be implemented by a derived class of `CustomBenchmark`.
-After running, the results are as follows:
-```
-2021-08-30T10:29:35+08:00
-Running ./benchmark_tool
-Run on (96 X 3100.75 MHz CPU s)
-CPU Caches:
- L1 Data 32 KiB (x48)
- L1 Instruction 32 KiB (x48)
- L2 Unified 1024 KiB (x48)
- L3 Unified 33792 KiB (x2)
-Load Average: 0.55, 0.53, 0.39
-----------------------------------------------------------
-Benchmark Time CPU Iterations
-----------------------------------------------------------
-custom_run_plus 0.812 ms 0.812 ms 861
-custom_run_mod 1.30 ms 1.30 ms 539
-```
diff --git a/community/developer-guide/bitmap-hll-file-format.md b/community/developer-guide/bitmap-hll-file-format.md
deleted file mode 100644
index 201e99a7e26..00000000000
--- a/community/developer-guide/bitmap-hll-file-format.md
+++ /dev/null
@@ -1,64 +0,0 @@
----
-{
-"title": "Bitmap/HLL data format",
-"language": "en"
-}
-
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-## Bitmap format
-### Format description
-The bitmap in Doris uses roaring bitmap storage, and the be side uses CRoaring. The serialization format of `Roaring` is compatible in languages such as C++/java/go, while the serialization result of the format of C++ `Roaring64Map` is the same as that of `Roaring64NavigableMap` in Java. Not compatible. There are 5 types of Doris bimap, each of which is represented by one byte
-
-The bitmap serialization format in Doris is explained as follows:
-
-```
- | flag | data .....|
- <--1Byte--><--n bytes-->
-```
-
-The Flag value description is as follows:
-
-| Value | Description |
-| ---- | ------------------------------------------------------------ |
-| 0 | EMPTY, empty bitmap, the following data part is empty, the whole serialization result is only one byte |
-| 1 | SINGLE32, there is only one 32-bit unsigned integer value in the bitmap, and the next 4 bytes represent the 32-bit unsigned integer value |
-| 2 | BITMAP32, 32-bit bitmap corresponds to the type `org.roaringbitmap.RoaringBitmap` in java. The type is `roaring::Roaring` in C++, and the following data is the structure after the sequence of roaring::Roaring. You can use `org in java. .roaringbitmap.RoaringBitmap` or `roaring::Roaring` in c++ directly deserialize |
-| 3 | SINGLE64, there is only one 64-bit unsigned integer value in the bitmap, and the next 8 bytes represent the 64-bit unsigned integer value |
-| 4 | BITMAP64, 64-bit bitmap corresponds to `org.roaringbitmap.RoaringBitmap` in java; `Roaring64Map` in doris in c++. The data structure is the same as the result in the roaring library, but the serialization and deserialization methods It is different, there will be 1-8 bytes of variable-length encoding uint64 in the bitmap representation of the size. The following data is a series of multiple high-order representations of 4 bytes and 32-bit roaring bitmap serialized data repeated |
-
-C++ serialization and deserialization examples are in the `BitmapValue::write()` method in `be/src/util/bitmap_value.h` and the Java examples are in the `serialize()` `deserialize()` method in `fe/fe-common/src/main/java/org/apache/doris/common/io/BitmapValue.java`.
-
-## HLL format description
-
-Serialized data in HLL format is implemented in Doris itself. Similar to the Bitmap type, the HLL format is composed of a 1-byte flag followed by multiple bytes of data. The meaning of the flag is as follows
-
-
-
-| Value | Description |
-| ---- | ------------------------------------------------------------ |
-| 0 | HLL_DATA_EMPTY, empty HLL, the following data part is empty, the entire serialization result is only one byte |
-| 1 | HLL_DATA_EXPLICIT, the next byte is explicit The number of data blocks, followed by multiple data blocks, each data block is composed of 8 bytes in length and data, |
-| 2 | HLL_DATA_SPARSE, only non-zero values are stored, the next 4 bytes indicate the number of registers, and there are multiple register structures in the back. Each register is composed of the index of the first 2 bytes and the value of 1 byte |
-| 3 | HLL_DATA_FULL, which means that all 16 * 1024 registers have values, followed by 16 * 1024 bytes of value data |
-
-C++ serialization and deserialization examples are in the `serialize()` `deserialize()` method of `be/src/olap/hll.h`, and the Java examples are in the `serialize()` `deserialize()` method in `fe/fe-common/src/main/java/org/apache/doris/common/io/hll.java`.
diff --git a/community/developer-guide/cpp-diagnostic-code.md b/community/developer-guide/cpp-diagnostic-code.md
deleted file mode 100644
index b4fa4f042a4..00000000000
--- a/community/developer-guide/cpp-diagnostic-code.md
+++ /dev/null
@@ -1,54 +0,0 @@
----
-{
- "title": "C++ Code Diagnostic",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# C++ Code Diagnostic
-
-Doris support to use [Clangd](https://clangd.llvm.org/) and [Clang-Tidy](https://clang.llvm.org/extra/clang-tidy/)
-to diagnostic code. Clangd and Clang-Tidy already has in [LDB-toolchain](/docs/install/source-install/compilation-with-ldb-toolchain)，also can install by self.
-
-### Clang-Tidy
-Clang-Tidy can do some diagnostic cofig, config file `.clang-tidy` is in Doris root path. Compared with vscode-cpptools, clangd can provide more powerful and accurate code jumping for vscode, and integrates the analysis and quick-fix functions of clang-tidy.
-
-### Enable clangd on VSCODE
-
-First we should install clangd plugin, then edit `settings.json` or just change config on gui.
-
-```json
- "clangd.path": "ldb_toolchain/bin/clangd", //clangd path
- "clangd.arguments": [
- "--background-index",
- "--clang-tidy", //enable clang-tidy
- "--compile-commands-dir=doris/be/build_RELEASE/", //clangd should read compile_commands.json create by cmake, so you should compile once
- "--completion-style=detailed",
- "-j=5", //clangd diagnostic parallelism
- "--all-scopes-completion",
- "--pch-storage=memory",
- "--pretty",
- "-log=verbose",
- "--query-driver=ldb_toolchain/bin/*" //path of compiler
- ],
- "clangd.trace": "/home/disk2/pxl/dev/baidu/bdg/doris/core/output/clangd-server.log" //clangd log path
-```
diff --git a/community/developer-guide/cpp-format-code.md b/community/developer-guide/cpp-format-code.md
deleted file mode 100644
index 937ba7b4646..00000000000
--- a/community/developer-guide/cpp-format-code.md
+++ /dev/null
@@ -1,104 +0,0 @@
----
-{
- "title": "C++ Format Code",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# C++ code formatting
-
-Doris uses clang-format for code formatting, and provides a package script in the build-support directory:
-
-* `clang-format.sh`.
-
- Format the C/C++ code in the `be/src` and `be/test` directories.
-
-* `check-format.sh`.
-
- Check the C/C++ code format in the `be/src` and `be/test` directories, and output diff, but the content of the file will not be modified.
-
-## Code style customization
-
-The code style of Doris is slightly modified on the basis of Google Style and is customized as a `.clang-format` file located in the root directory of Doris.
-
-Currently, the `.clang-format` configuration file is adapted to versions above clang-format-13.0.1.
-
-The code that you do not want to be formatted is recorded in the `.clang-format-ignore` file. These codes usually come from third-party code bases, and it is recommended to keep the original code style.
-
-## Environmental preparation
-
-You need to download and install clang-format, or you can use the clang-format plug-in provided by IDE or Editor, as described below.
-
-### Download and install clang-format
-
-It is recommended to use NPM to install clang-format 14 (different versions of clang-format may produce different code formats, it is recommended to use version 14):
-
-`npm install clang-format@1.6.0`
-
-Ubuntu: `apt-get install clang-format`
-
-The current version is 10.0, you can also specify the old version, for example: `apt-get install clang-format-9`. It is recommended to compile version 14.0 from source code.
-
-Mac: `brew install clang-format`
-
-Centos 7:
-
-The version of clang-format installed by centos yum is too old and supports too few StyleOptions. It is recommended to compile version 14.0 from source code.
-
-LDB toolchain:
-
-If you are using [LDB toolchain](/docs/install/source-install/compilation-with-ldb-toolchain),
-the latest version (>= v0.11) of [LDB toolchain](https://github.com/amosbird/ldb_toolchain_gen/releases) has already included clang-format with 13.0.1 version.
-
-### clang-format plugin
-
-Clion IDE can use the plug-in "ClangFormat", search and download in `File->Setting->Plugins`. But the version can’t match
-The version of the clang-format program matches, judging from the supported StyleOption, it should be lower than clang-format-9.0.
-
-## How to use
-
-### Command line operation
-
-cd to the root directory of Doris, and then execute the following command:
-
-`build-support/clang-format.sh`
-
-> Note: The `clang-format.sh` script requires python 3 to be installed on your machine
-
-### Use clang-format in IDE or Editor
-
-#### Clion
-
-If Clion uses a plug-in, just click `Reformat Code`.
-
-#### VS Code
-
-VS Code needs to install the extension Clang-Format, but you need to provide the location of the clang-format execution program.
-
-Open the VS Code configuration page, directly search for "clang_format", and fill in
-
-```
-"clang_format_path": "$clang-format path$",
-"clang_format_style": "file"
-```
-
-Then, right-click on `Format Document`.
diff --git a/community/developer-guide/debug-tool.md b/community/developer-guide/debug-tool.md
deleted file mode 100644
index 4e7f50e9ba2..00000000000
--- a/community/developer-guide/debug-tool.md
+++ /dev/null
@@ -1,333 +0,0 @@
----
-{
- "title": "Debug Tool",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Debug Tool
-
-In the process of using and developing Doris, we often encounter scenarios that need to debug Doris. Here are some common debugging tools.
-
-**The name of the BE binary that appears in this doc is `doris_be`, which was `palo_be` in previous versions.**
-
-## FE debugging
-
-Fe is a java process. Here are just a few simple and commonly used java debugging commands.
-
-1. Statistics of current memory usage details
-
- ```
- jmap -histo:live pid > 1. jmp
- ```
-
- This command can enumerate and sort the memory occupation of living objects. (replace PID with Fe process ID)
-
- ```
- num #instances #bytes class name
- ----------------------------------------------
- 1: 33528 10822024 [B
- 2: 80106 8662200 [C
- 3: 143 4688112 [Ljava.util.concurrent.ForkJoinTask;
- 4: 80563 1933512 java. lang.String
- 5: 15295 1714968 java. lang.Class
- 6: 45546 1457472 java. util. concurrent. ConcurrentHashMap$Node
- 7: 15483 1057416 [Ljava.lang.Object;
- ```
-
- You can use this method to view the total memory occupied by the currently living objects (at the end of the file) and analyze which objects occupy more memory.
-
- Note that this method will trigger fullgc because `: live 'is specified.
-
-2. Check JVM memory usage
-
- ```
- jstat -gcutil pid 1000 1000
- ```
-
- This command can scroll through the memory usage of each region of the current JVM. (replace PID with Fe process ID)
-
- ```
- S0 S1 E O M CCS YGC YGCT FGC FGCT GCT
- 0.00 0.00 22.61 3.03 95.74 92.77 68 1.249 5 0.794 2.043
- 0.00 0.00 22.61 3.03 95.74 92.77 68 1.249 5 0.794 2.043
- 0.00 0.00 22.61 3.03 95.74 92.77 68 1.249 5 0.794 2.043
- 0.00 0.00 22.92 3.03 95.74 92.77 68 1.249 5 0.794 2.043
- 0.00 0.00 22.92 3.03 95.74 92.77 68 1.249 5 0.794 2.043
- ```
-
- The main focus is on the percentage of old area (o) (3% in the example). If the occupancy is too high, oom or fullgc may occur.
-
-3. Print Fe thread stack
-
- ```
- jstack -l pid > 1. js
- ```
-
- This command can print the thread stack of the current Fe. (replace PID with Fe process ID).
- `-L ` the parameter will detect whether there is deadlock at the same time. This method can check the operation of Fe thread, whether there is deadlock, where it is stuck, etc.
-
-## BE debugging
-
-### Preparing
-
-[pprof](https://github.com/google/pprof): from gperftools, it is used to transform the content generated by gperftools into a format that is easy for people to read, such as PDF, SVG, text, etc.
-
-[graphviz](http://www.graphviz.org/): in the absence of this library, pprof can only be converted to text format, but this way is not easy to view. After the library is installed, pprof can be converted to SVG, PDF and other formats, and the call relationship is clearer.
-
-[perf](https://perf.wiki.kernel.org/index.php/main_page): Linux kernel comes with performance analysis tool. [here](http://www.brendangregg.com/perf.html) there are some examples of perf usage.
-
-[flamegraph](https://github.com/brendangregg/flamegraph): a visualization tool used to show the output of perf in the form of flame graph.
-
-### Memory
-
-Debugging memory is generally divided into two aspects. One is whether the total amount of memory use is reasonable. On the one hand, the excessive amount of memory use may be due to memory leak in the system, on the other hand, it may be due to improper use of program memory. The second is whether there is a problem of memory overrun and illegal access, such as program access to memory with an illegal address, use of uninitialized memory, etc. For the debugging of memory, we usually use [...]
-
-#### Log
-
-When we find that the memory usage is too large, we can first check the be.out log to see if there is a large memory application. Because of the TCMalloc currently used by Doris to manage memory, when a large memory application is encountered, the stack of the application will be printed to the be.out file. The general form is as follows:
-
-```
-tcmalloc: large alloc 1396277248 bytes == 0x3f3488000 @ 0x2af6f63 0x2c4095b 0x134d278 0x134bdcb 0x133d105 0x133d1d0 0x19930ed
-```
-
-This indicates that Doris be is trying to apply memory of '1396277248 bytes' on this stack. We can use the 'addr2line' command to restore the stack to a letter that we can understand. The specific example is shown below.
-
-```
-$ addr2line -e lib/doris_be 0x2af6f63 0x2c4095b 0x134d278 0x134bdcb 0x133d105 0x133d1d0 0x19930ed
-
-/home/ssd0/zc/palo/doris/core/thirdparty/src/gperftools-gperftools-2.7/src/tcmalloc.cc:1335
-/home/ssd0/zc/palo/doris/core/thirdparty/src/gperftools-gperftools-2.7/src/tcmalloc.cc:1357
-/home/disk0/baidu-doris/baidu/bdg/doris-baidu/core/be/src/exec/hash_table.cpp:267
-/home/disk0/baidu-doris/baidu/bdg/doris-baidu/core/be/src/exec/hash_table.hpp:86
-/home/disk0/baidu-doris/baidu/bdg/doris-baidu/core/be/src/exec/hash_join_node.cpp:239
-/home/disk0/baidu-doris/baidu/bdg/doris-baidu/core/be/src/exec/hash_join_node.cpp:213
-thread.cpp:?
-```
-
-#### HEAP PROFILE
-
-Sometimes the application of memory is not caused by the application of large memory, but by the continuous accumulation of small memory. Then there is no way to locate the specific application information by viewing the log, so you need to get the information through other ways.
-
-At this time, we can take advantage of TCMalloc's [heapprofile](https://gperftools.github.io/gperftools/heapprofile.html). If the heapprofile function is set, we can get the overall memory application usage of the process. The usage is to set the 'heapprofile' environment variable before starting Doris be. For example:
-
-```
-export HEAPPROFILE=/tmp/doris_be.hprof
-./bin/start_be.sh --daemon
-```
-
-In this way, when the dump condition of the heapprofile is met, the overall memory usage will be written to the file in the specified path. Later, we can use the 'pprof' tool to analyze the output content.
-
-```
-$ pprof --text lib/doris_be /tmp/doris_be.hprof.0012.heap | head -30
-
-Using local file lib/doris_be.
-Using local file /tmp/doris_be.hprof.0012.heap.
-Total: 668.6 MB
- 610.6 91.3% 91.3% 610.6 91.3% doris::SystemAllocator::allocate_via_malloc (inline)
- 18.1 2.7% 94.0% 18.1 2.7% _objalloc_alloc
- 5.6 0.8% 94.9% 63.4 9.5% doris::RowBatch::RowBatch
- 5.1 0.8% 95.6% 7.1 1.1% butil::ResourcePool::add_block (inline)
- 3.7 0.5% 96.2% 3.7 0.5% butil::iobuf::create_block (inline)
- 3.4 0.5% 96.7% 3.4 0.5% butil::FlatMap::init
- 3.2 0.5% 97.2% 5.2 0.8% butil::ObjectPool::add_block (inline)
- 2.6 0.4% 97.6% 2.6 0.4% __gnu_cxx::new_allocator::allocate (inline)
- 2.0 0.3% 97.9% 2.0 0.3% butil::ObjectPool::add_block_group (inline)
- 2.0 0.3% 98.2% 2.0 0.3% butil::ResourcePool::add_block_group (inline)
- 1.7 0.3% 98.4% 1.7 0.3% doris::SegmentReader::_load_index
-```
-
-Contents of each column of the above documents:
-
-* Column 1: the memory size directly applied by the function, in MB
-* Column 4: the total memory size of the function and all the functions it calls.
-* The second column and the fifth column are the proportion values of the first column and the fourth column respectively.
-* The third column is the cumulative value of the second column.
-
-Of course, it can also generate call relation pictures, which is more convenient for analysis. For example, the following command can generate a call graph in SVG format.
-
-```
-pprof --svg lib/doris_be /tmp/doris_be.hprof.0012.heap > heap.svg
-```
-
-**NOTE: turning on this option will affect the execution performance of the program. Please be careful to turn on the online instance.**
-
-#### pprof remote server
-
-Although heapprofile can get all the memory usage information, it has some limitations. 1. Restart be. 2. You need to enable this command all the time, which will affect the performance of the whole process.
-
-For Doris be, you can also use the way of opening and closing the heap profile dynamically to analyze the memory application of the process. Doris supports the [remote server debugging of gperftools](https://gperftools.github.io/gperftools/pprof_remote_servers.html). Then you can use 'pprof' to directly perform dynamic head profile on the remote running Doris be. For example, we can check the memory usage increment of Doris through the following command
-
-```
-$ pprof --text --seconds=60 http://be_host:be_webport/pprof/heap
-
-Total: 1296.4 MB
- 484.9 37.4% 37.4% 484.9 37.4% doris::StorageByteBuffer::create
- 272.2 21.0% 58.4% 273.3 21.1% doris::RowBlock::init
- 157.5 12.1% 70.5% 157.5 12.1% doris::RowBatch::RowBatch
- 90.7 7.0% 77.5% 90.7 7.0% doris::SystemAllocator::allocate_via_malloc
- 66.6 5.1% 82.7% 66.6 5.1% doris::IntegerColumnReader::init
- 47.9 3.7% 86.4% 47.9 3.7% __gnu_cxx::new_allocator::allocate
- 20.8 1.6% 88.0% 35.4 2.7% doris::SegmentReader::_load_index
- 12.7 1.0% 89.0% 12.7 1.0% doris::DecimalColumnReader::init
- 12.7 1.0% 89.9% 12.7 1.0% doris::LargeIntColumnReader::init
- 12.7 1.0% 90.9% 12.7 1.0% doris::StringColumnDirectReader::init
- 12.3 0.9% 91.9% 12.3 0.9% std::__cxx11::basic_string::_M_mutate
- 10.4 0.8% 92.7% 10.4 0.8% doris::VectorizedRowBatch::VectorizedRowBatch
- 10.0 0.8% 93.4% 10.0 0.8% doris::PlainTextLineReader::PlainTextLineReader
-```
-
-The output of this command is the same as the output and view mode of heap profile, which will not be described in detail here. Statistics will be enabled only during execution of this command, which has a limited impact on process performance compared with heap profile.
-
-#### LSAN
-
-[LSAN](https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer) is an address checking tool, GCC has been integrated. When we compile the code, we can enable this function by turning on the corresponding compilation options. When the program has a determinable memory leak, it prints the leak stack. Doris be has integrated this tool, only need to compile with the following command to generate be binary with memory leak detection version.
-
-```
-BUILD_TYPE=LSAN ./build.sh
-```
-
-When the system detects a memory leak, it will output the corresponding information in be. Out. For the following demonstration, we intentionally insert a memory leak code into the code. We insert the following code into the `open` function of `StorageEngine`.
-
-```
- char* leak_buf = new char[1024];
- strcpy(leak_buf, "hello world");
- LOG(INFO) << leak_buf;
-```
-
-We get the following output in be.out
-
-```
-=================================================================
-==24732==ERROR: LeakSanitizer: detected memory leaks
-
-Direct leak of 1024 byte(s) in 1 object(s) allocated from:
- #0 0xd10586 in operator new[](unsigned long) ../../../../gcc-7.3.0/libsanitizer/lsan/lsan_interceptors.cc:164
- #1 0xe333a2 in doris::StorageEngine::open(doris::EngineOptions const&, doris::StorageEngine**) /home/ssd0/zc/palo/doris/core/be/src/olap/storage_engine.cpp:104
- #2 0xd3cc96 in main /home/ssd0/zc/palo/doris/core/be/src/service/doris_main.cpp:159
- #3 0x7f573b5eebd4 in __libc_start_main (/opt/compiler/gcc-4.8.2/lib64/libc.so.6+0x21bd4)
-
-SUMMARY: LeakSanitizer: 1024 byte(s) leaked in 1 allocation(s).
-```
-
-From the above output, we can see that 1024 bytes have been leaked, and the stack information of memory application has been printed out.
-
-**NOTE: turning on this option will affect the execution performance of the program. Please be careful to turn on the online instance.**
-
-**NOTE: if the LSAN switch is turned on, the TCMalloc will be automatically turned off**
-
-#### ASAN
-
-Except for the unreasonable use and leakage of memory. Sometimes there will be memory access illegal address and other errors. At this time, we can use [ASAN](https://github.com/google/sanitizers/wiki/addresssanitizer) to help us find the cause of the problem. Like LSAN, ASAN is integrated into GCC. Doris can open this function by compiling as follows
-
-```
-BUILD_TYPE=ASAN ./build.sh
-```
-
-Execute the binary generated by compilation. When the detection tool finds any abnormal access, it will immediately exit and output the stack illegally accessed in be.out. The output of ASAN is the same as that of LSAN. Here we also actively inject an address access error to show the specific content output. We still inject an illegal memory access into the 'open' function of 'storageengine'. The specific error code is as follows
-
-```
- char* invalid_buf = new char[1024];
- for (int i = 0; i < 1025; ++i) {
- invalid_buf[i] = i;
- }
- LOG(INFO) << invalid_buf;
-```
-
-We get the following output in be.out
-
-```
-=================================================================
-==23284==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61900008bf80 at pc 0x00000129f56a bp 0x7fff546eed90 sp 0x7fff546eed88
-WRITE of size 1 at 0x61900008bf80 thread T0
- #0 0x129f569 in doris::StorageEngine::open(doris::EngineOptions const&, doris::StorageEngine**) /home/ssd0/zc/palo/doris/core/be/src/olap/storage_engine.cpp:106
- #1 0xe2c1e3 in main /home/ssd0/zc/palo/doris/core/be/src/service/doris_main.cpp:159
- #2 0x7fa5580fbbd4 in __libc_start_main (/opt/compiler/gcc-4.8.2/lib64/libc.so.6+0x21bd4)
- #3 0xd30794 (/home/ssd0/zc/palo/doris/core/output3/be/lib/doris_be+0xd30794)
-
-0x61900008bf80 is located 0 bytes to the right of 1024-byte region [0x61900008bb80,0x61900008bf80)
-allocated by thread T0 here:
- #0 0xdeb040 in operator new[](unsigned long) ../../../../gcc-7.3.0/libsanitizer/asan/asan_new_delete.cc:82
- #1 0x129f50d in doris::StorageEngine::open(doris::EngineOptions const&, doris::StorageEngine**) /home/ssd0/zc/palo/doris/core/be/src/olap/storage_engine.cpp:104
- #2 0xe2c1e3 in main /home/ssd0/zc/palo/doris/core/be/src/service/doris_main.cpp:159
- #3 0x7fa5580fbbd4 in __libc_start_main (/opt/compiler/gcc-4.8.2/lib64/libc.so.6+0x21bd4)
-
-SUMMARY: AddressSanitizer: heap-buffer-overflow /home/ssd0/zc/palo/doris/core/be/src/olap/storage_engine.cpp:106 in doris::StorageEngine::open(doris::EngineOptions const&, doris::StorageEngine**)
-```
-
-From this message, we can see that at the address of `0x61900008bf80`, we tried to write a byte, but this address is illegal. We can also see the application stack of the address `[0x61900008bb80, 0x61900008bf80]`.
-
-**NOTE: turning on this option will affect the execution performance of the program. Please be careful to turn on the online instance.**
-
-**NOTE: if the ASAN switch is turned on, the TCMalloc will be automatically turned off**
-
-In addition, if stack information is output in be.out, but there is no function symbol, then we need to handle it manually to get readable stack information. The specific processing method needs a script to parse the output of ASAN. At this time, we need to use [asan_symbolize](https://llvm.org/svn/llvm-project/compiler-rt/trunk/lib/asan/scripts/asan_symbolize.py) to help with parsing. The specific usage is as follows:
-
-```
-cat be.out | python asan_symbolize.py | c++filt
-```
-
-With the above command, we can get readable stack information.
-
-### CPU
-
-When the CPU idle of the system is very low, it means that the CPU of the system has become the main bottleneck. At this time, it is necessary to analyze the current CPU usage. For the be of Doris, there are two ways to analyze the CPU bottleneck of Doris.
-
-#### pprof
-
-Because Doris has integrated and compatible with GPERF rest interface, users can analyze remote Doris be through the 'pprof' tool. The specific usage is as follows:
-
-```
-pprof --svg --seconds=60 http://be_host:be_webport/pprof/profile > be.svg
-```
-
-In this way, a CPU consumption graph of be execution can be generated.
-
-
-
-#### perf + flamegragh
-
-This is a quite common CPU analysis method. Compared with `pprof`, this method must be able to log in to the physical machine of the analysis object. However, compared with pprof, which can only collect points on time, perf can collect stack information through different events. The specific usage is as follows:
-
-```
-perf record -g -p be_pid -- sleep 60
-```
-
-This command counts the CPU operation of be for 60 seconds and generates perf.data. For the analysis of perf.data, the command of perf can be used for analysis.
-
-```
-perf report
-```
-
-The analysis results in the following pictures
-
-
-
-To analyze the generated content. Of course, you can also use flash graph to complete the visual display.
-
-```
-perf script | ./FlameGraph/stackcollapse-perf.pl | ./FlameGraph/flamegraph.pl > be.svg
-```
-
-This will also generate a graph of CPU consumption at that time.
-
-
diff --git a/community/developer-guide/docker-dev.md b/community/developer-guide/docker-dev.md
deleted file mode 100644
index ccf855dd60d..00000000000
--- a/community/developer-guide/docker-dev.md
+++ /dev/null
@@ -1,161 +0,0 @@
----
-{
- "title": "Doris Docker quick build development environment",
- "language": "en"
-}
-
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Doris Docker quick build development environment
-
-## Related detailed document navigation
-
-- [Developing mirror compilation using Docker](/docs/install/source-install/compilation)
-- [Deploying Doris](/docs/install/install-deploy)
-- [VSCode Be Development Debugging](./be-vscode-dev)
-
-## Environment preparation
-
-- Install Docker
-- VSCode
- - Remote plugin
-
-## Build image
-
-create dockerfile
-
-VSCode replace all by using Ctrl-d
-
-- <!!! your user !!!>
-- <!!! your user password !!!>
-- <!!! root password !!!>
-- <!!! your git email !!!>
-- <!!! your git username !!!>
-
-```dockerfile
-FROM apache/incubator-doris:build-env-latest
-
-USER root
-WORKDIR /root
-RUN echo '<!!! root password !!!>' | passwd root --stdin
-
-RUN yum install -y vim net-tools man wget git mysql lsof bash-completion \
- && cp /var/local/thirdparty/installed/bin/thrift /usr/bin
-
-# safer usage, create new user instead of using root
-RUN yum install -y sudo \
- && useradd -ms /bin/bash <!!! your user !!!> && echo <!!! your user password !!!> | passwd <!!! your user !!!> --stdin \
- && usermod -a -G wheel <!!! your user !!!>
-
-USER <!!! your user !!!>
-WORKDIR /home/<!!! your user !!!>
-RUN git config --global color.ui true \
- && git config --global user.email "<!!! your git email !!!>" \
- && git config --global user.name "<!!! your git username !!!>"
-
-# install zsh and oh my zsh, easier to use on, you can remove it if you don't need it
-USER root
-RUN yum install -y zsh \
- && chsh -s /bin/zsh <!!! your user !!!>
-USER <!!! your user !!!>
-RUN wget https://github.com/robbyrussell/oh-my-zsh/raw/master/tools/install.sh -O - | zsh \
- && git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions \
- && git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting
-```
-
-run build command
-
-```bash
-docker build -t doris .
-```
-
-run image
-
-note! [problems with mounting](../../docs/install/source-install/compilation)
-
-> See the link above: It is recommended to run the image by mounting the local Doris source code directory as a volume .....
-
-if you are developing on windows, mounting may cause cross-filesystem access problems, please consider setting it manually
-
-```bash
-docker run -it doris:latest /bin/bash
-```
-
-if you installed zsh, replace plugins in ~/.zshrc after running the container
-
-```
-plugins=(git zsh-autosuggestions zsh-syntax-highlighting)
-```
-
-create directory and download doris
-
-```bash
-su <your user>
-mkdir code && cd code
-git clone https://github.com/apache/doris.git
-```
-
-## Compile
-
-Note:
-
-use the following command first time compiling
-
-```bash
-sh build.sh --clean --be --fe --ui
-```
-
-it is because build-env-for-0.15.0 version image upgraded thrift(0.9 -> 0.13), so you need to use --clean command to force use new version of thrift to generate code files, otherwise it will cause incompatibilities.
-
-compile Doris
-
-```bash
-sh build.sh
-```
-
-## Run
-
-manually create `meta_dir` metadata storage location, default value is `${DORIS_HOME}/doris-meta`
-
-```bash
-mkdir meta_dir
-```
-
-launch FE
-
-```bash
-cd output/fe
-sh bin/start_fe.sh --daemon
-```
-
-launch BE
-
-```bash
-cd output/be
-sh bin/start_be.sh --daemon
-```
-
-mysql-client connect
-
-```bash
-mysql -h 127.0.0.1 -P 9030 -u root
-```
diff --git a/community/developer-guide/fe-eclipse-dev.md b/community/developer-guide/fe-eclipse-dev.md
deleted file mode 100644
index dbbb130a805..00000000000
--- a/community/developer-guide/fe-eclipse-dev.md
+++ /dev/null
@@ -1,226 +0,0 @@
----
-{
- "title": "Setting FE dev env - Eclipse",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Setting FE dev env using Eclipse
-
-## Preparation
-
-* JDK 1.8+
-* Maven 3.x+
-* Eclipse, with [M2Eclipse](http://www.eclipse.org/m2e/) installed
-
-### Code Generation
-
-The FE module requires part of the generated code, such as Thrift, Protobuf, Jflex, CUP and other frameworks.
-
-#### Windows
-
-1. Under Linux, enter the source code directory `fe` and execute the following command:
-
- ```
- mvn generate-sources
- ```
-
-2. If use windows as development environment, then package the generated `fe/fe-core/target/generated-sources` directory:
-
- `fe/fe-core/target/ && tar czf java.tar.gz generated-sources/`
-
-3. Copy `java.tar.gz` to the `fe/fe-core/target/` directory of the development environment and unzip
-
- ```
- cp java.tar.gz /path/to/doris/fe/fe-core/target/
- cd /path/to/doris/fe/fe-core/target/ && tar xzf java.tar.gz
- ```
-
-#### MacOS
-
-We can use maven to do code generation or in other words build FE project
-directly on MacOS.
-
-1. Install apache thrift from source code or `brew` if you don't have one
-2. Create a directory `thirdparty/installed/bin`, and link command thrift into
- it.
-
- ```
- mkdir -p thirdparty/installed/bin
- ln -s ${thrift_installed_full_path} thirdparty/installed/bin/thrift
- ```
-3. Call `maven` to build FE project, if something went wrong, check you
- `$JAVA_HOME`, java version and newly installed command `thrift` work
- correctly.
-
- ```
- cd fe && mvn package -DskipTests=true -Dos.arch=x86_64
- ```
-
-Option `-Dos.arch=x86_64` in step 3 is for compatibility of Apple's M series
-CPU.
-
-Note:
-0. cup and jfex use java jar to do code generation, the process is platform
- independent.
-1. Code generation of protobuf is done by `protoc-jar-maven-plugin`, which uses
- precompiled binaries of different archs to make it seems platform independent.
-2. thrift is the only one relies on `thirdparty/installed`, which needs to be
- built from source. We will make it independent on `thirdparty/installed`
- someday (TODO).
-
-## Import FE project
-
-### Import as eclipse project
-
-1. In the `fe/` directory of the development environment, execute the following command to generate the Eclipse project file:
-
- `cd /path/to/doris/fe/ && mvn -npr eclipse:eclipse -Dskip.plugin=true`
-
- After the execution is completed, the `.project` and `.classpath` files will be generated in the `fe/` directory.
-
-2. Import FE project
-
- * Open Eclipse, choose `File -> Import`.
- * Choose `General -> Existing Projects into Workspace`.
- * `Select root directory` and choose `fe/` directory, click `Finish` to finish.
- * Right click the project, and choose `Build Path -> Configure Build Path`.
- * In the `Java Build Path` dialog, choose the `Source` tab, click `Add Folder`, and select the `java/` directory that was copied and unzipped before adding.
- * Click `Apply and Close` to finish.
-
-At this point, FE project import is complete. The project directory in Eclipse is roughly as follows:
-
-
-
-### Import as maven project
-
-We are able to build FE with maven, if we have done all the operations related
-to MacOS. And we are now of course able to import FE project as a maven project
-in eclipse.
-
-In eclipse menu `File` select `Import -> Maven -> Existing Maven Projects`,
-Choose doris fe directory to finish import. It's recommended to use working set
-to manage the all modules of FE.
-
-Enjoy developing and debugging FE in eclipse!
-
-## Run Unit Test
-
-Right-click on the unit test file you want to run and select `Run As -> JUnit Test`. (If you want to debug, select `Debug As -> JUnit Test`).
-
-If the following error occurs:
-
-```
-java.lang.Exception: Method xxxx should have no parameters
-```
-
-Then right-click the unit test file and select `Run As -> Run Configurations...`. (If you want to debug, select `Debug As -> Debug Configurations...`).
-
-Add to the `VM arguments` in the `Arguments` tab:
-
-```
--javaagent:${settings.localRepository}/org/jmockit/jmockit/1.48/jmockit-1.48.jar
-```
-
-Among them, `${settings.localRepository}` should be replaced with the path of the maven library path, such as:
-
-```
--javaagent:/Users/cmy/.m2/repository/org/jmockit/jmockit/1.48/jmockit-1.48.jar
-```
-
-Then just run `Run/Debug`.
-
-## Run FE
-
-You can directly start an FE process in Eclipse to facilitate debugging the code.
-
-1. Create a runtime directory
-
- ```
- mkdir /path/to/doris/fe/run/
- cd /path/to/doris/fe/run/
- mkdir conf/ log/ palo-meta/
- ```
-
-2. Create configuration file
-
- Create the configuration file `fe.conf` in the `conf/` directory created in the first step. You can directly copy `conf/fe.conf` in the source directory and make simple changes.
-
-3. Find the `src/main/java/org/apache/doris/PaloFe.java` file in Eclipse, right-click and select `Run As -> Run Configurations...`. Add the following environment variables to the `Environment` tab:
-
- * `DORIS_HOME: /path/to/doris/fe/run/`
- * `PID_DIR: /path/to/doris/fe/run/`
- * `LOG_DIR: /path/to/doris/fe/run/log`
-
-4. Right-click `PaloFe.java` and select `Run As -> Java Application` to start FE.
-
-To run a UT, a FE service will be started at first. And then, UT cases execute as client and test corresponding logics. When errors occur, only client logs will be print in UT logs. If you need more information from **server logs**, you can view logs in `${DORIS_HOME}/fe/mocked`.
-
-## Code Update
-
-### Imported as eclipse project
-1. Update lexical and grammar files or proto and thrift files
-
- If you modified `fe/src/main/cup/sql_parser.cup` or `fe/src/main/jflex/sql_scanner.flex` file or proto and thrift files. You need to execute the following commands in the `fe/` directory:
-
- ```
- mvn generate-sources
- ```
-
- Then refresh the project in Eclipse.
-
-2. Update maven dependencies
-
- If you update the dependency in `fe/pom.xml`, you need to execute the following command in the `fe/` directory:
-
- `mvn -npr eclipse:eclipse -Dskip.plugin=true`
-
- Then refresh the project in Eclipse. If it cannot be updated, it is recommended to delete the project and import it again according to this document.
-
-### Imported as maven project
-
-1. Update lexical and grammar files or proto and thrift files
- ```
- cd fe && mvn package -DskipTests=true -Dos.arch=x86_64
- ```
-2. Update maven dependencies, in eclipse `Package Explorer` right click on the
- project `maven -> update project...`
-
-3. Refresh project in eclipse
-
-## Imports Order
-
-In order to maintain the Imports order of Java, please perform the following operations to set the Imports Order of the project.
-
-1. Create the file `fe_doris.importorder` and write the following:
-
- ```
- #Organize Import Order
- #Wed Jul 01 16:42:47 CST 2020
- 4=javax
- 3=java
- 2=org
- 1=com
- 0=org.apache.doris
- ```
-
-2. Open Eclipse Preferences, select `Java -> Code Style -> Organize Imports`. Click `Import` to import the above file.
diff --git a/community/developer-guide/fe-idea-dev.md b/community/developer-guide/fe-idea-dev.md
deleted file mode 100644
index 40fc8ccf06c..00000000000
--- a/community/developer-guide/fe-idea-dev.md
+++ /dev/null
@@ -1,191 +0,0 @@
----
-{
- "title": "Setting Up dev env for FE - IntelliJ IDEA",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Setting Up Development Environment for FE using IntelliJ IDEA
-
-## 1. Environmental Preparation
-
-* Git
-* JDK1.8+
-* IntelliJ IDEA
-* Maven (Optional, IDEA shipped embedded Maven3)
-
-1. Git clone codebase from https://github.com/apache/doris.git
-
-
-2. Use IntelliJ IDEA to open the code root directory
-
-
-3. If your are only interested in FE module, and for some reason you can't or don't want to compile full thirdparty libraries,
- the minimum tool required for FE module is `thrift`, so you can manually install `thrift` and copy or create a link of
- the executable `thrift` command to `./thirdparty/installed/bin`.
- ```
- Doris build against `thrift` 0.13.0 ( note : `Doris` 0.15 and later version build against `thrift` 0.13.0 , the previous version is still `thrift` 0.9.3)
-
- Windows:
- 1. Download：`http://archive.apache.org/dist/thrift/0.13.0/thrift-0.13.0.exe`
- 2. Copy：copy the file to `./thirdparty/installed/bin`
-
- MacOS:
- 1. Download：`brew install thrift@0.13.0`
- 2. Establish soft connection：
- `mkdir -p ./thirdparty/installed/bin`
- # For ARM macOS
- `ln -s /opt/homebrew/Cellar/thrift@0.13.0/0.13.0/bin/thrift ./thirdparty/installed/bin/thrift`
- # For Intel macOS
- `ln -s /usr/local/Cellar/thrift@0.13.0/0.13.0/bin/thrift ./thirdparty/installed/bin/thrift`
-
- Note：The error that the version cannot be found may be reported when MacOS execute `brew install thrift@0.13.0`. The solution is execute at the terminal as follows:
- 1. `brew tap-new $USER/local-tap`
- 2. `brew extract --version='0.13.0' thrift $USER/local-tap`
- 3. `brew install thrift@0.13.0`
- Reference link: `https://gist.github.com/tonydeng/02e571f273d6cce4230dc8d5f394493c`
- ```
-
-4. Go to `./fe` folder and run the following maven command to generate sources.
-
- ```
- mvn generate-sources
- ```
-
- If fails, run following command.
-
- ```
- mvn clean install -DskipTests
- ```
-
- You can also use IDE embedded GUI tools to run maven command to generate sources
-
-
-
-If you are developing on the OS which lack of support to run `shell script` and `make` such as Windows, a workround here
-is generate codes in Linux and copy them back. Using Docker should also be an option.
-
-5. If a help document has not been generated, go to the docs directory and run`sh build_help_zip.sh`，
- Then copy help-resource.zip from build to fe/fe-core/target/classes
-
-## 2. Debug
-
-1. Import `./fe` into IDEA
-
-2. Follow the picture to create the folders (The directory may exist in the new version. If it exists, skip it, otherwise create it.)
-
-
-
-3. Build `ui` project , and copy files from directory `ui/dist` into directory `webroot` ( you can skip this step , if you don't need `Doris` UI )
-
-## 3. Custom FE configuration
-
-Copy below content into `conf/fe.conf` and tune it to fit your environment(Note: If developed using`Mac`, since`docker for Mac`does not support`Host`mode,`be`needs to be exposed using`-p` and `fe.conf` `priority_networks` configured to be accessible within the container, such as WIFI Ip).
-
-```
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-#####################################################################
-## The uppercase properties are read and exported by bin/start_fe.sh.
-## To see all Frontend configurations,
-## see fe/src/org/apache/doris/common/Config.java
-#####################################################################
-
-# the output dir of stderr and stdout
-LOG_DIR = ${DORIS_HOME}/log
-
-DATE = `date +%Y%m%d-%H%M%S`
-JAVA_OPTS="-Xmx2048m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$DATE"
-
-# For jdk 9+, this JAVA_OPTS will be used as default JVM options
-JAVA_OPTS_FOR_JDK_9="-Xmx4096m -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xlog:gc*:$DORIS_HOME/log/fe.gc.log.$DATE:time"
-
-##
-## the lowercase properties are read by main program.
-##
-
-# INFO, WARN, ERROR, FATAL
-sys_log_level = INFO
-
-# store metadata, create it if it is not exist.
-# Default value is ${DORIS_HOME}/doris-meta
-# meta_dir = ${DORIS_HOME}/doris-meta
-
-http_port = 8030
-rpc_port = 9020
-query_port = 9030
-edit_log_port = 9010
-mysql_service_nio_enabled = true
-
-# Choose one if there are more than one ip except loopback address.
-# Note that there should at most one ip match this list.
-# If no ip match this rule, will choose one randomly.
-# use CIDR format, e.g. 10.10.10.0/24
-# Default value is empty.
-# priority_networks = 10.10.10.0/24;192.168.0.0/16
-
-# Advanced configurations
-# log_roll_size_mb = 1024
-# sys_log_dir = ${DORIS_HOME}/log
-# sys_log_roll_num = 10
-# sys_log_verbose_modules =
-# audit_log_dir = ${DORIS_HOME}/log
-# audit_log_modules = slow_query, query
-# audit_log_roll_num = 10
-# meta_delay_toleration_second = 10
-# qe_max_connection = 1024
-# max_conn_per_user = 100
-# qe_query_timeout_second = 300
-# qe_slow_log_ms = 5000
-
-```
-
-## 4. Setting Environment Variables
-
-Follow the picture to set runtime Environment Variables in IDEA
-
-
-
-## 5. Config options
-
-Because part of the dependency is `provided`, idea needs to do a special config. Click on the right `Modify Options` in the `Run/Debug Configurations` setting. Check the `Add Dependencies with "Provided" scope to classpath` option.
-
-
-
-## 6. Start FE
-
-Having fun with Doris FE!
diff --git a/community/developer-guide/fe-vscode-dev.md b/community/developer-guide/fe-vscode-dev.md
deleted file mode 100644
index 95dc4650c55..00000000000
--- a/community/developer-guide/fe-vscode-dev.md
+++ /dev/null
@@ -1,80 +0,0 @@
----
-{
- "title": "FE development and debugging environment - Visual Studio Code (VSCode)",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Apache Doris Be development and debugging in VS Code
-
-Some developers are building FE development environment on a development machine/WSL/docker, but this kind of development environment is not supported for local development, some developers are used to use VSCode to configure remote develop and debug.
-
-## Preparation
-
-* JDK11+ (Java Extension Pack need JDK11+) (author is creating a `lib` directory under `home`, and install [JDK11](https://github.com/adoptium/temurin11-binaries/releases/) and JDK8 in it, and use them for `Extensions` and `Compilation`)
-* VSCode
- + Extension Pack for Java
- + Remote Extensions
-
-## Download code for compilation
-
-1. https://github.com/apache/doris.git Download the doris source code
-
-2. use VSCode to open the code `/fe` directory
-
-## Setting for VSCode
-
-Create `settings.json` in `.vscode/` , and set settings:
-
-* `"java.configuration.runtimes"`
-* `"java.jdt.ls.java.home"` -- must set it to the directory of JDK11+, used for vscode-java plugin
-* `"maven.executable.path"` -- maven path，for maven-language-server plugin
-
-example:
-
-```json
-{
- "java.configuration.runtimes": [
- {
- "name": "JavaSE-1.8",
- "path": "/!!!path!!!/jdk-1.8.0_191"
- },
- {
- "name": "JavaSE-11",
- "path": "/!!!path!!!/jdk-11.0.14.1+1",
- "default": true
- },
- ],
- "java.jdt.ls.java.home": "/!!!path!!!/jdk-11.0.14.1+1",
- "maven.executable.path": "/!!!path!!!/maven/bin/mvn"
-}
-```
-
-## Build
-
-Other articles have already explained:
-* [Build with LDB toolchain ](/docs/install/source-install/compilation-with-ldb-toolchain)
-* ......
-
-In order to debug, you need to add debugging parameters when fe starts, such as `-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005` .
-
-In `incubator-doris/output/fe/bin/start_fe.sh` , after `$JAVA $final_java_opt` add this param.
diff --git a/community/developer-guide/github-checks.md b/community/developer-guide/github-checks.md
deleted file mode 100644
index 41b4afdfc8b..00000000000
--- a/community/developer-guide/github-checks.md
+++ /dev/null
@@ -1,31 +0,0 @@
----
-{
- "title": "Github Checks",
- "language": "en"
-}
-
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Github Checks
-
-Please refer to: [Github Checks Guidance](https://cwiki.apache.org/confluence/display/DORIS/Github+Checks+Guidance)
-
diff --git a/community/developer-guide/how-to-share-blogs.md b/community/developer-guide/how-to-share-blogs.md
deleted file mode 100644
index 56ed87e0728..00000000000
--- a/community/developer-guide/how-to-share-blogs.md
+++ /dev/null
@@ -1,39 +0,0 @@
----
-{
- "title": "How to share blogs",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# How to share a blog
-
-The Doris community welcomes the sharing of Doris-related articles. Once merged, these articles will appear on the official Doris website.
-
-Articles include, but are not limited to.
-
-* Doris usage tips
-* Introduction to Doris features
-* Doris system tuning
-* How Doris works
-* Doris business scenarios in practice
-
-For details, please go to: https://github.com/apache/doris-website [README](https://github.com/apache/doris-website)
diff --git a/community/developer-guide/java-format-code.md b/community/developer-guide/java-format-code.md
deleted file mode 100644
index bad37cc7f15..00000000000
--- a/community/developer-guide/java-format-code.md
+++ /dev/null
@@ -1,99 +0,0 @@
----
-{
- "title": "Java Format Code",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Java Format Code
-
-The formatting of the Java part of the code in Doris is usually done automatically by the IDE. Only the general format rules are listed here. For developer, you need to set the corresponding code styles in different IDEs according to the format rules.
-
-## Import Order
-
-```
-org.apache.doris
-<blank line>
-third party package
-<blank line>
-standard java package
-<blank line>
-```
-
-* Do not use `import *`
-* Do not use `import static`
-
-## Check when compile
-
-Now, when compiling with `caven`, `CheckStyle` checks are done by default. This will slightly slow down compilation. If you want to skip checkstyle, please use the following command to compile
-```
-mvn clean install -DskipTests -Dcheckstyle.skip
-```
-
-## Checkstyle Plugin
-
-Now we have `formatter-check` in `CI` to check the code format.
-
-### IDEA
-
-If you use `IDEA` to develop Java code, please install `Checkstyle-IDEA` plugin.
-
-Setting the `checkstyle.xml` file in `Tools->Checkstyle`.
-
-Click the plus sign under Configuration File, select `Use a local Checkstyle file`, and select the `fe/check/checkstyle/checkstyle.xml` file.
-
-**NOTE:** Make sure that the version of `Checkstyle` is 9.3 or newer (the latest version is recommended).
-
-
-
-**You can use `Checkstyle-IDEA` plugin to check `Checkstyle` of your code real-time.**
-
-
-
-### VS Code
-
-If you use VS Code to develop Java code, please install `Checkstyle for Java` plugin, and config according to the [document](https://code.visualstudio.com/docs/java/java-linting) and the picture
-
-## IDEA
-
-### Auto format code
-
-The automatic formatting function of `IDEA` is also recommended.
-
-Go to `Preferences->Editor->Code Style->Java` click the config sign and select `Import Scheme`，select `IntelliJ IDEA code style XML`，and select the `build-support/IntelliJ-code-format.xml` file.
-
-### Auto rearrange code
-
-Checkstyle will check declarations order according to [Class and Interface Declarations](https://www.oracle.com/java/technologies/javase/codeconventions-fileorganization.html#1852) .
-
-After add the `build-support/IntelliJ-code-format.xml` file. Click `Code/Rearrange Code` to auto rearrange code.
-
-
-
-## Remove unused header
-
-**CTRL + ALT + O --->** to remove the unused imports in windows.
-
-Auto remove unused header and reorder according to configure xml:
-
-Click `Preferences->Editor->Auto Import->Optimize Imports on the Fly`
-
diff --git a/community/developer-guide/minidump.md b/community/developer-guide/minidump.md
deleted file mode 100644
index 1203b753a20..00000000000
--- a/community/developer-guide/minidump.md
+++ /dev/null
@@ -1,177 +0,0 @@
----
-{
- "title": "Minidump",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Minidump(removed)
-
-> Minidump has been removed, it's useless in real online environment and instead introduces additional bugs
-
-Minidump is a file format defined by Microsoft for reporting errors after program crashes. It includes thread information, register information, call stack information, etc. at the time of the crash, which helps developers quickly locate the problem.
-
-Unlike [Coredump](https://en.wikipedia.org/wiki/Core_dump), Minidump files are smaller and easier to report and network transmission. Coredump file will contain a complete memory image, so the volume may be dozens or hundreds of GB. The Minidump file only contains the call stack and register information of the key thread, so the size is usually only MB level.
-
-[Breakpad](https://github.com/google/breakpad) is a cross-platform crash dump and analysis framework and tool collection. Users can use Breakpad to conduct self-service analysis of Minidump files. You can also collect Minidump files and report them to Doris cluster operation and maintenance or developers.
-
-## How to enable Minidump
-
-Minidump function is a function introduced in Doris 0.15.0 or later. This function is controlled by the following configuration files of BE:
-
-* `disable_minidump`
-
- Whether to enable Minidump function. The default is false, which means it is turned on.
-
-* `minidump_dir`
-
- The storage directory of the Minidump file. The default is `${DORIS_HOME}/Minidump/`
-
-* `max_minidump_file_size_mb`
-
- Minidump file size limit. The default is 200MB. If the size exceeds the threshold, breakpad will try to reduce the information recorded in the file, such as the number of threads and the number of registers to introduce the Minidump file size. But this is only an expected value, and the actual file size may be larger than the set value.
-
-* `max_minidump_file_number`
-
- The maximum number of Minidump files to keep. The default is 10, which means that the most recent 10 files are kept.
-
-## How to generate Minidump
-
-There are two ways to generate Minidump:
-
-1. The program crashes
-
- When the program encounters a problem and crashes, it will automatically generate a Minidump file. The following information will appear in be.out at this time:
-
- ```
- Minidump created at: /doris/be/Minidump/4f8d4fe5-15f8-40a3-843109b3-d49993f3.dmp
- *** Aborted at 1636970042 (unix time) try "date -d @1636970042" if you are using GNU date ***
- PC: @ 0x1b184e4 doris::OlapScanNode::scanner_thread()
- *** SIGSEGV (@0x0) received by PID 71567 (TID 0x7f173a5df700) from PID 0; stack trace: ***
- @ 0x220c992 google::(anonymous namespace)::FailureSignalHandler()
- @ 0x7f174fb5e1d0 (unknown)
- @ 0x1b184e4 doris::OlapScanNode::scanner_thread()
- @ 0x15a19af doris::PriorityThreadPool::work_thread()
- @ 0x21d9107 thread_proxy
- @ 0x7f174fb53f84 start_thread
- @ 0x7f174f943ddf __GI___clone
- @ 0x0 (unknown)
- ```
-
- Among them, `/doris/be/Minidump/4f8d4fe5-15f8-40a3-843109b3-d49993f3.dmp` is the Minidump file. And the following stack is the call stack information where the program crashed.
-
-2. Manual trigger
-
- The user can actively send the SIGUSR1 signal to the BE process to trigger Minidump. For example, use the following command:
-
- ```
- kill -s SIGUSR1 71567
- ```
-
- 71567 is the process id (pid) of BE. After that, the following information will appear in be.out:
-
- ```
- Receive signal: SIGUSR1
- Minidump created at: /doris/be/Minidump/1af8fe8f-3d5b-40ea-6b76ad8f-0cf6756f.dmp
- ```
-
- Among them, `Receive signal: SIGUSR1` means that this is a Minidump operation triggered by the user. Following is the location of the Minidump file.
-
- The Minidump operation manually triggered by the user will not kill the BE process and will not generate an error stack in be.out.
-
-## How to analyze Minidump
-
-We can use various tools provided by breakpad to analyze Minidump to see the cause of the error.
-
-### Get the breakpad tool
-
-Users can go to [Breakpad](https://github.com/google/breakpad) code base to download and compile breakpad. For the compilation method, please refer to the `build_breakpad()` method in [thirdparty/vars.sh](https://github.com/apache/incubator-doris/blob/master/thirdparty/vars.sh) in the Doris source code library.
-
-You can also find various tools compiled by breakpad from the `/var/local/thirdparty/installed/bin` directory of the image container in the version 1.4.2 and above of the Docker compiled image provided by Doris.
-
-### Analyze Minidump
-
-We can use the following two methods to analyze Minidump files.
-
-1. Dump into coredump file
-
- Use the `minidump-2-core` tool provided by breakpad to dump the Minidump file into a coredump file:
-
- ```
- ./minidump-2-core /doris/be/Minidump/1af8fe8f-3d5b-40ea-6b76ad8f-0cf6756f.dmp> 1.coredump
- ```
-
- Then we can use the gdb tool to analyze the coredump file:
-
- ```
- gdb lib/doris_be -c 1.coredump
- ```
-
-2. Generate a readable call stack
-
- The Minidump file only contains the address of the call stack, and we need to map these addresses to the actual function file location. Therefore, we first need to generate the symbol table `doris_be.sym` of the BE binary file through `dump_syms`:
-
- ```
- ./dump_syms ./lib/doris_be> doris_be.sym
- ```
-
- Next, we need the information in the first row of the symbol table to build a corresponding symbol table directory.
-
- ```
- head -n1 doris_be.sym
- ```
-
- The above command will print the first line of doris_be.sym as follows:
-
- ```
- MODULE Linux x86_64 137706CC745F5EC3EABBF730D4B229370 doris_be
- ```
-
- Then we create a directory structure:
-
- ```
- mkdir -p ./symbols/doris_be/137706CC745F5EC3EABBF730D4B229370
- ```
-
- The `doris_be` and `137706CC745F5EC3EABBF730D4B229370` in the directory path must be consistent with the first line of the doris_be.sym file. Then we move the doris_be.sym file to this directory:
-
- ```
- cp doris_be.sym ./symbols/doris_be/137706CC745F5EC3EABBF730D4B229370
- ```
-
- Finally, we can use `minidump_stackwalk` to produce readable call stack information:
-
- ```
- minidump_stackwalk 4f8d4fe5-15f8-40a3-843109b3-d49993f3.dmp ./symbols/> readable.stack
- ```
-
- Among them, `4f8d4fe5-15f8-40a3-843109b3-d49993f3.dmp` is a minidump file. `./symbols/` is the previously created directory containing doris_be.sym. `readable.stack` redirects the generated results to this file. At the same time, when this command is executed, some program running logs will be flashed on the screen, so you can ignore it.
-
- At this point, we have obtained a readable thread call stack file: readable.stack. It contains the call stack information of all threads when the BE program is writing the Minidump file, and the corresponding register information. Among them, `Crash reason` explains why the program crashed. If it is `DUMP_REQUESTED`, it means that this is a Minidump triggered by the user.
-
- We can filter out the register information with the following command to get a clear view of the call stack:
-
- ```
- grep -v = readable.stack |grep -v "Found by" |vi-
- ```
-
- The result is similar to the thread call stack information obtained through the pstack command.
diff --git a/community/developer-guide/regression-testing.md b/community/developer-guide/regression-testing.md
deleted file mode 100644
index c5981fd3cae..00000000000
--- a/community/developer-guide/regression-testing.md
+++ /dev/null
@@ -1,30 +0,0 @@
----
-{
- "title": "Regression Testing",
- "language": "en"
-}
-
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Regression Testing
-
-TODO
\ No newline at end of file
diff --git a/community/feedback.md b/community/feedback.md
deleted file mode 100644
index 9774c1f0539..00000000000
--- a/community/feedback.md
+++ /dev/null
@@ -1,45 +0,0 @@
----
-{ "title": "Feedback", "language": "zh-CN" }
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Feedback
-
-## Feedback Tips
-
-- Describe the problem in detail, providing important details (logs, critical errors, investigations/analyses that have been performed)
-- Minimize the problem, analyze the problem by yourself - narrow the scope of the problem, easy to answer / troubleshoot
-- Give the minimization specific reproduction steps
-- focus on the problem itself, as unrelated to the business/scenario as possible (if necessary, provide as clear and unambiguous context as possible)
-
-## Feedback Method
-
-If you find any problems or suggestions while using Doris, you can provide feedback in the following ways:
-
-1. Mailing List
-
- You can send an email with any subject and content to `dev-subscribe@doris.apache.org`, and then follow the instructions to subscribe to this mailing list.
-
- After that, you can send an email to `dev@doris.apache.org` to describe the problem or suggestion.
-
-2. Github
-
- Submit a [Github Issue](https://github.com/apache/doris/issues/new/choose)
diff --git a/community/gitter.md b/community/gitter.md
deleted file mode 100644
index b0cbb94e3fb..00000000000
--- a/community/gitter.md
+++ /dev/null
@@ -1,59 +0,0 @@
----
-{
- "title": "Gitter Manual",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Gitter Manual
-
-## Gitter introduction
-
-Gitter is a Markdown-enabled instant messaging software for developers. It can be seamlessly linked to github, PR on Github can be linked in chat, relevant historical records of discussions can be retained, historical records can be queried, and Chinese and English can be supported.
-
-Like many other open source projects, Doris can use Gitter as an instant messaging medium for technology exchange and community development. This article describes how to use Gitter to participate in Doris's open source development and community development.
-
-## Log in using links
-
-Entering [https://gitter.im/apache-doris/Lobby](https://gitter.im/apache-doris/Lobby) in the browser automatically jumps to the Doris community chat room interface on Gitter.
-
-Click on the `SIGN IN TO START TALKING` below to login. It can support two login modes, Github account or Twitter account. The author uses Github account to login, as follows:
-
-
-
-After clicking on the red circle, enter the Github account and password to log into the chat room and start technical or community discussions:
-
-
-
-You can use Gitter as well as Wechat, and get functions that are more comfortable for developers and technicians than Wechat, such as directly mentioning an activity for discussion, directly searching history chat records, etc.
-
-Don't forget to click on the Pentagon in the upper right corner to collect, which will make the chat room easier for you to find.
-
-## Install Mobile Client
-
-You can download Gitter's mobile client and participate in technical discussions on your mobile phone at any time and anywhere. Download links:
-
-[https://gitter.im/home](https://gitter.im/home)
-
-## Search Gitter and join Doris Community Chat Room
-
-Partners already using Gitter log in directly to search for `apache-doris` and can join the chat room when they find it. Other functions are used in the same chapter, which is not discussed here.
diff --git a/community/how-to-contribute/commit-format-specification.md b/community/how-to-contribute/commit-format-specification.md
deleted file mode 100644
index b42b9a17f8b..00000000000
--- a/community/how-to-contribute/commit-format-specification.md
+++ /dev/null
@@ -1,164 +0,0 @@
----
-{
- "title": "Commit Format Specification",
- "language": "en"
-}
-
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-
-
-## Commit Format Specification
-
-Commit is divided into ‘ title ’ and ‘ content ’ , the title should be lowercase and the contents should be capitalized in principle .
-
-1. Title
-
- `[<type>](<scope>) <subject> (#pr)`
-
- * `<type>`
-
- The types of this pull request are limited to the following types (all lowercase)
-
- * fix: Bug fix
- * feature: New feature
- * feature-wip: Feature works-in-porgress.
- * improvement: Optimization and improvement for the original feature.
- * style: Code style adjustment
- * typo: Code or Document correction
- * refactor: Code refactoring (no function changes involved)
- * performance/optimize: Performance optimization
- * test: Addition or repair of unit test
- * chore: Modification of build tool
- * revert: Revert a previous commit
- * deps: Modification of third-party dependency Library
- * community: Such as modification of Github issue template.
-
- Some tips：
-
- 1. If there are multiple types in one commit, multiple types need to be added
- 2. If code refactoring brings performance improvement, [refactor][optimize] can be added at the same time
- 3. There should be no other types than those listed above. If necessary, you need to add new types to this document.
-
- * `<scope>`
-
- Because there are many functional modules, only part of the module scope of the design submitted this time is listed here, which will be continuously improved according to the needs in the future.
-
- * planner
- * meta
- * storage
- * stream-load
- * broker-load
- * routine-load
- * sync-job
- * export
- * executor
- * spark-connector
- * flink-connector
- * datax
- * log
- * cache
- * config
- * vectorization
- * docs
- * profile
-
- Some tips：
-
- 1. Try to use options that already exist in the list. If you need to add, please update this document in time
-
- * `<subject>`
-
- The title should clearly indicate the main contents of this commit as far as possible.
-
-2. Content
-
- commit message should follow the following format:
-
- ```
- issue：#7777
-
- your message
- ```
-
- 1. If there is no issue, it can be left blank. Issue can also appear in message.
- 1. One line should not exceed 100 characters
-
-3. Example
-
- ```
- [fix](executor) change DateTimeValue's memory layout to load (#7022)
-
- Change DateTimeValue memory's layout to old to fix compatibility problems.
- ```
-
- ```
- [feat](log) extend logger interface, support structured log output(#6600)
-
- Support structured logging.
- ```
-
- ```
- [fix][feat-opt](executor)(load)(config) fix some memory bugs (#6699)
-
- 1. Fix a memory leak in `collect_iterator.cpp` (Fix #6700)
- 2. Add a new BE config `max_segment_num_per_rowset` to limit the num of segment in new rowset.(Fix #6701)
- 3. Make the error msg of stream load more friendly.
- ```
-
- ```
- [feat-opt](load) Reduce the number of segments when loading a large volume data in one batch (#6947)
-
- ## Case
-
- In the load process, each tablet will have a memtable to save the incoming data,
- and if the data in a memtable is larger than 100MB, it will be flushed to disk
- as a `segment` file. And then a new memtable will be created to save the following data.
-
- Assume that this is a table with N buckets(tablets). So the max size of all memtables
- will be `N * 100MB`. If N is large, it will cost too much memory.
-
- So for memory limit purpose, when the size of all memtables reach a threshold(2GB as default),
- Doris will try to flush all current memtables to disk(even if their size are not reach 100MB).
-
- So you will see that the memtable will be flushed when it's size reach `2GB/N`, which maybe much
- smaller than 100MB, resulting in too many small segment files.
-
- ## Solution
-
- When decide to flush memtable to reduce memory consumption, NOT to flush all memtable,
- but to flush part of them.
-
- For example, there are 50 tablets(with 50 memtables). The memory limit is 1GB,
- so when each memtable reach 20MB, the total size reach 1GB, and flush will occur.
-
- If I only flush 25 of 50 memtables, then next time when the total size reach 1GB,
- there will be 25 memtables with size 10MB, and other 25 memtables with size 30MB.
- So I can flush those memtables with size 30MB, which is larger than 20MB.
-
- The main idea is to introduce some jitter during flush to ensure the small unevenness
- of each memtable, so as to ensure that flush will only be triggered when the memtable
- is large enough.
-
- In my test, loading a table with 48 buckets, mem limit 2G, in previous version,
- the average memtable size is 44MB, after modification, the average size is 82MB
- ```
diff --git a/community/how-to-contribute/contribute-doc.md b/community/how-to-contribute/contribute-doc.md
deleted file mode 100644
index 44291710072..00000000000
--- a/community/how-to-contribute/contribute-doc.md
+++ /dev/null
@@ -1,313 +0,0 @@
----
-{
- "title": "Docs Contribute",
- "language": "en"
-}
-
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Doris Documentation Contributions
-
-Here we mainly introduce how to modify and contribute to Doris' documents.
-
-How to submit your document modification, please refer to
-
-[Contribute to Doris](./)
-
-[Code Submission Guidelines](./pull-request)
-
-Documents of historical versions can be submitted directly on [apache/doris-website](https://github.com/apache/doris-website) PR, if it is the latest version, it needs to be in [apache/doris-website] (https://github.com/apache/doris-website) and the [apache/doris](https://github.com/apache/doris) codebase at the same time commit changes.
-
-The following introduces the directory structure of the Doris Website site to facilitate users to modify and submit documents
-
-## Doris Website Directory Structure
-
-
-
-````
-.
-├── README.md
-├── babel.config.js
-├── blog
-│ ├── 1.1 Release.md
-│ ├── Annoucing.md
-│ ├── jd.md
-│ ├── meituan.md
-│ ├── release-note-0.15.0.md
-│ ├── release-note-1.0.0.md
-│ └── xiaomi.md
-├── build.sh
-├── community
-│ ├── design
-│ │ ├── Flink-doris-connector-Design.md
-│ │ ├── doris_storage_optimization.md
-│ │ ├── grouping_sets_design.md
-│ │ └── metadata-design.md
-│ ├──
-├── docs
-│ ├── admin-manual
-│ │ ├── cluster-management
-│ │ ├── config
-│ │ ├── data-admin
-│ │ ├── http-actions
-│ │ ├── maint-monitor
-│ │ ├── multi-tenant.md
-│ │ ├── optimization.md
-│ │ ├── privilege-ldap
-│ │ ├── query-profile.md
-│ │ └── sql-interception.md
-│ ├──
-├── docusaurus.config.js
-├── i18n
-│ └── en-US
-│ ├── code.json
-│ ├── docusaurus-plugin-content-blog
-│ ├── docusaurus-plugin-content-docs
-│ ├── docusaurus-plugin-content-docs-community
-│ └── docusaurus-theme-classic
-├── package.json
-├── sidebars.json
-├── sidebarsCommunity.json
-├── src
-│ ├── components
-│ │ ├── Icons
-│ │ ├── More
-│ │ ├── PageBanner
-│ │ └── PageColumn
-│ ├──
-├── static
-│ ├── images
-│ │ ├── Bloom_filter.svg.png
-│ │ ├── .....
-│ └── js
-│ └── redirect.js
-├── tree.out
-├── tsconfig.json
-├── versioned_docs
-│ ├── version-0.15
-│ │ ├── administrator-guide
-│ │ ├── best-practices
-│ │ ├── extending-doris
-│ │ ├── getting-started
-│ │ ├── installing
-│ │ ├── internal
-│ │ ├── sql-reference
-│ │ └── sql-reference-v2
-│ └── version-1.0
-│ ├── administrator-guide
-│ ├── benchmark
-│ ├── extending-doris
-│ ├── faq
-│ ├── getting-started
-│ ├── installing
-│ ├── internal
-│ ├── sql-reference
-│ └── sql-reference-v2
-├── versioned_sidebars
-│ ├── version-0.15-sidebars.json
-│ └── version-1.0-sidebars.json
-├── versions.json
-
-````
-
-Directory structure description:
-
-1. Blog Directory
-
- - The English blog directory is under the blog in the root directory, and the English files of all blogs are placed in this directory
- - The directory of the Chinese blog is in the `i18n/zh-CN/docusaurus-plugin-content-blog` directory, all Chinese blog files are placed under this
- - The file names of Chinese and English blogs should be the same
-
-2. Document Content Directory
-
- - The latest version of the English document content is under docs in the root directory
-
- - The version of the English documentation is under `versioned_docs/` in the root directory
-
- - This directory only holds documents from historical versions
-
- ````
- .
- ├── version-0.15
- │ ├── administrator-guide
- │ ├── best-practices
- │ ├── extending-doris
- │ ├── getting-started
- │ ├── installing
- │ ├── internal
- │ ├── sql-reference
- │ └── sql-reference-v2
- └── version-1.0
- ├── administrator-guide
- ├── benchmark
- ├── extending-doris
- ├── faq
- ├── getting-started
- ├── installing
- ├── internal
- ├── sql-reference
- └── sql-reference-v2
- ````
-
- - Versioning of English documents is under `versioned_sidebars` in the root directory
-
- ````
- .
- ├── version-0.15-sidebars.json
- └── version-1.0-sidebars.json
- ````
-
- The json file here is written according to the directory structure of the corresponding version
-
- - Chinese documentation at `i18n/zh-CN/docusaurus-plugin-content-docs`
-
- - Below this corresponds to different version directories and json files corresponding to the version, as follows
-
- current is the current latest version of the document. The example corresponds to version 1.1. When modifying, according to the document version to be modified, find the corresponding file modification in the corresponding directory and submit it.
-
- ````
- .
- ├── current
- │ ├── admin-manual
- │ ├── advanced
- │ ├── benchmark
- │ ├── data-operate
- │ ├── data-table
- │ ├── ecosystem
- │ ├── faq
- │ ├── get-starting
- │ ├── install
- │ ├── sql-manual
- │ └── summary
- ├── current.json
- ├── version-0.15
- │ ├── administrator-guide
- │ ├── best-practices
- │ ├── extending-doris
- │ ├── getting-started
- │ ├── installing
- │ ├── internal
- │ ├── sql-reference
- │ └── sql-reference-v2
- ├── version-0.15.json
- ├── version-1.0
- │ ├── administrator-guide
- │ ├── benchmark
- │ ├── extending-doris
- │ ├── faq
- │ ├── getting-started
- │ ├── installing
- │ ├── internal
- │ ├── sql-reference
- │ └── sql-reference-v2
- └── version-1.0.json
- ````
-
- - Version Json file
-
- Current.json corresponds to the Chinese translation of the latest version of the document, for example:
-
- ````json
- {
- "version.label": {
- "message": "1.1",
- "description": "The label for version current"
- },
- "sidebar.docs.category.Getting Started": {
- "message": "Quick Start",
- "description": "The label for category Getting Started in sidebar docs"
- }
- .....
- }
- ````
-
- Here `sidebar.docs.category.Getting Started` corresponds to `label` in `sidebars.json` in the root directory
-
- For example, the `sidebar.docs.category.Getting Started` just now corresponds to the `sidebar` prefix and the structure in `sidebars.json`
-
- The first is `sidebar + "." + docs + ".'" + [ type ] + [ label ] `.
-
- ````json
- {
- "docs": [
- {
- "type": "category",
- "label": "Getting Started",
- "items": [
- "get-starting/get-starting"
- ]
- },
- {
- "type": "category",
- "label": "Doris Introduction",
- "items": [
- "summary/basic-summary"
- ]
- }
- .....
- }
- ````
-
- - Support label translation in the Chinese version json file, no need to describe the document hierarchy, which is described in the `sidebar.json` file
-
- - All documents must be in English, and Chinese can only be displayed. If English is not written, you can create an empty file, otherwise Chinese documents will not be displayed. This applies to all blogs, documents, and community content
-
-2. Community Documentation
-
- This document does not distinguish between versions and is generic
-
- - English documentation is under the `community/` directory in the root directory.
-
- - Chinese documentation is under `i18n/zh-CN/docusaurus-plugin-content-docs-community/` directory.
-
- - The directory structure of community documents is controlled in the `sidebarsCommunity.json` file in the root directory,
-
- - The Chinese translation corresponding to the community documentation directory structure is in the `i18n/zh-CN/docusaurus-plugin-content-docs-community/current.json` file
-
- ````json
- {
- "version.label": {
- "message": "Next",
- "description": "The label for version current"
- },
- "sidebar.community.category.How to Contribute": {
- "message": "Contribution Guidelines",
- "description": "The label for category How to Contribute in sidebar community"
- },
- "sidebar.community.category.Release Process & Verification": {
- "message": "Version release and verification",
- "description": "The label for category Release Process & Verification in sidebar community"
- },
- "sidebar.community.category.Design Documents": {
- "message": "Design document",
- "description": "The label for category Design Documents in sidebar community"
- },
- "sidebar.community.category.Developer Guide": {
- "message": "Developer's Manual",
- "description": "The label for category Developer Guide in sidebar community"
- }
- }
- ````
-
-3. Pictures
-
- All images are in the `static/images` directory
\ No newline at end of file
diff --git a/community/how-to-contribute/contributor-guide.md b/community/how-to-contribute/contributor-guide.md
deleted file mode 100644
index 8e96cfa3356..00000000000
--- a/community/how-to-contribute/contributor-guide.md
+++ /dev/null
@@ -1,92 +0,0 @@
----
-{
- "title": "Contributor Guide",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Contributor Guide
-
-## New Contributor Guidelines
-
-### Subscribe to the public mailing lists
-
-If you haven’t yet, subscribe to {dev,commits}@doris.apache.org mailing lists.
-Commits mailing list is especially important because all of Github Issue, Pull Request and build notifications are sent there.
-
-### Subscribe to the private mailing list
-
-Subscribe to private@doris.apache.org by sending an email to private-subscribe@doris.apache.org.
-Keep in mind that this list is private and your subscription needs to be approved by a moderator.
-If you are PPMC, you can subscribe the mailing list through [Subscription Helper](https://whimsy.apache.org/committers/subscribe)
-
-### Link your ASF and Github account
-
-We use Github for managing issues and user contributions (pull requests).
-As such, you need to link your Github.com account with your ASF account using [Gitbox](https://gitbox.apache.org/setup/).
-This way you will get write access to [Doris](https://github.com/apache/doris) repository
-and you will be able to manage issues and pull request directly through our Github repository.
-
-## Code Review Guidelines
-
-1. Always maintain a high standard of review so that the quality of the entire product can be better guaranteed.
-
-2. Changes to the architecture or the user interface need to be fully discussed by the community. This can be either in a mail group or in a GitHub issue.
-
-3. Test coverage. The added logic needs to be covered by a corresponding test. When the old code for which there is no test is changed, this requirement can be appropriately relaxed.
-
-4. Documentation. Newly added features must be documented, otherwise such code is not allowed.
-
-5. Readability of the code. If reviewers are not very clear about the logic of the code, then you can ask the contributor to explain the logic. And writing sufficient comments in the code to explain the logic is encouraged.
-
-6. Try to give a clear conclusion at the end of your comments, "approve" or "change request". If it's a minor issue, you can just leave a comment.
-
-7. Recommend to submit a "+1 Comment" rather than a "+1 Approve" to indicate that it looks good to me but I am not sure whether this part of the function is correct. It needs someone else's approve.
-
-8. Respect each other and learn from each other. Maintain a polite tone when commenting, try to give reasons for the suggestions.
-
-## Pull Request Guidelines
-
-1. During a PR, there are three types of roles. Contributor: the PR submitter; Reviewer: the person who needs to make code-level review on the PR; and Moderator: the person responsible for coordinating the entire PR process. For example, the moderator should assign reviewers, push the author to make changes according to the comments, set different tags for the PR, and merge the PR etc. For a specific PR, a person may act in different roles, such as when a contributor submitted one PR, he [...]
-
-2. Contributors can assign a PR to itself as a moderator. After assigning it to themselves, the other contributor will know that this PR has been in charge.
-
-3. The contributor is encouraged to act as a moderator for its own PR.
-
-4. The reviewer needs to perform a code-level review, refer Code Review Guidelines
-
-5. Once a reviewer has commented on a PR, they need to keep following up on subsequent changes to that PR.
-
-6. A PR must get at least a +1 approved from committer who is not the author.
-
-7. After the first +1 to the PR, wait at least one working day before merging. The main purpose is to wait for the rest of the community to come to review.
-
-8. For architecture or user interface changes, a PR needs to get at least 3 +1's to merge.
-
-9. Regression cases should pass before merging.
-
-10. Moderator needs to make sure all comments are resolved before merging.
-
-11. Select "squash and merge" to merge.
-
-12. When there is a disagreement about a modification, try to discuss the resolution. If the discussion doesn't work out, it can be resolved by a vote in private@doris.apache.org by the majority rules.
-
diff --git a/community/how-to-contribute/how-to-be-a-committer.md b/community/how-to-contribute/how-to-be-a-committer.md
deleted file mode 100644
index 5da0e6db647..00000000000
--- a/community/how-to-contribute/how-to-be-a-committer.md
+++ /dev/null
@@ -1,37 +0,0 @@
----
-{
- "title": "How to Become Doris Committer",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# How to Become Doris Committer
-
-In an Apache project, developers have three roles:
-
-1. Contributor: When the developer's code is officially merged into the code, the developer automatically becomes the project's Contributor.
-2. Committer: Committer has permission to merge into the code base. Committers are generally elected by the PMC (Project Management Committee).
-3. PMC Member: a member of the project management committee. Have the right to vote on project decisions (such as version releases, etc.). It is generally elected by the PMC (Project Management Committee) through voting.
-
-Different roles have different rights and obligations. There are no strict conditions on how to become a Committer or PMC Member, you can refer to here for more information:
-
-[Guidance for committer promotion](https://cwiki.apache.org/confluence/display/DORIS/Guidance+for+committer+promotion)
diff --git a/community/how-to-contribute/how-to-contribute.md b/community/how-to-contribute/how-to-contribute.md
deleted file mode 100644
index b4566c23f47..00000000000
--- a/community/how-to-contribute/how-to-contribute.md
+++ /dev/null
@@ -1,85 +0,0 @@
----
-{
- "title": "Contribute to Doris",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Contribute to Doris
-
-Thank you very much for your interest in the Doris project. We welcome your suggestions, comments (including criticisms) and contributions to the Doris project.
-
-Your suggestions, comments and contributions on Doris can be made directly through GitHub's [Issues](https://github.com/apache/doris/issues/new/choose).
-
-There are many ways to participate in and contribute to Doris projects: code implementation, test writing, process tool improvement, document improvement, and so on. Any contribution will be welcomed and you will be added to the list of contributors. Further, with sufficient contributions, you will have the opportunity to become a Committer of Apache with Apache mailbox and be included in the list of [Apache Committers](http://people.apache.org/committer-index.html).
-
-Any questions, you can contact us to get timely answers, including Wechat, Gitter (GitHub instant messaging tool), e-mail and so on.
-
-## Initial contact
-
-For the first time in Doris community, you can:
-
-* Follow [Doris Github](https://github.com/apache/doris)
-* Subscribe to our [mailing list](./subscribe-mail-list);
-* Join Doris Wechat Group (add WeChat-ID: morningman-cmy, note: join Doris Group) and ask questions at any time.
-* Enter Doris's [Slack](https://join.slack.com/t/apachedoriscommunity/shared_invite/zt-11jb8gesh-7IukzSrdea6mqoG0HB4gZg)
-
-Learn the development trends of Doris project in time and give your opinions on the topics you are concerned about.
-
-## Doris's code and documentation
-
-As you can see from [GitHub](https://github.com/apache/doris), Apache Doris (incubating) code base mainly consists of three parts: Frontend (FE), Backend (BE) and Broker (to support file reading on external storage systems such as HDFS). Documents are mainly the wiki on Doris website and GitHub, as well as the online help manual when running Doris. Details of these components can be found in the following table:
-
-| Component Name | Component Description | Related Language|
-|--------|----------------------------|----------|
-| [Frontend daemon (FE)](https://github.com/apache/doris) | consists of a query coordinator and a metadata manager | Java|
-| [Backend daemon (BE)](https://github.com/apache/doris) | Responsible for storing data and executing query fragments | C++|
-| [Broker](https://github.com/apache/doris) | Read HDFS data to Doris | Java|
-| [Website](https://github.com/apache/doris-website) | Doris Website | Markdown |
-| [Manager](https://github.com/apache/doris-manager) | Doris Manager | Java |
-| [Flink-Connector](https://github.com/apache/doris-flink-connector) | Doris Flink Connector | Java |
-| [Spark-Connector](https://github.com/apache/doris-spark-connector) | Doris Spark Connector | Java |
-| Doris Runtime Help Document | Online Help Manual at Doris Runtime | Markdown|
-
-## Improving documentation
-
-Documentation is the most important way for you to understand Apache Doris, and it's where we need help most!
-
-Browse the document, you can deepen your understanding of Doris, can also help you understand Doris's function and technical details, if you find that the document has problems, please contact us in time;
-
-If you are interested in improving the quality of documents, whether it is revising the address of a page, correcting a link, and writing a better introductory document, we are very welcome!
-
-Most of our documents are written in markdown format, and you can modify and submit document changes directly through `docs/` in [GitHub](https://github.com/apache/doris). If you submit code changes, you can refer to [Pull Request](./pull-request).
-
-## If a Bug or problem is found
-
-If a Bug or problem is found, you can directly raise a new Issue through GitHub's [Issues](https://github.com/apache/doris/issues/new/choose), and we will have someone deal with it regularly.
-
-You can also fix it yourself by reading the analysis code (of course, it's better to talk to us before that, maybe someone has fixed the same problem) and submit a [Pull Request](./pull-request).
-
-## Modify the code and submit PR (Pull Request)
-
-You can download the code, compile and install it, deploy and run it for a try (refer to the [compilation document](/docs/install/source-install/compilation) to see if it works as you expected. If you have problems, you can contact us directly, ask questions or fix them by reading and analyzing the source code.
-
-Whether it's fixing Bugs or adding Features, we're all very welcome. If you want to submit code to Doris, you need to create a new branch for your submitted code from the fork code library on GitHub to your project space, add the source project upstream, and submit PR.
-
-About how to submit a PR refer to [Pull Request](./pull-request).
diff --git a/community/how-to-contribute/pull-request.md b/community/how-to-contribute/pull-request.md
deleted file mode 100644
index f7cf3ac73ef..00000000000
--- a/community/how-to-contribute/pull-request.md
+++ /dev/null
@@ -1,259 +0,0 @@
----
-{
- "title": "Code Submission Guide",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Code Submission Guide
-
-[Pull Request (PR)](https://help.github.com/articles/about-pull-requests/) can be easily submitted on [Github](https://github.com/apache/doris). The PR method of Doris project is described below.
-
-## Fork Repository
-
-Go to the [github page](https://github.com/apache/doris) of apache/doris , and click the button `Fork` in the upper right corner for Fork.
-
-
-
-### 2. Configuring GIT and submitting modifications
-
-#### (1) Clone the code locally:
-
-```
-git clone https://github.com/<your_github_name>/doris.git
-```
-
-Note: Please replace your GitHub name with your yourgithubname.
-
-When clone is completed, origin defaults to the remote fork address on github.
-
-#### (2) Add apache/doris to the remote branch upstream of the local warehouse:
-
-```
-cd doris
-git remote add upstream https://github.com/apache/doris.git
-```
-
-#### (3) Check remote warehouse settings:
-
-```
-git remote -v
-origin https://github.com/<your_github_name>/doris.git (fetch)
-origin https://github.com/<your_github_name>/doris.git (push)
-upstream https://github.com/apache/doris.git (fetch)
-upstream https://github.com/apache/doris.git (push)
-```
-
-#### (4) New branches to modify them:
-
-```
-git checkout -b <your_branch_name>
-```
-
-Note: \<your\_branch\_name\> name is customized for you.
-
-Code changes can be made after creation.
-
-#### (5) Submit code to remote branch:
-
-```
-git commit -a -m "<you_commit_message>"
-git push origin <your_branch_name>
-```
-
-For more git usage, please visit: [git usage](https://www.atlassian.com/git/tutorials/set-up-a-repository), not to mention here.
-
-### 3. Create PR
-
-#### (1) New PR
-Switch to your GitHub page in the browser, switch to the submitted branch yourbranchname\\ and click the `Compare & pull request` button to create it, as shown in the following figure:
-
-
-
-#### (2) preparation branch
-At this time, the `Create pull request` button will appear. If not, please check whether the branch is selected correctly or click on `compare across forks' to re-select the repo and branch.
-
-
-
-#### (3) Fill Commit Message
-Here, please fill in the summary and details of the comment, and then click `Create pull request` to create it.
-
-For how to write Commit Message, here are some Tips:
-
-* Please use the form of English verb + object. The verb does not use the past tense and the sentence uses imperative sentence.
-* Subject and body should be written, and they should be separated by blank lines (fill in separately on GitHub PR interface).
-* Message topic length should not exceed **50** characters;
-* Message content should not exceed **72** characters per line, and the excess should be replaced manually.
-* Message content is used to explain what has been done, why and how.
-* The first letter of the message subject should be **capitalized**, and the end of the sentence **should not** have a full stop.
-* The message content specifies the associated issue (if any), such as # 233;
-
-For more details, see <https://chris.beams.io/posts/git-commit>.
-
-
-
-#### (4) Complete the creation
-After successful creation, you can see that Doris project needs review, you can wait for us to review and join, you can also contact us directly.
-
-
-
-So far, your PR creation is complete. Read more about PR [collaborating-with-issues-and-pull-requests] (https://help.github.com/categories/collaborating-with-issues-and-pull-requests/).
-
-### 4. Conflict Resolution
-
-When submitting PR, code conflicts are usually caused by multiple people editing the same file. The main steps to resolve conflicts are as follows:
-
-#### (1) Switch to the main branch
-
-```
-git checkout master
-```
-
-#### (2) Synchronize remote main branch to local
-
-```
-git pull upstream master
-```
-
-#### (3) Switch back to the previous branch (assuming the branch is named fix)
-
-```
-git checkout fix
-```
-
-#### (4) rebase
-
-```
-git rebase -i master
-```
-
-At this point, a file that modifies the record will pop up and can be saved directly. Then, we will prompt which files have conflicts. At this time, we can open the conflict file to modify the conflict part. After all the conflicts of the conflict files are resolved, we will execute them.
-
-```
-git add .
-git rebase --continue
-```
-
-Then you can go back and forth until the screen appears something like * rebase successful * and then you can update the branch that submitted PR:
-
-```
-git push -f origin fix
-```
-
-### 5. An example
-
-#### (1) fetch to the latest code for the local branch of upstream that has been configured
-
-```
-$ git branch
-* master
-
-$ git fetch upstream
-remote: Counting objects: 195, done.
-remote: Compressing objects: 100% (68/68), done.
-remote: Total 141 (delta 75), reused 108 (delta 48)
-Receiving objects: 100% (141/141), 58.28 KiB, done.
-Resolving deltas: 100% (75/75), completed with 43 local objects.
-From https://github.com/apache/doris
- 9c36200..0c4edc2 master -> upstream/master
-```
-
-#### (2) rebase
-
-```
-$ git rebase upstream/master
-First, rewinding head to replay your work on top of it...
-Fast-forwarded master to upstream/master.
-```
-
-#### (3) Check to see if other submissions are not synchronized to their own repo submissions
-
-```
-$ git status
-# On branch master
-# Your branch is ahead of 'origin/master' by 8 commits.
-#
-# Untracked files:
-# (use "git add <file>..." to include in what will be committed)
-#
-# custom_env.sh
-nothing added to commit but untracked files present (use "git add" to track)
-```
-
-#### (4) Merge code submitted by others into their own repo
-
-```
-$ git push origin master
-Counting objects: 195, done.
-Delta compression using up to 32 threads.
-Compressing objects: 100% (41/41), done.
-Writing objects: 100% (141/141), 56.66 KiB, done.
-Total 141 (delta 76), reused 140 (delta 75)
-remote: Resolving deltas: 100% (76/76), completed with 44 local objects.
-To https://lide-reed:xxxx@github.com/lide-reed/doris.git
- 9c36200..0c4edc2 master -> master
-```
-
-#### (5) New branch, ready for development
-
-```
-$ git checkout -b my_branch
-Switched to a new branch 'my_branch'
-
-$ git branch
- master
-* my_branch
-```
-
-#### (6) Prepare to submit after code modification is completed
-
-```
-$ git add -u
-```
-
-#### (7) Fill in the message and submit it it to the new local branch
-
-```
-$ git commit -m "Fix a typo"
-[my_branch 55e0ba2] Fix a typo
-1 files changed, 2 insertions(+), 2 deletions(-)
-```
-
-#### (8) Push the branch into GitHub's own repo far away
-
-```
-$ git push origin my_branch
-Counting objects: 11, done.
-Delta compression using up to 32 threads.
-Compressing objects: 100% (6/6), done.
-Writing objects: 100% (6/6), 534 bytes, done.
-Total 6 (delta 4), reused 0 (delta 0)
-remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
-remote:
-remote: Create a pull request for 'my_branch' on GitHub by visiting:
-remote: https://github.com/lide-reed/doris/pull/new/my_branch
-remote:
-To https://lide-reed:xxxx@github.com/lide-reed/doris.git
- * [new branch] my_branch -> my_branch
-```
-
-At this point, you can create PR according to the previous process.
diff --git a/community/release-and-verify/release-complete.md b/community/release-and-verify/release-complete.md
deleted file mode 100644
index 6582be182b7..00000000000
--- a/community/release-and-verify/release-complete.md
+++ /dev/null
@@ -1,183 +0,0 @@
----
-{
- "title": "Complete Release",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Complete release
-
-The steps in this document follow after the release has been voted on and approved in the dev@doris.
-
-## Upload package to release
-
-When the official release poll is successful, send the [Result] email first, then prepare the release package.
-Copy the source package, signature file and hash file from the corresponding folder of the previous release under dev to another directory 1.xx, note that the file name should not be rcxx (you can rename, but do not recalculate the signature, the hash can be recalculated, the result will not change)
-
-> Only PMC members have permission to operate this step.
-
-```
-From:
-https://dist.apache.org/repos/dist/dev/doris/
-
-To:
-https://dist.apache.org/repos/dist/release/doris/
-
-Eg:
-svn mv -m "move doris 1.1.0-rc05 to release" https://dist.apache.org/repos/dist/dev/doris/1.1 https://dist.apache.org/repos/dist/release/doris/1.1
-```
-
-For the first release, you need to copy the KEYS file as well. Then add it to the svn release.
-
-```
-add 成功后就可以在下面网址上看到你发布的文件
-https://dist.apache.org/repos/dist/release/doris/1.xx/
-
-稍等一段时间后，能在 apache 官网看到：
-http://www.apache.org/dist/doris/1.xx/
-```
-
-## Post links on Doris official website and github
-
-We will use Doris Core as an example. For other components, replace the name with the corresponding one.
-
-### Create a download link
-
-Download Link:
-
-```
-http://www.apache.org/dyn/closer.cgi?filename=doris/1.xx/apache-doris-1.xx-src.tar.gz&action=download
-
-wget --trust-server-names "https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download&filename=doris/1.xx/apache-doris-1.xx-src.tar.gz"
-```
-
-Original Location:
-
-```
-https://www.apache.org/dist/doris/1.xx/
-
-http://www.apache.org/dyn/closer.cgi/doris/1.xx/apache-doris-1.xx-src.tar.gz
-```
-
-Ssource package:
-
-```
-http://www.apache.org/dyn/closer.cgi/doris/1.xx/apache-doris-1.xx-src.tar.gz
-
-ASC:
-http://archive.apache.org/dist/doris/1.xx/apache-doris-1.xx-src.tar.gz.asc
-
-sha512:
-http://archive.apache.org/dist/doris/1.xx/apache-doris-1.xx-src.tar.gz.sha512
-```
-
-KEYS:
-
-```
-http://archive.apache.org/dist/doris/KEYS
-```
-
-refer to: <http://www.apache.org/dev/release-download-pages#closer>
-
-### Maven
-
-Find staging repo on [https://repository.apache.org/#stagingRepositories](https://repository.apache.org/#stagingRepositories) and click `Release` to release.
-
-### Prepare the release note
-
-The following two places need to be modified.
-
-1. Github's release page
-
-```
-https://github.com/apache/doris/releases/tag/0.9.0-rc01
-```
-
-2、Doris official website download page
-
-The download page is a markdown file with the following address.
-
-```
-docs/zh-cn/downloads/downloads.md
-docs/en/downloads/downloads.md
-```
-
-1. you need to change the download package address of the last release to the archive address of apache (see later).
-2. Add the download information for the new version.
-
-### Clean up old versions of packages on svn
-
-1. Deleting old packages on svn
-
-Since svn only needs to keep the latest version of packages, old versions of packages should be cleaned from svn when a new version is released.
-
-```
-https://dist.apache.org/repos/dist/release/doris/
-https://dist.apache.org/repos/dist/dev/doris/
-```
-
-Keep these two addresses with only the latest package versions. 2.
-
-2. Change the download address of the older packages on the official Doris website to the address of the archive page
-
-```
-Download page: http://doris.apache.org/downloads.html
-Archive page: http://archive.apache.org/dist/doris
-```
-
-Apache has a synchronization mechanism to archive the history of releases, see [how to archive](https://www.apache.org/legal/release-policy.html#how-to-archive)
-So even if an old package is removed from svn, it can still be found on the archive page.
-
-## Announce
-
-Title:
-
-```
-[ANNOUNCE] Apache Doris 1.xx release
-```
-
-To mail：
-
-```
-dev@doris.apache.org
-```
-
-Email body:
-
-```
-Hi All,
-
-We are pleased to announce the release of Apache Doris 1.xx.
-
-Apache Doris is an MPP-based interactive SQL data warehousing for reporting and analysis.
-
-The release is available at:
-http://doris.apache.org/master/zh-CN/downloads/downloads.html
-
-Thanks to everyone who has contributed to this release, and the release note can be found here:
-https://github.com/apache/doris/releases
-
-Best Regards,
-
-On behalf of the Doris team,
-xxx
-
diff --git a/community/release-and-verify/release-doris-connectors.md b/community/release-and-verify/release-doris-connectors.md
deleted file mode 100644
index d039fc3bbaf..00000000000
--- a/community/release-and-verify/release-doris-connectors.md
+++ /dev/null
@@ -1,231 +0,0 @@
----
-{
-"title": "Release Doris Connectors",
-"language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Releases Doris Connectors
-
-Doris Connectors currently contains:
-
-* Flink Doris Connector
-* Spark Doris Connector
-
-The code base is separate from the main Doris code base and is located at:
-
-- https://github.com/apache/doris-flink-connector
-- https://github.com/apache/doris-spark-connector
-
-## Preparing for release
-
-First, see the [release preparation](./release-prepare.md) documentation to prepare for the release.
-
-## Releasing to Maven
-
-Let's take the example of releasing Flink Connector v1.0.0.
-
-### 1. Prepare the branch
-
-Create a branch in the codebase: branch-1.0, and checkout to that branch.
-
-### 2. release to Maven staging
-
-Since Flink Connector releases different releases for different Flink versions (e.g. 1.11, 1.12, 1.13), we need to handle each version separately.
-
-Let's take Flink version 1.14.3 and scala version 2.12 as an example.
-
-First, replace flink.version and scala.version in pom.xml with
-
-```
-cd flink-doris-connector/
-sed -i 's/\${env.flink.version}/1.14.3/g' pom.xml
-sed -i 's/\${env.flink.minor.version}/1.14/g' pom.xml
-sed -i 's/\${env.scala.version}/2.12/g' pom.xml
-
-Mac:
-
-sed -i '' 's/\${env.flink.version}/1.14.3/g' pom.xml
-sed -i '' 's/\${env.flink.minor.version}/1.14/g' pom.xml
-sed -i '' 's/\${env.scala.version}/2.12/g' pom.xml
-```
-
-After replacing, commit the local changes to.
-
-```
-git add . -u
-git commit -m "prepare for 1.14_2.12-1.0.0"
-```
-
-Execute the following command to start generating the release tag.
-
-```bash
-cd flink-doris-connector/
-mvn release:clean
-mvn release:prepare -DpushChanges=false
-```
-
-where `-DpushChanges=false` means that the newly generated branches and tags are not pushed to the codebase during execution.
-
-After executing the `release:prepare` command, the following three pieces of information will be requested.
-
-1. the version of the Doris Flink Connector: which we can do by default, either by entering a carriage return or by typing in the version you want. The version format is `{connector.version}`, e.g. `1.0.0`.
-2. The release tag of Doris Flink Connector: the release process will generate a tag locally, we can use the default tag name, such as `1.14_2.12-1.0.0`.
-3. The version number of the next version of Doris Flink Connector: This version number is only used for generating local branches and has no real meaning. For example, if the current release is `1.0.0`, then the next version number should be `1.0.1-SNAPSHOT`.
-
-`mvn release:prepare` may ask for GPG passphrase, if you get `gpg: no valid OpenPGP data found` error, you can try after executing `export GPG_TTY=$(tty)`.
-
-If `mvn release:prepare` succeeds, a tag and a branch will be created locally, and two new commits will be added to the current branch, the first one corresponding to the newly created tag and the second one to the branch of the next release, which can be viewed via `git log`.
-
-Once the local tag is verified, you need to push the tag to the repository.
-
-`git push upstream --tags`
-
-where upstream points to the `apache/doris-flink-connector` repository.
-
-Finally, execute perform:
-
-```
-mvn release:perform
-```
-
-After successful execution, the version just released can be found in [https://repository.apache.org/#stagingRepositories](https://repository.apache.org/#stagingRepositories)
-
-
-
-**Note that the `.asc` signature file needs to be included.**
-
-If there is an error. You need to delete the local tag, the tag in the codebase, and the two newly generated local commits. And drop the staging. Then re-execute the above steps.
-
-After checking, click the `close` button in the figure to finish staging release.
-
-### 3. Prepare svn
-
-Check out the svn repository.
-
-```
-svn co https://dist.apache.org/repos/dist/dev/doris/
-```
-
-Package the tag source code and generate the signature file and sha256 checksum file. Here we take `1.14_2.12-1.0.0` as an example.
-
-```
-git archive --format=tar 1.14_2.12-1.0.0 --prefix=apache-doris-flink-connector-1.14_2.12-1.0.0-src/ | gzip > apache-doris-flink-connector-1.14_2.12-1.0.0-src.tar.gz
-gpg -u xxx@apache.org --armor --output apache-doris-flink-connector-1.14_2.12-1.0.0-src.tar.gz.asc --detach-sign apache-doris-flink-connector-1.14_2.12-1.0.0-src.tar.gz
-sha512sum apache-doris-flink-connector-1.14_2.12-1.0.0-src.tar.gz > apache-doris-flink-connector-1.14_2.12-1.0.0-src.tar.gz.sha512
-
-Mac:
-shasum -a 512 apache-doris-flink-connector-1.14_2.12-1.0.0-src.tar.gz > apache-doris-flink-connector-1.14_2.12-1.0.0-src.tar.gz.sha512
-```
-
-The end result is three files:
-
-```
-apache-doris-flink-connector-1.14_2.12-1.0.0-src.tar.gz
-apache-doris-flink-connector-1.14_2.12-1.0.0-src.tar.gz.asc
-apache-doris-flink-connector-1.14_2.12-1.0.0-src.tar.gz.sha512
-```
-
-Move these three files to the svn directory:
-
-```
-doris/flink-connector/1.0.0/
-```
-
-The final svn directory structure will look like this:
-
-```
-|____0.15
-| |____0.15.0-rc04
-| | |____apache-doris-0.15.0-incubating-src.tar.gz.sha512
-| | |____apache-doris-0.15.0-incubating-src.tar.gz.asc
-| | |____apache-doris-0.15.0-incubating-src.tar.gz
-|____KEYS
-|____flink-connector
-| |____1.0.0
-| | |____apache-doris-flink-connector-1.14_2.12-1.0.0-incubating-src.tar.gz
-| | |____apache-doris-flink-connector-1.14_2.12-1.0.0-incubating-src.tar.gz.sha512
-| | |____apache-doris-flink-connector-1.14_2.12-1.0.0-incubating-src.tar.gz.asc
-```
-
-Where 0.15 is the directory of Doris main code, and under `flink-connector/1.0.0` is the content of this release.
-
-Note that the preparation of the KEYS file can be found in [release preparation](. /release-prepare.md).
-
-### 4. Polling
-
-Initiate a poll in the dev@doris mailgroup, with the following template.
-
-```
-Hi All,
-
-This is a call for vote to release Flink Connectors v1.0.0 for Apache Doris
-
-- apache-doris-flink-connector-1.14_2.12-1.0.0
-
-The release node:
-xxxxx
-
-The release candidates:
-https://dist.apache.org/repos/dist/dev/doris/flink-connector/1.0.0/
-
-Maven 2 staging repository:
-https://repository.apache.org/content/repositories/orgapachedoris-1002/org/apache/doris/doris-flink-connector/
-
-Git tag for the release:
-https://github.com/apache/doris-flink-connector/tree/1.14_2.12-1.0.0
-
-Keys to verify the Release Candidate:
-https://downloads.apache.org/doris/KEYS
-
-Look at here for how to verify this release candidate:
-hhttps://doris.apache.org/zh-CN/community/release-and-verify/release-verify
-
-The vote will be open for at least 72 hours or until necessary number of votes are reached.
-
-Please vote accordingly:
-
-[ ] +1 approve
-[ ] +0 no opinion
-[ ] -1 disapprove with the reason
-```
-
-## Completing the release
-
-Please refer to the [Release Completion](./release-complete.md) document to complete the release process.
-
-## Appendix: Releasing to SNAPSHOT
-
-Snapshot is not an Apache Release version and is only used for pre-release previews. Snapshot versions can be released after discussion and approval by the PMC
-
-Switch to the flink connector directory, we will use flink version 1.13.5, scalar 2.12 as an example
-
-```
-cd flink-doris-connector
-mvn deploy -Dflink.version=1.13.5 -Dscala.version=2.12
-```
-
-After that you can see the snapshot version here.
-
-```
-https://repository.apache.org/content/repositories/snapshots/org/apache/doris/doris-flink-connector/
-```
diff --git a/community/release-and-verify/release-doris-core.md b/community/release-and-verify/release-doris-core.md
deleted file mode 100644
index 2f53f7a9a7c..00000000000
--- a/community/release-and-verify/release-doris-core.md
+++ /dev/null
@@ -1,324 +0,0 @@
----
-{
- "title": "Release Doris Core",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Release Doris Core
-
-Doris Core refers to the content published in https://github.com/apache/doris.
-
-## Preparing for release
-
-First, see the [release preparation](./release-prepare.md) documentation for release preparation.
-
-### Preparing a branch
-
-You need to create a new branch before releasing. For example.
-
-```
-$ git checkout -b branch-0.9
-```
-
-This branch should be more fully tested to make features available, bugs converged, and important bugs fixed.
-This process requires waiting for the community to see if there are any necessary patches that need to be merged in for this release, and if so, cherry-picking it to the release branch.
-
-### Clean up issues
-
-Go through all the issues that belong to this release, close the ones that are done, and if they can't be done, defer them to a later release.
-
-### Merge necessary patches
-
-If someone in the community says there are important bugs that need to be merged in, then the Release Manager needs to evaluate and merge the important patches into the release branch.
-
-## Validation branch
-
-### Stability testing
-
-Pass the batched branch to the QA students for stability testing. If during the testing process, there are issues that need to be fixed, then if during the testing process, there are issues that need to be fixed, then after they are fixed, the PRs that fix the issues need to be merged into the branch of the pending release.
-
-Only after the whole branch is stable, can you prepare to release the version.
-
-### Compile verification
-
-Please refer to the compilation documentation for compilation to ensure that the source code is compiled correctly.
-
-### Prepare Release Notes
-
-## Community Posting Voting Process
-
-### Tagging
-
-Once the above branch is more stable, you can tag this branch.
-Remember to modify the `build_version` variable in `gensrc/script/gen_build_version.sh` when creating the tag. For example `build_version="0.10.0-release"`
-
-Example:
-
-```
-$ git checkout branch-0.9
-$ git tag -a 0.9.0-rc01 -m "0.9.0 release candidate 01"
-$ git push origin 0.9.0-rc01
-Counting objects: 1, done.
-Writing objects: 100% (1/1), 165 bytes | 0 bytes/s, done.
-Total 1 (delta 0), reused 0 (delta 0)
-To git@github.com:apache/incubator-doris.git
- * [new tag] 0.9.0-rc01 -> 0.9.0-rc01
-
-$ git tag
-```
-
-### Package, sign and upload
-
-For the following steps, you also need to log in to the user account directly through a terminal such as SecureCRT, not through `su - user` or `ssh`, otherwise the password input box will not be displayed and an error will be reported.
-
-```
-$ git checkout 0.9.0-rc01
-
-$ git archive --format=tar 0.9.0-rc01 --prefix=apache-doris-0.9.0-incubating-src/ | gzip > apache-doris-0.9.0-incubating-src.tar.gz
-
-$ gpg -u xxx@apache.org --armor --output apache-doris-0.9.0-incubating-src.tar.gz.asc --detach-sign apache-doris-0.9.0-incubating-src.tar.gz
-
-$ gpg --verify apache-doris-0.9.0-incubating-src.tar.gz.asc apache-doris-0.9.0-incubating-src.tar.gz
-
-$ sha512sum apache-doris-0.9.0-incubating-src.tar.gz > apache-doris-0.9.0-incubating-src.tar.gz.sha512
-
-$ sha512sum --check apache-doris-0.9.0-incubating-src.tar.gz.sha512
-```
-
-Then upload the packaged content to the svn repository by first downloading the svn library at:
-
-```
-svn co https://dist.apache.org/repos/dist/dev/incubator/doris/
-```
-
-Organize all the previously obtained files into the following svn path:
-
-```
-./doris/
-|-- 0.11.0-rc1
-| |-- apache-doris-0.11.0-incubating-src.tar.gz
-| |-- apache-doris-0.11.0-incubating-src.tar.gz.asc
-| `-- apache-doris-0.11.0-incubating-src.tar.gz.sha512
-`-- KEYS
-```
-
-Upload these file:
-
-```
-svn add 0.11.0-rc1
-svn commit -m "Add 0.11.0-rc1"
-```
-
-### Email the community at dev@doris.apache.org to vote
-
-[VOTE] Release Apache Doris 0.9.0-incubating-rc01
-
-```
-Hi all,
-
-Please review and vote on Apache Doris 0.9.0-incubating-rc01 release.
-
-The release candidate has been tagged in GitHub as 0.9.0-rc01, available
-here:
-https://github.com/apache/incubator-doris/releases/tag/0.9.0-rc01
-
-Release Notes are here:
-https://github.com/apache/incubator-doris/issues/1891
-
-Thanks to everyone who has contributed to this release.
-
-The artifacts (source, signature and checksum) corresponding to this release
-candidate can be found here:
-https://dist.apache.org/repos/dist/dev/incubator/doris/0.9/0.9.0-rc1/
-
-This has been signed with PGP key 33DBF2E0, corresponding to
-lide@apache.org.
-KEYS file is available here:
-https://downloads.apache.org/incubator/doris/KEYS
-It is also listed here:
-https://people.apache.org/keys/committer/lide.asc
-
-To verify and build, you can refer to following wiki:
-https://github.com/apache/incubator-doris/wiki/How-to-verify-Apache-Release
-https://wiki.apache.org/incubator/IncubatorReleaseChecklist
-
-The vote will be open for at least 72 hours.
-[ ] +1 Approve the release
-[ ] +0 No opinion
-[ ] -1 Do not release this package because ...
-
-Best Regards,
-xxx
-
-----
-DISCLAIMER:
-Apache Doris (incubating) is an effort undergoing incubation at The
-Apache Software Foundation (ASF), sponsored by the Apache Incubator PMC.
-
-Incubation is required of all newly accepted
-projects until a further review indicates that the
-infrastructure, communications, and decision making process have
-stabilized in a manner consistent with other successful ASF
-projects.
-
-While incubation status is not necessarily a reflection
-of the completeness or stability of the code, it does indicate
-that the project has yet to be fully endorsed by the ASF.
-```
-
-### After the vote is approved, send the Result email
-
-[Result][VOTE] Release Apache Doris 0.9.0-incubating-rc01
-
-```
-Thanks to everyone, and this vote is now closed.
-
-It has passed with 4 +1 (binding) votes and no 0 or -1 votes.
-
-Binding:
-+1 Zhao Chun
-+1 xxx
-+1 Li Chaoyong
-+1 Mingyu Chen
-
-Best Regards,
-xxx
-
-```
-
-### Email general@incubator.apache.org to vote
-
-**If not an incubator program, please skip**
-
-[VOTE] Release Apache Doris 0.9.0-incubating-rc01
-
-```
-Hi all,
-
-Please review and vote on Apache Doris 0.9.0-incubating-rc01 release.
-
-Apache Doris is an MPP-based interactive SQL data warehousing for reporting and analysis.
-
-The Apache Doris community has voted on and approved this release:
-https://lists.apache.org/thread.html/d70f7c8a8ae448bf6680a15914646005c6483564464cfa15f4ddc2fc@%3Cdev.doris.apache.org%3E
-
-The vote result email thread:
-https://lists.apache.org/thread.html/64d229f0ba15d66adc83306bc8d7b7ccd5910ecb7e842718ce6a61da@%3Cdev.doris.apache.org%3E
-
-The release candidate has been tagged in GitHub as 0.9.0-rc01, available here:
-https://github.com/apache/incubator-doris/releases/tag/0.9.0-rc01
-
-There is no CHANGE LOG file because this is the first release of Apache Doris.
-Thanks to everyone who has contributed to this release, and there is a simple release notes can be found here:
-https://github.com/apache/incubator-doris/issues/406
-
-The artifacts (source, signature and checksum) corresponding to this release candidate can be found here:
-https://dist.apache.org/repos/dist/dev/incubator/doris/0.9/0.9.0-rc01/
-
-This has been signed with PGP key 33DBF2E0, corresponding to lide@apache.org.
-KEYS file is available here:
-https://downloads.apache.org/incubator/doris/KEYS
-It is also listed here:
-https://people.apache.org/keys/committer/lide.asc
-
-The vote will be open for at least 72 hours.
-[ ] +1 Approve the release
-[ ] +0 No opinion
-[ ] -1 Do not release this package because ...
-
-To verify and build, you can refer to following instruction:
-
-Firstly, you must be install and start docker service, and then you could build Doris as following steps:
-
-Step1: Pull the docker image with Doris building environment
-$ docker pull apache/incubator-doris:build-env-1.3.1
-You can check it by listing images, its size is about 3.28GB.
-
-Step2: Run the Docker image
-You can run image directly:
-$ docker run -it apache/incubator-doris:build-env-1.3.1
-
-Step3: Download Doris source
-Now you should in docker environment, and you can download Doris source package.
-(If you have downloaded source and it is not in image, you can map its path to image in Step2.)
-$ wget https://dist.apache.org/repos/dist/dev/incubator/doris/0.9/0.9.0-rc01/apache-doris-0.9.0.rc01-incubating-src.tar.gz
-
-Step4: Build Doris
-Now you can decompress and enter Doris source path and build Doris.
-$ tar zxvf apache-doris-0.9.0.rc01-incubating-src.tar.gz
-$ cd apache-doris-0.9.0.rc01-incubating-src
-$ sh build.sh
-
-Best Regards,
-xxx
-
-----
-DISCLAIMER:
-Apache Doris (incubating) is an effort undergoing incubation at The
-Apache Software Foundation (ASF), sponsored by the Apache Incubator PMC.
-
-Incubation is required of all newly accepted
-projects until a further review indicates that the
-infrastructure, communications, and decision making process have
-stabilized in a manner consistent with other successful ASF
-projects.
-
-While incubation status is not necessarily a reflection
-of the completeness or stability of the code, it does indicate
-that the project has yet to be fully endorsed by the ASF.
-```
-
-The thread link for the email can be found here.
-
-`https://lists.apache.org/list.html?dev@doris.apache.org`
-
-### Send Result email to general@incubator.apache.org
-
-**If not an incubator project, please skip**
-
-[RESULT][VOTE] Release Apache Doris 0.9.0-incubating-rc01
-
-
-```
-Hi,
-
-Thanks to everyone, and the vote for releasing Apache Doris 0.9.0-incubating-rc01 is now closed.
-
-It has passed with 4 +1 (binding) votes and no 0 or -1 votes.
-
-Binding:
-+1 Willem Jiang
-+1 Justin Mclean
-+1 ShaoFeng Shi
-+1 Makoto Yui
-
-The vote thread:
-https://lists.apache.org/thread.html/da05fdd8d84e35de527f27200b5690d7811a1e97d419d1ea66562130@%3Cgeneral.incubator.apache.org%3E
-
-Best Regards,
-xxx
-```
-
-## Completing the release
-
-Please refer to the [Release Completion](./release-complete.md) document to complete the release process.
\ No newline at end of file
diff --git a/community/release-and-verify/release-doris-manager.md b/community/release-and-verify/release-doris-manager.md
deleted file mode 100644
index 3ae04524508..00000000000
--- a/community/release-and-verify/release-doris-manager.md
+++ /dev/null
@@ -1,302 +0,0 @@
----
-{
-"title": "Release Doris Manager",
-"language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Publish Doris Manager
-
-Its codebase is separate from the main Doris codebase at:
-
-- https://github.com/apache/doris-manager
-
-## ready to publish
-
-First, refer to the [release preparation](./release-prepare.md) documentation for release preparation.
-
-## prepare branch
-
-Before publishing, you need to create a new branch. E.g:
-
-````text
-$ git checkout -b branch-1.0.0
-````
-
-This branch needs to be fully tested, so that functions are available, bugs are closed, and important bugs are fixed. This process requires waiting for the community to see if the necessary patch needs to be merged in this release, and if so, cherry-pick it to the release branch.
-
-## clean up issues
-
-Go through all issues that belong to this version, close the ones that have been completed, and postpone them to a later version if they cannot be completed
-
-## Merge necessary patches
-
-During the release waiting process, there may be more important patches merged. If someone in the community says that there are important bugs that need to be merged, then the Release Manager needs to evaluate and merge important patches into the release branch
-
-## Verify branch
-
-### Stability Test
-
-Give the prepared branch to QA students for stability testing. If there is a problem that needs to be fixed during the testing process, if there is a problem that needs to be fixed during the testing process, after the repair is completed, the PR that fixes the problem needs to be merged into the branch of the to-be-released version.
-
-After the entire branch is stable, the release version can be prepared.
-
-### Compile and verify
-
-Please refer to the compilation documentation for compilation to ensure that the source code is compiled correctly.
-
-## Community release voting process
-
-### tag
-
-When the above branch is relatively stable, you can tag this branch.
-
-E.g:
-
-````text
-$ git checkout branch-1.0.0
-$ git tag -a 1.0.0-rc01 -m "doris manager 1.0.0 release candidate 01"
-$ git push origin 1.0.0-rc01
-Counting objects: 1, done.
-Writing objects: 100% (1/1), 165 bytes | 0 bytes/s, done.
-Total 1 (delta 0), reused 0 (delta 0)
-To git@github.com:apache/incubator-doris-manager.git
- * [new tag] 1.0.0-rc01 -> 1.0.0-rc01
-
-$ git tag
-````
-
-### Package, sign and upload
-
-In the following steps, you also need to log in to the user account directly through a terminal such as SecureCRT. You cannot switch it through su - user or ssh, otherwise the password input box will not be displayed and an error will be reported.
-
-````
-git archive --format=tar 1.0.0-rc01 --prefix=apache-doris-incubating-manager-src-1.0.0-rc01/ | gzip > apache-doris-incubating-manager-src-1.0.0-rc01 .tar.gz
-
-gpg -u xxx@apache.org --armor --output apache-doris-incubating-manager-src-1.0.0-rc01.tar.gz.asc --detach-sign apache-doris-incubating-manager-src- 1.0.0-rc01.tar.gz
-
-gpg --verify apache-doris-incubating-manager-src-1.0.0-rc01.tar.gz.asc apache-doris-incubating-manager-src-1.0.0-rc01.tar.gz
-
-sha512sum apache-doris-incubating-manager-src-1.0.0-rc01.tar.gz > apache-doris-incubating-manager-src-1.0.0-rc01.tar.gz.sha512
-
-sha512sum --check apache-doris-incubating-manager-src-1.0.0-rc01.tar.gz.sha512
-````
-
-Then upload the packaged content to the svn repository, first download the svn library:
-
-````
-svn co https://dist.apache.org/repos/dist/dev/incubator/doris/
-````
-
-Organize all the files obtained before into the following svn paths
-
-````
-./doris/
-├── doris-manager
-│ └── 1.0.0
-│ ├── apache-doris-incubating-manager-src-1.0.0-rc01.tar.gz
-│ ├── apache-doris-incubating-manager-src-1.0.0-rc01.tar.gz.asc
-│ └── apache-doris-incubating-manager-src-1.0.0-rc01.tar.gz.sha512
-````
-
-upload these files
-
-````text
-svn add 1.0.0-rc01
-svn commit -m "Add doris manager 1.0.0-rc01"
-````
-
-### Email the community dev@doris.apache.org to vote
-
-[VOTE] Release Apache Doris Manager 1.0.0-incubating-rc01
-
-````
-Hi All,
-
-This is a call for vote to release Doris Manager v1.0.0 for Apache Doris(Incubating).
-
-- apache-doris-incubating-manager-src-1.0.0-rc01
-
-The release node:
-
-
-
-The release candidates:
-https://dist.apache.org/repos/dist/dev/incubator/doris/doris-manager/1.0.0/
-
-Keys to verify the Release Candidate:
-https://downloads.apache.org/incubator/doris/KEYS
-
-Look at here for how to verify this release candidate:
-http://doris.incubator.apache.org/community/release-and-verify/release-verify.html
-
-Vote thread at dev@doris: [1]
-
-The vote will be open for at least 72 hours or until necessary number of votes are reached.
-
-Please vote accordingly:
-
-[ ] +1 approve
-[ ] +0 no opinion
-[ ] -1 disapprove with the reason
-
-[1] vote thread in dev@doris
-
-
-Brs,
-xxxx
-------------------
-DISCLAIMER:
-Apache Doris (incubating) is an effort undergoing incubation at The
-Apache Software Foundation (ASF), sponsored by the Apache Incubator PMC.
-
-Incubation is required of all newly accepted
-projects until a further review indicates that the
-infrastructure, communications, and decision making process have
-stabilized in a manner consistent with other successful ASF
-projects.
-
-While incubation status is not necessarily a reflection
-of the completeness or stability of the code, it does indicate
-that the project has yet to be fully endorsed by the ASF.
-
-````
-
-### After voting, send Result email
-
-[Result][VOTE] Release Apache Doris Manager 1.0.0-incubating-rc01
-
-````text
-Thanks to everyone, and this vote is now closed.
-
-It has passed with 4 +1 (binding) votes and no 0 or -1 votes.
-
-Binding:
-+1 jiafeng Zhang
-+1 xxx
-+1 EmmyMiao87
-+1 Mingyu Chen
-
-Best Regards,
-xxx
-````
-
-After passing the dev mailing group, send an email to the general@incubator mailing group for IPMC voting.
-
-````text
-Hi all,
-
-Please review and vote on Apache Doris Manager 1.0.0-incubating-rc01 release.
-
-Doris manager is a platform for automatic installation, deployment and management of Doris groups
-
-The Apache Doris community has voted on and approved this release:
-https://lists.apache.org/thread.html/d70f7c8a8ae448bf6680a15914646005c6483564464cfa15f4ddc2fc@%3Cdev.doris.apache.org%3E
-
-The vote result email thread:
-https://lists.apache.org/thread.html/64d229f0ba15d66adc83306bc8d7b7ccd5910ecb7e842718ce6a61da@%3Cdev.doris.apache.org%3E
-
-The release candidate has been tagged in GitHub as 1.0.0-rc01, available here:
-https://github.com/apache/incubator-doris-manager/releases/tag/1.0.0-rc01
-
-There is no CHANGE LOG file because this is the first release of Apache Doris.
-Thanks to everyone who has contributed to this release, and there is a simple release notes can be found here:
-https://github.com/apache/incubator-doris/issues/406
-
-The artifacts (source, signature and checksum) corresponding to this release candidate can be found here:
-https://dist.apache.org/repos/dist/dev/incubator/doris/doris-manager/1.0.0/
-
-This has been signed with PGP key 33DBF2E0, corresponding to lide@apache.org.
-KEYS file is available here:
-https://downloads.apache.org/incubator/doris/KEYS
-It is also listed here:
-https://people.apache.org/keys/committer/lide.asc
-
-The vote will be open for at least 72 hours.
-[ ] +1 Approve the release
-[ ] +0 No opinion
-[ ] -1 Do not release this package because ...
-
-To verify and build, you can refer to the following instruction:
-
-Firstly, you must be install and start docker service, and then you could build Doris as following steps:
-
-$ wget https://dist.apache.org/repos/dist/dev/incubator/doris/doris-manager/1.0.0/apache-doris-incubating-manager-src-1.0.0-rc01.tar.gz
-
-Step4: Build Doris
-Now you can decompress and enter Doris source path and build Doris.
-$ tar zxvf apache-doris-incubating-manager-src-1.0.0-rc01.tar.gz
-$ cd apache-doris-incubating-manager-src-1.0.0-rc01
-$ sh build.sh
-
-Best Regards,
-xxx
-
-----
-DISCLAIMER:
-Apache Doris (incubating) is an effort undergoing incubation at The
-Apache Software Foundation (ASF), sponsored by the Apache Incubator PMC.
-
-Incubation is required of all newly accepted
-projects until a further review indicates that the
-infrastructure, communications, and decision making process have
-stabilized in a manner consistent with other successful ASF
-projects.
-
-While incubation status is not necessarily a reflection
-of the completeness or stability of the code, it does indicate
-that the project has yet to be fully endorsed by the ASF.
-````
-
-The thread link for the mail can be found here:
-
-````
-https://lists.apache.org/list.html?dev@doris.apache.org
-````
-
-### Send Result email to general@incubator.apache.org
-
-[RESULT][VOTE] Release Apache Doris Manager 1.0.0-incubating-rc01
-
-````text
-Hi,
-
-Thanks to everyone, and the vote for releasing Apache Doris Manager 1.0.0-incubating-rc01 is now closed.
-
-It has passed with 4 +1 (binding) votes and no 0 or -1 votes.
-
-Binding:
-+1 Willem Jiang
-+1 Justin Mclean
-+1 ShaoFeng Shi
-+1 Makoto Yui
-
-The vote thread:
-https://lists.apache.org/thread.html/da05fdd8d84e35de527f27200b5690d7811a1e97d419d1ea66562130@%3Cgeneral.incubator.apache.org%3E
-
-Best Regards,
-xxx
-````
-
-## Finish publishing
-
-Please refer to the [Completing the Release](https://doris.apache.org/en-US/community/release-and-verify/release-complete.html) documentation to complete all release processes.
diff --git a/community/release-and-verify/release-prepare.md b/community/release-and-verify/release-prepare.md
deleted file mode 100644
index 51c84657790..00000000000
--- a/community/release-and-verify/release-prepare.md
+++ /dev/null
@@ -1,346 +0,0 @@
----
-{
-"title": "Release Preparation",
-"language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Release Preparation
-
-Releases of Apache projects must strictly follow the Apache Foundation release process. Related guidance and policies can be found at.
-
-* [Release Creation Process](https://infra.apache.org/release-publishing)
-* [Release Policy](https://www.apache.org/legal/release-policy.html)
-* [Publishing Maven Releases to Maven Central Repository](https://infra.apache.org/publishing-maven-artifacts.html)
-
-This document describes the main process and prep work for release. For specific Doris component release steps, you can refer to the respective documentation:
-
-* [Doris Core Release](./release-doris-core.md)
-* [Doris Connectors Release](./release-doris-connectors.md)
-* [Doris Manager Release](./release-doris-manager.md)
-
-There are three main forms of releases for Apache projects.
-
-* Source Release: i.e. source release, this is mandatory.
-* Binary Release: e.g., release of a compiled executable. This is optional.
-* Convenience Binaries: Release to third-party platforms for user convenience, such as Maven, Docker, etc. This is also optional.
-
-## Release Process
-
-Each project release requires a PMC member or Committer as the **Release Manager**.
-
-The overall release process is as follows.
-
-1. Environment preparation
-2. Release preparation
- 1. the community initiates DISCUSS and communicates with the community about the specific release plan
- 2. create a branch for the release
- 3. clean up the issue
- 4. merge the necessary patches into the released branch
-3. verify the branch
- 1. stability testing
- 2. verify the compilation flow of the branch code
- 3. Prepare Release Notes
-4. prepare release materials
- 1. Tagging
- 2. upload the content to be released to the [Apache Dev SVN repository](https://dist.apache.org/repos/dist/dev/doris)
- 3. preparation of other Convenience Binaries (e.g. upload to [Maven Staging repository](https://repository.apache.org/#stagingRepositories))
-4. Community Release Polling Process
- 2. Initiate a VOTE in the [Doris Community Dev Mail Group](dev@doris.apache.org).
- 3. After the vote is approved, send a Result email in the Doris community.
-5. Complete the work
- 1. Upload the signed packages to the [Apache Release repository](https://dist.apache.org/repos/dist/release/doris) and generate the relevant links.
- 2. Post the download links on the Doris website and github, and clean up the old packages on svn.
- 3. Send an Announce email to dev@doris.apache.org
-
-## Prepare signatures
-
-If this is your first time as Release Manager, then you need to prepare the following tools in your environment
-
-1. [Release Signing](https://www.apache.org/dev/release-signing.html)
-2. [gpg](https://www.apache.org/dev/openpgp.html)
-3. [svn](https://www.apache.org/dev/openpgp.html)
-
-### Prepare gpg key
-
-Release manager needs to create its own signature public key before release, and upload it to the public key server, then you can use this public key to sign the package to be released.
-If your KEY already exists in [KEYS](https://downloads.apache.org/doris/KEYS), then you can skip this step.
-
-#### Installation and configuration of the signature software GnuPG
-
-##### GnuPG
-
-In 1991, programmer Phil Zimmermann developed the encryption software PGP in order to avoid government surveillance; it worked so well that it quickly spread and became an essential tool for many programmers. However, it was commercial software and could not be used freely. So, the Free Software Foundation decided to develop a replacement for PGP, named GnuPG, and that's how GPG came to be.
-
-##### installation configuration
-
-CentOS installation command.
-
-```
-yum install gnupg
-```
-
-After installation, the default configuration file gpg.conf will be placed in the home directory.
-
-```
-~/.gnupg/gpg.conf
-```
-
-If this directory or file does not exist, you can just create an empty file.
-
-Apache recommends SHA512 for signatures, which can be done by configuring gpg.
-Edit gpg.conf, adding the following three lines.
-
-```
-personal-digest-preferences SHA512
-cert-digest-algo SHA512
-default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 ZLIB BZIP2 ZIP Uncompressed
-```
-
-#### Generate a new signature
-
-##### Preparing a signature
-
-Recommended settings for generating new signatures.
-
-Here you must log in to the user account directly through a terminal such as SecureCRT, not through su - user or ssh, otherwise the password input box will not show up and an error will be reported.
-
-First look at the version of gpg and whether it supports SHA512.
-
-```
-$ gpg --version
-gpg (GnuPG) 2.0.22
-libgcrypt 1.5.3
-Copyright (C) 2013 Free Software Foundation, Inc.
-License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
-This is free software: you are free to change and redistribute it.
-There is NO WARRANTY, to the extent permitted by law.
-
-Home: ~/.gnupg
-Supported algorithms:
-Pubkey: RSA, ?, ?, ELG, DSA
-Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
- CAMELLIA128, CAMELLIA192, CAMELLIA256
-Hash: MD5, SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
-Compression: Uncompressed, ZIP, ZLIB, BZIP2
-```
-
-##### Generate a new signature
-
-```
-$ gpg --gen-key
-gpg (GnuPG) 2.0.22; Copyright (C) 2013 Free Software Foundation, Inc.
-This is free software: you are free to change and redistribute it.
-There is NO WARRANTY, to the extent permitted by law.
-
-Please select what kind of key you want:
- (1) RSA and RSA (default)
- (2) DSA and Elgamal
- (3) DSA (sign only)
- (4) RSA (sign only)
-Your selection? 1
-RSA keys may be between 1024 and 4096 bits long.
-What keysize do you want? (2048) 4096
-Requested keysize is 4096 bits
-Please specify how long the key should be valid.
- 0 = key does not expire
- <n> = key expires in n days
- <n>w = key expires in n weeks
- <n>m = key expires in n months
- <n>y = key expires in n years
-Key is valid for? (0)
-Key does not expire at all
-Is this correct? (y/N) y
-
-GnuPG needs to construct a user ID to identify your key.
-
-Real name: xxx
-Name must be at least 5 characters long
-Real name: xxx-yyy
-Email address: xxx@apache.org
-Comment: xxx's key
-You selected this USER-ID:
- "xxx-yyy (xxx's key) <xx...@apache.org>"
-
-Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
-```
-
-Real name should be the same as the id shown in id.apache.org.
-Email address is the apache email address.
-
-Enter passphrase, twice, more than 8 characters.
-
-**The secret key here must be remembered, it will be used later when signing. It will also be used for publishing other components**
-
-**If the `gpg --gen-key` command gets stuck for a long time, try opening another terminal and execute the `find / | xargs file` command to generate enough random characters, usually after a few minutes, the gpg command will complete. **
-
->**Notice:**
->
->If the generation is stuck when it can be generated and cannot be completed for a long time, the following solutions can be used to solve it:
->
->Install the rng-tools tool by `yum install rng-tools` to complete the installation.
->Then open a new window and execute the command: rngd -r /dev/urandom, and the key generation can be completed instantly.
-
-##### View and output
-
-The first line shows the public key file name (pubring.gpg), the second line shows the public key characteristics (4096 bits, hash string and generation time), the third line shows the "user ID", comments, emails, and the fourth line shows the private key characteristics.
-
-```
-$ gpg --list-keys
-/home/lide/.gnupg/pubring.gpg
------------------------------
-pub 4096R/33DBF2E0 2018-12-06
-uid xxx-yyyy (xxx's key) <xx...@apache.org>
-sub 4096R/0E8182E6 2018-12-06
-```
-
-where xxx-yyyy is the user ID.
-
-```
-gpg --armor --output public-key.txt --export [user-id]
-```
-
-```
-$ gpg --armor --output public-key.txt --export xxx-yyyy
-The file 'public-key.txt' already exists. Is it overwritten? (y/N)y
-$ cat public-key.txt
------BEGIN PGP PUBLIC KEY BLOCK-----
-Version: GnuPG v2.0.22 (GNU/Linux)
-
-mQINBFwJEQ0BEACwqLluHfjBqD/RWZ4uoYxNYHlIzZvbvxAlwS2mn53BirLIU/G3
-9opMWNplvmK+3+gNlRlFpiZ7EvHsF/YJOAP59HmI2Z...
-```
-
-#### Uploading Signed Public Keys
-
-A public key server is a server on the network dedicated to storing the user's public key. send-keys parameter can upload the public key to the server.
-
-```
-gpg --send-keys xxxx --keyserver https://keyserver.ubuntu.com/
-
-```
-where xxxx is the string after pub in the `-list-keys` result of the previous step, e.g., 33DBF2E0
-
-You can also upload the contents of the above public-key.txt via [https://keyserver.ubuntu.com/](https://keyserver.ubuntu.com/).
-
-After successful upload, you can query this [https://keyserver.ubuntu.com/](https://keyserver.ubuntu.com/) by entering 0x33DBF2E0. (Note that it needs to start with 0x)
-
-There is a delay in querying this website, you may need to wait for 1 hour.
-
-#### generates fingerprint and uploads it to apache user information
-
-Since the public key server has no checking mechanism, anyone can upload a public key in your name, so there is no way to guarantee the reliability of the public key on the server. Usually, you can publish a public key fingerprint on your website and let other people check whether the downloaded public key is genuine or not.
-
-The fingerprint parameter generates a public key fingerprint.
-
-```
-gpg --fingerprint [user-id]
-```
-
-```
-$ gpg --fingerprint xxx-yyyy
-pub 4096R/33DBF2E0 2018-12-06
- Key fingerprint = 07AA E690 B01D 1A4B 469B 0BEF 5E29 CE39 33DB F2E0
-uid xxx-yyyy (xxx's key) <xx...@apache.org>
-sub 4096R/0E8182E6 2018-12-06
-```
-
-Paste the fingerprint above (i.e. 07AA E690 B01D 1A4B 469B 0BEF 5E29 CE39 33DB F2E0) into your own user information at
-
-https://id.apache.org
-
-`OpenPGP Public Key Primary Fingerprint:`
-
-> Note: Each person can have more than one Public Key.
-
-#### generates keys
-
-**Be careful not to delete existing content in the KEYS file, it can only be added.**
-
-```
-svn co https://dist.apache.org/repos/dist/dev/doris/
-# edit doris/KEYS file
-gpg --list-sigs [user-id] >> doris/KEYS
-gpg --armor --export [user ID] >> doris/KEYS
-svn ci --username $ASF_USERNAME --password "$ASF_PASSWORD" -m "UPDATE KEYS"
-```
-
-Note that the KEYS file should also be published to the following svn library.
-
-```
-svn co https://dist.apache.org/repos/dist/release/doris
-# edit doris/KEYS file
-svn ci --username $ASF_USERNAME --password "$ASF_PASSWORD" -m "UPDATE KEYS"
-```
-
-After that it will automatically sync to.
-```
-https://downloads.apache.org/doris/KEYS
-```
-
-In subsequent release poll emails, use the KEYS file here in ``https://downloads.apache.org/doris/KEYS``.
-
-## Maven Release Preparation
-
-For components such as the Doris Connector, you need to use maven for the release.
-
-1. Generate a master password
-
- `mvn --encrypt-master-password <password>`
-
- This password is only used to encrypt other passwords that follow, and the output is something like `{VSb+6+76djkH/43...} ` Then create the `~/.m2/settings-security.xml` file with the following content
-
- ```
- <settingsSecurity>
- <master>{VSb+6+76djkH/43...}</master>
- </settingsSecurity>
- ```
-
-2. Encrypt apache passwords
-
-
- `mvn --encrypt-password <password>`
-
- The password is the password for the apache account. The output is similar to `{GRKbCylpwysHfV...}`
-
- Add in `~/.m2/settings.xml`
-
- ```
- <servers>
- <!-- To publish a snapshot of your project -->
- <server>
- <id>apache.snapshots.https</id>
- <username>yangzhg</username>
- <password>{GRKbCylpwysHfV...}</password>
- </server>
- <!-- To stage a release of your project -->
- <server>
- <id>apache.releases.https</id>
- <username>yangzhg</username>
- <password>{GRKbCylpwysHfV...}</password>
- </server>
- </servers>
- ```
-
-## Initiating DISCUSS in the community
-
-DISCUSS is not a required process before a release, but it is highly recommended to start a discussion in the dev@doris mail group before a major release. Content includes, but is not limited to, descriptions of important features, bug fixes, etc.
diff --git a/community/release-and-verify/release-verify.md b/community/release-and-verify/release-verify.md
deleted file mode 100644
index f95c73d85de..00000000000
--- a/community/release-and-verify/release-verify.md
+++ /dev/null
@@ -1,100 +0,0 @@
----
-{
- "title": "Verify the Apache release version",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Validate Apache Releases
-
-This validation step can be used for validation during release polling and also for validation of released versions.
-
-The following steps can be followed to verify.
-
-1. [ ] The download link is legal.
-2. [ ] The PGP signature are valid.
-3. [ ] The source code matches the current release version.
-4. [ ] The LICENSE and NOTICE files are correct.
-5. [ ] All files carry the necessary protocol header.
-6. [ ] The compiled content is not included in the source package.
-7. [ ] The compilation can be executed smoothly.
-
-Here we use the verification of the Doris Core version as an example. Note that other components have their corresponding names changed.
-
-## 1. download the source package, signature file, checksum file and KEYS
-
-Download all relevant files, using a.b.c-incubating as an example:
-
-``` shell
-wget https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download&filename=/incubator/doris/a.b.c-incubating/apache-doris-a.b.c-incubating-src.tar.gz
-
-wget https://www.apache.org/dist/incubator/doris/a.b.c-incubating/apache-doris-a.b.c-incubating-src.tar.gz.sha512
-
-wget https://www.apache.org/dist/incubator/doris/a.b.c-incubating/apache-doris-a.b.c-incubating-src.tar.gz.asc
-
-wget https://downloads.apache.org/incubator/doris/KEYS
-```
-
-> In case of poll verification, you need to get the relevant files from the svn address provided in the email.
-
-## 2. Check signature and checksum value
-
-It is recommended to use GunPG, which can be installed by the following command.
-
-``` shell
-CentOS: yum install gnupg
-Ubuntu: apt-get install gnupg
-```
-
-``` shell
-gpg --import KEYS
-gpg --verify apache-doris-a.b.c-incubating-src.tar.gz.asc apache-doris-a.b.c-incubating-src.tar.gz
-sha512sum --check apache-doris-a.b.c-incubating-src.tar.gz.sha512
-```
-
-## 3. Verify the source protocol header
-
-Here we use [skywalking-eyes](https://github.com/apache/skywalking-eyes) for protocol validation.
-
-Go to the root of the source code and execute:
-
-```
-sudo docker run -it --rm -v $(pwd):/github/workspace apache/skywalking-eyes header check
-```
-
-The results of the run are as follows.
-
-```
-INFO GITHUB_TOKEN is not set, license-eye won't comment on the pull request
-INFO Loading configuration from file: .licenserc.yaml
-INFO Totally checked 5611 files, valid: 3926, invalid: 0, ignored: 1685, fixed: 0
-```
-
-If invalid is 0, then the validation passes.
-
-## 4. Verify compilation
-
-Please see the compilation documentation of each component to verify the compilation.
-
-* For Doris Core, see [compilation documentation](../../install/source-install/compilation.md)
-* Flink Doris Connector, see [compilation documentation](../../ecosystem/flink-doris-connector.md)
-* Spark Doris Connector, see [compilation documentation](../../ecosystem/spark-doris-connector.md)
diff --git a/community/security.md b/community/security.md
deleted file mode 100644
index cac4d8c6df2..00000000000
--- a/community/security.md
+++ /dev/null
@@ -1,38 +0,0 @@
----
-{
- "title": "Security",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Security
-
-The Apache Software Foundation takes a rigorous stance on eliminating security issues in its software projects. Likewise, Apache Doris is also vigilant and takes security issues related to its features and functionality into the highest consideration.
-
-If you have any concerns regarding Doris’s security, or you discover a vulnerability or potential threat, please don’t hesitate to get in touch with the [Apache Security Team](http://www.apache.org/security/) by dropping an email at [security@apache.org](mailto:security@apache.org).
-
-Please specify the project name as "Doris" in the email, and provide a description of the relevant problem or potential threat. You are also urged to recommend how to reproduce and replicate the issue.
-
-The Apache Security Team and the Doris community will get back to you after assessing and analyzing the findings.
-
-**Please note** that the security issue should be reported on the security email first, before disclosing it on any public domain.
-
diff --git a/community/subscribe-mail-list.md b/community/subscribe-mail-list.md
deleted file mode 100644
index 7f3a08ba773..00000000000
--- a/community/subscribe-mail-list.md
+++ /dev/null
@@ -1,69 +0,0 @@
----
-{
- "title": "Subscribe Mailing Lists",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Subscribe Mailing Lists
-
-Mail List is the most recognized form of communication in Apache community. Generally speaking, open source community questions and answers, technical discussions, transaction decisions are carried through mailing lists. The asynchronous and broadcasting features of mailing lists are also very suitable for communication in open source communities. So how do you subscribe to Apache Doris mailing lists? It mainly includes the following five steps.
-
-## 1. Send Subscription Mail
-
-Open your own email, create a new email, and send an email to `dev-subscribe@doris.apache.org` (subject and content are arbitrary)
-
-
-
-## 2. Receive confirmation emails from dev-help@doris.apache.org
-
-After the first step, you will receive a confirmation email from `dev-help@doris.apache.org`, which is shown below. (**If you fail to receive it for a long time, please confirm that the mail has been intercepted, or has been automatically grouped into "Subscribed Mail", "Spam Mail", "Promotional Mail" folders**)
-
-
-
-## 3. Reply to confirmation mail
-
-For the mail received in the previous step,
-
-**a. Reply to this email directly**
-
-***or***
-
-**B. Create a new `recipient` e-mail for the `reply address` in the previous step**
-
-Every subject is acceptable.
-
-
-
-
-## 4. Receiving Welcome Emails
-
-After completing the third step, you will receive a welcome email entitled **WELCOME to dev@doris.apache.org**. So far, the work of subscribing to mailing lists has been completed, and community dynamics will be notified by mail.
-
-
-
-
-## 5. Initiate e-mail discussion (optional)
-
-After successfully subscribing to the mailing list, if you want to initiate a discussion, send an email directly to `dev@doris.apache.org`. Anyone who subscribes to the mailing list receives the mail.
-
-
diff --git a/community/team.md b/community/team.md
deleted file mode 100644
index 6512bd917e8..00000000000
--- a/community/team.md
+++ /dev/null
@@ -1,90 +0,0 @@
----
-{
- "title": "Doris Team",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# PMC Members & Committers
-
-We'd like to thank the following committers to the Apache Doris project who have helped get the project to where it is today. This list might be stale, the canonical list is located on [Apache's website](https://people.apache.org/committers-by-project.html#doris).
-
-## PMC (15)
-
-| Apache ID | Github Username | Public Name |
-| :----------------------------------------------------------- | :------------------ | :------------ |
-| [lingbin](https://people.apache.org/committer-index.html#lingbin) | lingbin | Bin Ling |
-| [wangbo](https://people.apache.org/committer-index.html#wangbo) | wangbo | Bo Wang |
-| [caiconghui](https://people.apache.org/committer-index.html#caiconghui) | caiconghui | Conghui Cai |
-| [gaodayue](https://people.apache.org/committer-index.html#gaodayue) | gaodayue | Dayue Gao |
-| [lide](https://people.apache.org/committer-index.html#lide) | lide-reed, doris-ci | De Li |
-| [chenhao](https://people.apache.org/committer-index.html#chenhao) | chenhao7253886 | Hao Chen |
-| [lihaopeng](https://people.apache.org/committer-index.html#lihaopeng) | HappenLee | Haopeng Li |
-| [jiafengzheng](https://people.apache.org/committer-index.html#jiafengzheng) | hf200012 | Jiafeng Zhang |
-| [lingmiao](https://people.apache.org/committer-index.html#lingmiao) | EmmyMiao87 | Ling Miao |
-| [wenming](https://people.apache.org/committer-index.html#wenming) | moonming | Ming Wen |
-| [morningman](https://people.apache.org/committer-index.html#morningman) | morningman | Mingyu Chen |
-| [maruyue](https://people.apache.org/committer-index.html#maruyue) | maruyue | Ruyue Ma |
-| [ningjiang](https://people.apache.org/committer-index.html#ningjiang) | WillemJiang | Willem Ning Jiang |
-| [yangzhg](https://people.apache.org/committer-index.html#yangzhg) | yangzhg | Zhengguo Yang |
-| [weizuo](https://people.apache.org/committer-index.html#weizuo) | weizuo93 | Zuo Wei |
-
-## Committers (45)
-
-(the listing excludes PMC members above)
-
-| Apache ID | Github Username | Public Name |
-| :----------------------------------------------------------- | :--------------- | :------------- |
-| [lichaoyong](https://people.apache.org/committer-index.html#lichaoyong) | chaoyli | Chaoyong Li |
-| [zhaoc](https://people.apache.org/committer-index.html#zhaoc) | imay | Chun Zhao |
-| [diwu](https://people.apache.org/committer-index.html#diwu) | JNSimba | Di Wu |
-| [yiguolei](https://people.apache.org/committer-index.html#yiguolei) | yiguolei | Guolei Yi |
-| [liuhangyuan](https://people.apache.org/committer-index.html#liuhangyuan) | HangyuanLiu | Hangyuan Liu |
-| [huajianlan](https://people.apache.org/committer-index.html#huajianlan) | 924060929 | Huajian Lan |
-| [huangwei](https://people.apache.org/committer-index.html#huangwei) | vagetablechicken | Huang Wei |
-| [jianliangqi](https://people.apache.org/committer-index.html#jianliangqi) | qidaye | Jianliang Qi |
-| [kangkaisen](https://people.apache.org/committer-index.html#kangkaisen) | kangkaisen | Kaisen Kang |
-| [kangpinghuang](https://people.apache.org/committer-index.html#kangpinghuang) | kangpinghuang | Kangping Huang |
-| [liulijia](https://people.apache.org/committer-index.html#liulijia) | liutang123 | Lijia Liu |
-| [panxiaolei](https://people.apache.org/committer-index.html#panxiaolei) | BiteTheDDDDt | Xiaolei Pan |
-| [shaofengshi](https://people.apache.org/committer-index.html#shaofengshi) | shaofengshi | Shao Feng Shi |
-| [sijie](https://people.apache.org/committer-index.html#sijie) | sijie | Sijie Guo |
-| [weixiang](https://people.apache.org/committer-index.html#weixiang) | spaces-X | Xiang Wei |
-| [zouxinyi](https://people.apache.org/committer-index.html#zouxinyi) | xinyiZzz | Xinyi Zou |
-| [wangxixu](https://people.apache.org/committer-index.html#wangxixu) | xinghuayu007 | Xixu Wang |
-| [xuyang](https://people.apache.org/committer-index.html#xuyang) | xy720 | Yang Xu |
-| [laiyingchun](https://people.apache.org/committer-index.html#laiyingchun) | acelyc111 | Yingchun Lai |
-| [dataroaring](https://people.apache.org/committer-index.html#dataroaring) | dataroaring | Yongqiang Yang |
-| [wyf](https://people.apache.org/committer-index.html#wyf) | wuyunfeng | Yunfeng Wu |
-| [zenoyang](https://people.apache.org/committer-index.html#zenoyang) | zenoyang | Zeno Yang |
-| [zshao](https://people.apache.org/committer-index.html#zshao) | zshao | Zheng Shao |
-| [morrysnow](https://people.apache.org/committer-index.html#morrysnow) | morrySnow | Wenxin Zhang |
-| [jakevin](https://people.apache.org/committer-index.html#jakevin) | jackwener | Jie Wen |
-| [gabriellee](https://people.apache.org/committer-index.html#gabriellee) | Gabriel39 | Wenqiang Li |
-| [stalary](https://people.apache.org/committer-index.html#stalary) | stalary | Rongqian Li |
-| [cambyzju](https://people.apache.org/committer-index.html#cambyzju) | cambyzju | Xiaoli Zhu |
-| [adonisling](https://people.apache.org/committer-index.html#adonisling) | adonis0147 | Cong Ling |
-| [pengxiangyu](https://people.apache.org/committer-index.html#pengxiangyu) | pengxiangyu | Xiangyu Peng |
-
-## Contributors
-
-[All Contributors](https://github.com/apache/doris/graphs/contributors)
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/doris_storage_optimization.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/doris_storage_optimization.md
deleted file mode 100644
index 6e51eab8a5c..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/doris_storage_optimization.md
+++ /dev/null
@@ -1,234 +0,0 @@
----
-{
- "title": "Doris存储文件格式优化",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Doris存储文件格式优化 #
-
-## 文件格式 ##
-
-
-<center>图1. doris segment文件格式</center>
-
-文件包括：
-- 文件开始是8个字节的magic code，用于识别文件格式和版本
-- Data Region：用于存储各个列的数据信息，这里的数据是按需分page加载的
-- Index Region: doris中将各个列的index数据统一存储在Index Region，这里的数据会按照列粒度进行加载，所以跟列的数据信息分开存储
-- Footer信息
- - FileFooterPB:定义文件的元数据信息
- - 4个字节的footer pb内容的checksum
- - 4个字节的FileFooterPB消息长度，用于读取FileFooterPB
- - 8个字节的MAGIC CODE，之所以在末位存储，是方便不同的场景进行文件类型的识别
-
-文件中的数据按照page的方式进行组织，page是编码和压缩的基本单位。现在的page类型包括以下几种:
-
-### DataPage ###
-
-DataPage分为两种：nullable和non-nullable的data page。
-
-nullable的data page内容包括：
-```
-
- +----------------+
- | value count |
- |----------------|
- | first row id |
- |----------------|
- | bitmap length |
- |----------------|
- | null bitmap |
- |----------------|
- | data |
- |----------------|
- | checksum |
- +----------------+
-```
-
-non-nullable data page结构如下：
-
-```
- |----------------|
- | value count |
- |----------------|
- | first row id |
- |----------------|
- | data |
- |----------------|
- | checksum |
- +----------------+
-```
-
-其中各个字段含义如下：
-
-- value count
- - 表示page中的行数
-- first row id
- - page中第一行的行号
-- bitmap length
- - 表示接下来bitmap的字节数
-- null bitmap
- - 表示null信息的bitmap
-- data
- - 存储经过encoding和compress之后的数据
- - 需要在数据的头部信息中写入：is_compressed
- - 各种不同编码的data需要在头部信息写入一些字段信息，以实现数据的解析
- - TODO：添加各种encoding的header信息
-- checksum
- - 存储page粒度的校验和，包括page的header和之后的实际数据
-
-
-### Bloom Filter Pages ###
-
-针对每个bloom filter列,会在page的粒度相应的生成一个bloom filter的page，保存在bloom filter pages区域
-
-### Ordinal Index Page ###
-
-针对每个列，都会按照page粒度，建立行号的稀疏索引。内容为这个page的起始行的行号到这个block的指针（包括offset和length）
-
-### Short Key Index page ###
-
-我们会每隔N行（可配置）生成一个short key的稀疏索引，索引的内容为：short key->行号(ordinal)
-
-### Column的其他索引 ###
-
-该格式设计支持后续扩展其他的索引信息，比如bitmap索引，spatial索引等等，只需要将需要的数据写到现有的列数据后面，并且添加对应的元数据字段到FileFooterPB中
-
-### 元数据定义 ###
-SegmentFooterPB的定义为：
-
-```
-message ColumnPB {
- required int32 unique_id = 1; // 这里使用column id, 不使用column name是因为计划支持修改列名
- optional string name = 2; // 列的名字, 当name为__DORIS_DELETE_SIGN__, 表示该列为隐藏的删除列
- required string type = 3; // 列类型
- optional bool is_key = 4; // 是否是主键列
- optional string aggregation = 5; // 聚合方式
- optional bool is_nullable = 6; // 是否有null
- optional bytes default_value = 7; // 默认值
- optional int32 precision = 8; // 精度
- optional int32 frac = 9;
- optional int32 length = 10; // 长度
- optional int32 index_length = 11; // 索引长度
- optional bool is_bf_column = 12; // 是否有bf词典
- optional bool has_bitmap_index = 15 [default=false]; // 是否有bitmap索引
-}
-
-// page偏移
-message PagePointerPB {
- required uint64 offset; // page在文件中的偏移
- required uint32 length; // page的大小
-}
-
-message MetadataPairPB {
- optional string key = 1;
- optional bytes value = 2;
-}
-
-message ColumnMetaPB {
- optional ColumnMessage encoding; // 编码方式
-
- optional PagePointerPB dict_page // 词典page
- repeated PagePointerPB bloom_filter_pages; // bloom filter词典信息
- optional PagePointerPB ordinal_index_page; // 行号索引数据
- optional PagePointerPB page_zone_map_page; // page级别统计信息索引数据
-
- optional PagePointerPB bitmap_index_page; // bitmap索引数据
-
- optional uint64 data_footprint; // 列中索引的大小
- optional uint64 index_footprint; // 列中数据的大小
- optional uint64 raw_data_footprint; // 原始列数据大小
-
- optional CompressKind compress_kind; // 列的压缩方式
-
- optional ZoneMapPB column_zone_map; //文件级别的过滤条件
- repeated MetadataPairPB column_meta_datas;
-}
-
-message SegmentFooterPB {
- optional uint32 version = 2 [default = 1]; // 用于版本兼容和升级使用
- repeated ColumnPB schema = 5; // 列Schema
- optional uint64 num_values = 4; // 文件中保存的行数
- optional uint64 index_footprint = 7; // 索引大小
- optional uint64 data_footprint = 8; // 数据大小
- optional uint64 raw_data_footprint = 8; // 原始数据大小
-
- optional CompressKind compress_kind = 9 [default = COMPRESS_LZO]; // 压缩方式
- repeated ColumnMetaPB column_metas = 10; // 列元数据
- optional PagePointerPB key_index_page; // short key索引page
-}
-
-```
-
-## 读写逻辑 ##
-
-### 写入 ###
-
-大体的写入流程如下：
-1. 写入magic
-2. 根据schema信息，生成对应的ColumnWriter，每个ColumnWriter按照不同的类型，获取对应的encoding信息（可配置），根据encoding，生成对应的encoder
-3. 调用encoder->add(value)进行数据写入，每隔K行，生成一个short key index entry，并且，如果当前的page满足一定条件（大小超过1M或者行数为K），就生成一个新的page，缓存在内存中。
-4. 不断的循环步骤3，直到数据写入完成。将各个列的数据依序刷入文件中
-5. 生成FileFooterPB信息，写入文件中。
-
-相关的问题：
-
-- short key的索引如何生成？
- - 现在还是按照每隔多少行生成一个short key的稀疏索引，保持每隔1024行生成一个short的稀疏索引,具体的内容是：short key -> ordinal
-
-- ordinal索引里面应该存什么？
- - 存储page的第一个ordinal到page pointer的映射信息
-- 不同encoding类型的page里存什么？
- - 词典压缩
- - plain
- - rle
- - bshuf
-
-### 读取 ###
-
-1. 读取文件的magic，判断文件类型和版本
-2. 读取FileFooterPB，进行checksum校验
-3. 按照需要的列，读取short key索引和对应列的数据ordinal索引信息
-4. 使用start key和end key，通过short key索引定位到要读取的行号，然后通过ordinal索引确定需要读取的row ranges, 同时需要通过统计信息、bitmap索引等过滤需要读取的row ranges
-5. 然后按照row ranges通过ordinal索引读取行的数据
-
-相关的问题：
-1. 如何实现在page内部快速的定位到某一行？
-
- page内部是的数据是经过encoding的，无法快速进行行级数据的定位。不同的encoding方式，在内部进行快速的行号定位的方案不一样，需要具体分析：
- - 如果是rle编码的，需要通过解析rle的header进行skip，直到到达包含该行的那个rle块之后，再进行反解。
- - binary plain encoding：会在page的中存储offset信息，并且会在page header中指定offset信息的offset，读取的时候会先解析offset信息到数组中，这样子就可以通过各个行的offset数据信息快速的定位block某一行的数据
-2. 如何实现块的高效读取？可以考虑将相邻的块在读取的时候进行merge，一次性读取？
- 这个需要在读取的时候，判断block是否连续，如果连续，就一次性的读取
-
-## 编码 ##
-
-现有的doris存储中，针对string类型的编码，采用plain encoding的方式，效率比较低。经过对比，发现在百度统计的场景下，数据会因为string类型的编码膨胀超过一倍。所以，计划引入基于词典的编码压缩。
-
-## 压缩 ##
-
-实现可扩展的压缩框架，支持多种压缩算法，方便后续添加新的压缩算法，计划引入zstd压缩。
-
-## TODO ##
-1. 如何实现嵌套类型？如何在嵌套类型中进行行号定位？
-2. 如何优化现在的ScanRange拆分导致的下游bitmap、column statistic统计等进行多次？
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/flink_doris_connector_design.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/flink_doris_connector_design.md
deleted file mode 100644
index ca91982d37b..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/flink_doris_connector_design.md
+++ /dev/null
@@ -1,272 +0,0 @@
----
-{
- "title": "Flink Doris Connector设计方案",
- "language": "zh-CN"
-}
-
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under
-
-# Flink Doris Connector设计方案
-
-该方案首先感谢社区Spark Doris Connector的作者
-
-从Doris角度看，将其数据引入Flink，可以使用Flink一系列丰富的生态产品，拓宽了产品的想象力，也使得Doris和其他数据源的联合查询成为可能
-
-从我们业务架构出发和业务需求，我们选择了Flink作为我们架构的一部分，用于数据的ETL及实时计算框架，社区目前支持Spark doris connector，因此我们参照Spark doris connector 设计开发了Flink doris Connector。
-
-## 技术选型
-
-一开始我们选型的时候，也是和Spark Doris Connector 一样，开始考虑的是JDBC的方式，但是这种方式就像Spark doris connector那篇文章中说的，有优点，但是缺点更明显。后来我们阅读及测试了Spark的代码，决定站在巨人的肩上来实现（备注：直接拷贝代码修改）。
-
-以下内容来自Spark Doris Connector博客的，直接拷贝了
-
-```
-于是我们开发了针对Doris的新的Datasource，Spark-Doris-Connector。这种方案下，Doris可以暴露Doris数据分布给Spark。Spark的Driver访问Doris的FE获取Doris表的Schema和底层数据分布。之后，依据此数据分布，合理分配数据查询任务给Executors。最后，Spark的Executors分别访问不同的BE进行查询。大大提升了查询的效率
-```
-
-## 使用方法
-
-在Doris的代码库的 extension/flink-doris-connector/ 目录下编译生成doris-flink-1.0.0-SNAPSHOT.jar，将这个jar包加入flink的ClassPath中，即可使用Flink-on-Doris功能了
-
-#### SQL方式
-
-支持功能：
-
-1. 支持通过Flink SQL方式读取Doris数仓里表的数据到Flink里进行计算
-2. 支持通过Flink SQL将数据insert 到数仓对应的表中，后端实现是通过Stream Load直接和BE进行通讯完成数据插入操作
-3. 可以通过Flink关联非doris的外部数据源表进行关联分析
-
-示例：
-
-```java
- final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
- env.setParallelism(1);
- final StreamTableEnvironment tEnv = StreamTableEnvironment.create(env);
- tEnv.executeSql(
- "CREATE TABLE test_aggregation01 (" +
- "user_id STRING," +
- "user_city STRING," +
- "age INT," +
- "last_visit_date STRING" +
- ") " +
- "WITH (\n" +
- " 'connector' = 'doris',\n" +
- " 'fenodes' = 'doris01:8030',\n" +
- " 'table.identifier' = 'demo.test_aggregation',\n" +
- " 'username' = 'root',\n" +
- " 'password' = ''\n" +
- ")");
- tEnv.executeSql(
- "CREATE TABLE test_aggregation02 (" +
- "user_id STRING," +
- "user_city STRING," +
- "age INT," +
- "last_visit_date STRING" +
- ") " +
- "WITH (\n" +
- " 'connector' = 'doris',\n" +
- " 'fenodes' = 'doris01:8030',\n" +
- " 'table.identifier' = 'demo.test_aggregation_01',\n" +
- " 'username' = 'root',\n" +
- " 'password' = ''\n" +
- ")");
-
- tEnv.executeSql("INSERT INTO test_aggregation02 select * from test_aggregation01");
- tEnv.executeSql("select count(1) from test_aggregation01");
-```
-
-#### DataStream方式
-
-```java
-DorisOptions.Builder options = DorisOptions.builder()
- .setFenodes("$YOUR_DORIS_FE_HOSTNAME:$YOUR_DORIS_FE_RESFUL_PORT")
- .setUsername("$YOUR_DORIS_USERNAME")
- .setPassword("$YOUR_DORIS_PASSWORD")
- .setTableIdentifier("$YOUR_DORIS_DATABASE_NAME.$YOUR_DORIS_TABLE_NAME");
-env.addSource(new DorisSourceFunction<>(options.build(),new SimpleListDeserializationSchema())).print();
-```
-
-## 适用场景
-
-
-
-
-
-
-
-#### 1.使用Flink对Doris中的数据和其他数据源进行联合分析
-
-很多业务部门会将自己的数据放在不同的存储系统上，比如一些在线分析、报表的数据放在Doris中，一些结构化检索数据放在Elasticsearch中、一些需要事物的数据放在MySQL中，等等。业务往往需要跨多个存储源进行分析，通过Flink Doris Connector打通Flink和Doris后，业务可以直接使用Flink，将Doris中的数据与多个外部数据源做联合查询计算。
-
-#### 2.实时数据接入
-
-Flink Doris Connector之前：针对业务不规则数据，经常需要针对消息做规范处理，空值过滤等写入新的topic，然后再启动Routine load写入Doris。
-
-
-
-Flink Doris Connector之后：flink读取kafka，直接写入doris。
-
-
-
-
-
-## 技术实现
-
-### 架构图
-
-
-
-
-
-### Doris对外提供更多能力
-
-#### Doris FE
-
-对外开放了获取内部表的元数据信息、单表查询规划和部分统计信息的接口。
-
-所有的Rest API接口都需要进行HttpBasic认证，用户名和密码是登录数据库的用户名和密码，需要注意权限的正确分配。
-
-```
-// 获取表schema元信息
-GET api/{database}/{table}/_schema
-
-// 获取对单表的查询规划模版
-POST api/{database}/{table}/_query_plan
-{
-"sql": "select k1, k2 from {database}.{table}"
-}
-
-// 获取表大小
-GET api/{database}/{table}/_count
-```
-
-#### Doris BE
-
-
-通过Thrift协议，直接对外提供数据的过滤、扫描和裁剪能力。
-
-```
-service TDorisExternalService {
- // 初始化查询执行器
-TScanOpenResult open_scanner(1: TScanOpenParams params);
-
-// 流式batch获取数据，Apache Arrow数据格式
- TScanBatchResult get_next(1: TScanNextBatchParams params);
-
-// 结束扫描
- TScanCloseResult close_scanner(1: TScanCloseParams params);
-}
-```
-
-Thrift相关结构体定义可参考：
-
-https://github.com/apache/incubator-doris/blob/master/gensrc/thrift/DorisExternalService.thrift
-
-
-
-### 实现DataStream
-
-继承 org.apache.flink.streaming.api.functions.source.RichSourceFunction ，自定义DorisSourceFunction，初始化时，获取相关表的执行计划，获取对应的分区。
-
-重写run方法，循环从分区中读取数据。
-
-```java
-public void run(SourceContext sourceContext){
- //循环读取各分区
- for(PartitionDefinition partitions : dorisPartitions){
- scalaValueReader = new ScalaValueReader(partitions, settings);
- while (scalaValueReader.hasNext()){
- Object next = scalaValueReader.next();
- sourceContext.collect(next);
- }
- }
-}
-```
-
-
-
-### 实现Flink SQL on Doris
-
-参考了[Flink自定义Source&Sink](https://ci.apache.org/projects/flink/flink-docs-stable/zh/dev/table/sourceSinks.html) 和 Flink-jdbc-connector，实现了下面的效果，可以实现用Flink SQL直接操作Doris的表，包括读和写。
-
-#### 实现细节
-
-1.实现DynamicTableSourceFactory , DynamicTableSinkFactory 注册 doris connector
-
-2.自定义DynamicTableSource和DynamicTableSink 生成逻辑计划
-
-3.DorisRowDataInputFormat和DorisDynamicOutputFormat获取到逻辑计划后开始执行。
-
-
-
-
-
-实现中最主要的是基于RichInputFormat和RichOutputFormat 定制的DorisRowDataInputFormat和DorisDynamicOutputFormat。
-
-在DorisRowDataInputFormat中，将获取到的dorisPartitions 在createInputSplits中 切分成多个分片，用于并行计算。
-
-```java
-public DorisTableInputSplit[] createInputSplits(int minNumSplits) {
- List<DorisTableInputSplit> dorisSplits = new ArrayList<>();
- int splitNum = 0;
- for (PartitionDefinition partition : dorisPartitions) {
- dorisSplits.add(new DorisTableInputSplit(splitNum++,partition));
- }
- return dorisSplits.toArray(new DorisTableInputSplit[0]);
-}
-
-
-public RowData nextRecord(RowData reuse) {
- if (!hasNext) {
- //已经读完数据，返回null
- return null;
- }
- List next = (List)scalaValueReader.next();
- GenericRowData genericRowData = new GenericRowData(next.size());
- for(int i =0;i<next.size();i++){
- genericRowData.setField(i, next.get(i));
- }
- //判断是否还有数据
- hasNext = scalaValueReader.hasNext();
- return genericRowData;
-}
-
-```
-
-
-
-在DorisRowDataOutputFormat中，通过streamload的方式向doris中写数据。streamload程序参考org.apache.doris.plugin.audit.DorisStreamLoader
-
-```java
-public void writeRecord(RowData row) throws IOException {
- //streamload 默认分隔符 \t
- StringJoiner value = new StringJoiner("\t");
- GenericRowData rowData = (GenericRowData) row;
- for(int i = 0; i < row.getArity(); ++i) {
- value.add(rowData.getField(i).toString());
- }
- //streamload 写数据
- DorisStreamLoad.LoadResponse loadResponse = dorisStreamLoad.loadBatch(value.toString());
- System.out.println(loadResponse);
-}
-```
-
-
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/grouping_sets_design.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/grouping_sets_design.md
deleted file mode 100644
index 0c19094cffc..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/grouping_sets_design.md
+++ /dev/null
@@ -1,517 +0,0 @@
----
-{
- "title": "GROUPING SETS 设计文档",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# GROUPING SETS 设计文档
-
-## 1. GROUPING SETS 相关背景知识
-
-### 1.1 GROUPING SETS 子句
-
-GROUP BY GROUPING SETS 是对 GROUP BY 子句的扩展，它能够在一个 GROUP BY 子句中一次实现多个集合的分组。其结果等价于将多个相应 GROUP BY 子句进行 UNION 操作。
-
-特别地，一个空的子集意味着将所有的行聚集到一个分组。
-GROUP BY 子句是只含有一个元素的 GROUP BY GROUPING SETS 的特例。
-
-例如，GROUPING SETS 语句：
-
-```
-SELECT k1, k2, SUM( k3 ) FROM t GROUP BY GROUPING SETS ( (k1, k2), (k1), (k2), ( ) );
-```
-
-其查询结果等价于：
-
-```
-SELECT k1, k2, SUM( k3 ) FROM t GROUP BY k1, k2
-UNION
-SELECT k1, null, SUM( k3 ) FROM t GROUP BY k1
-UNION
-SELECT null, k2, SUM( k3 ) FROM t GROUP BY k2
-UNION
-SELECT null, null, SUM( k3 ) FROM t
-```
-
-下面是一个实际数据的例子：
-
-```
-mysql> SELECT * FROM t;
-+------+------+------+
-| k1 | k2 | k3 |
-+------+------+------+
-| a | A | 1 |
-| a | A | 2 |
-| a | B | 1 |
-| a | B | 3 |
-| b | A | 1 |
-| b | A | 4 |
-| b | B | 1 |
-| b | B | 5 |
-+------+------+------+
-8 rows in set (0.01 sec)
-
-mysql> SELECT k1, k2, SUM(k3) FROM t GROUP BY GROUPING SETS ( (k1, k2), (k2), (k1), ( ) );
-+------+------+-----------+
-| k1 | k2 | sum(`k3`) |
-+------+------+-----------+
-| b | B | 6 |
-| a | B | 4 |
-| a | A | 3 |
-| b | A | 5 |
-| NULL | B | 10 |
-| NULL | A | 8 |
-| a | NULL | 7 |
-| b | NULL | 11 |
-| NULL | NULL | 18 |
-+------+------+-----------+
-9 rows in set (0.06 sec)
-```
-
-### 1.2 ROLLUP 子句
-
-ROLLUP 是对 GROUPING SETS 的扩展。
-
-```
-SELECT a, b,c, SUM( d ) FROM tab1 GROUP BY ROLLUP(a,b,c)
-```
-
-这个 ROLLUP 等价于下面的 GROUPING SETS：
-
-```
-GROUPING SETS (
-(a,b,c),
-( a, b ),
-( a),
-( )
-)
-```
-
-### 1.3 CUBE 子句
-
-CUBE 也是对 GROUPING SETS 的扩展。
-
-```
-CUBE ( e1, e2, e3, ... )
-```
-
-其含义是 GROUPING SETS 后面列表中的所有子集。
-
-例如，CUBE ( a, b, c ) 等价于下面的 GROUPING SETS：
-
-```
-GROUPING SETS (
-( a, b, c ),
-( a, b ),
-( a, c ),
-( a ),
-( b, c ),
-( b ),
-( c ),
-( )
-)
-```
-
-### 1.4 GROUPING 和 GROUPING_ID 函数
-当我们没有统计某一列时，它的值显示为 NULL，这也可能是列本身就有 NULL 值，这就需要一种方法区分是没有统计还是值本来就是 NULL。为此引入 GROUPING 和 GROUPING_ID 函数。
-GROUPING(column:Column) 函数用于区分分组后的单个列是普通列和聚合列。如果是聚合列，则返回1，反之，则是0. GROUPING() 只能有一个参数列。
-
-GROUPING_ID(column1, column2) 则根据指定的column 顺序，否则根据聚合的时候给的集合的元素顺序，计算出一个列列表的 bitmap 值，一个列如果是聚合列为0，否则为1. GROUPING_ID()函数返回位向量的十进制值。
-比如 [0 1 0] ->2 从下列第三个查询可以看到这种对应关系
-
-例如，对于下面的表：
-
-```
-mysql> select * from t;
-+------+------+------+
-| k1 | k2 | k3 |
-+------+------+------+
-| a | A | 1 |
-| a | A | 2 |
-| a | B | 1 |
-| a | B | 3 |
-| b | A | 1 |
-| b | A | 4 |
-| b | B | 1 |
-| b | B | 5 |
-+------+------+------+
-```
-
-grouping sets 的结果如下：
-
-```
-mysql> SELECT k1, k2, GROUPING(k1), GROUPING(k2), SUM(k3) FROM t GROUP BY GROUPING SETS ( (k1, k2), (k2), (k1), ( ) );
-+------+------+----------------+----------------+-----------+
-| k1 | k2 | grouping(`k1`) | grouping(`k2`) | sum(`k3`) |
-+------+------+----------------+----------------+-----------+
-| a | A | 0 | 0 | 3 |
-| a | B | 0 | 0 | 4 |
-| a | NULL | 0 | 1 | 7 |
-| b | A | 0 | 0 | 5 |
-| b | B | 0 | 0 | 6 |
-| b | NULL | 0 | 1 | 11 |
-| NULL | A | 1 | 0 | 8 |
-| NULL | B | 1 | 0 | 10 |
-| NULL | NULL | 1 | 1 | 18 |
-+------+------+----------------+----------------+-----------+
-9 rows in set (0.02 sec)
-
-mysql> SELECT k1, k2, GROUPING_ID(k1,k2), SUM(k3) FROM t GROUP BY GROUPING SETS ( (k1, k2), (k2), (k1), ( ) );
-+------+------+-------------------------+-----------+
-| k1 | k2 | grouping_id(`k1`, `k2`) | sum(`k3`) |
-+------+------+-------------------------+-----------+
-| a | A | 0 | 3 |
-| a | B | 0 | 4 |
-| a | NULL | 1 | 7 |
-| b | A | 0 | 5 |
-| b | B | 0 | 6 |
-| b | NULL | 1 | 11 |
-| NULL | A | 2 | 8 |
-| NULL | B | 2 | 10 |
-| NULL | NULL | 3 | 18 |
-+------+------+-------------------------+-----------+
-9 rows in set (0.02 sec)
-
-mysql> SELECT k1, k2, grouping(k1), grouping(k2), GROUPING_ID(k1,k2), SUM(k4) FROM t GROUP BY GROUPING SETS ( (k1, k2), (k2), (k1), ( ) ) order by k1, k2;
-+------+------+----------------+----------------+-------------------------+-----------+
-| k1 | k2 | grouping(`k1`) | grouping(`k2`) | grouping_id(`k1`, `k2`) | sum(`k4`) |
-+------+------+----------------+----------------+-------------------------+-----------+
-| a | A | 0 | 0 | 0 | 3 |
-| a | B | 0 | 0 | 0 | 4 |
-| a | NULL | 0 | 1 | 1 | 7 |
-| b | A | 0 | 0 | 0 | 5 |
-| b | B | 0 | 0 | 0 | 6 |
-| b | NULL | 0 | 1 | 1 | 11 |
-| NULL | A | 1 | 0 | 2 | 8 |
-| NULL | B | 1 | 0 | 2 | 10 |
-| NULL | NULL | 1 | 1 | 3 | 18 |
-+------+------+----------------+----------------+-------------------------+-----------+
-9 rows in set (0.02 sec)
-
-```
-
-### 1.5 GROUPING SETS 的组合与嵌套
-
-首先，一个 GROUP BY 子句本质上是一个 GROUPING SETS 的特例, 例如：
-
-```
- GROUP BY a
-等同于
- GROUP BY GROUPING SETS((a))
-同样地，
- GROUP BY a,b,c
-等同于
- GROUP BY GROUPING SETS((a,b,c))
-```
-
-同样的，CUBE 和 ROLLUP 也可以展开成 GROUPING SETS，因此 GROUP BY, CUBE, ROLLUP, GROUPING SETS 的各种组合和嵌套本质上就是 GROUPING SETS 的组合与嵌套。
-
-对于 GROUPING SETS 的嵌套，语义上等价于将嵌套内的语句直接写到外面。（参考：<https://www.brytlyt.com/documentation/data-manipulation-dml/grouping-sets-rollup-cube/>），其中写道：
-
-```
-The CUBE and ROLLUP constructs can be used either directly in the GROUP BY clause, or nested inside a GROUPING SETS clause. If one GROUPING SETS clause is nested inside another, the effect is the same as if all the elements of the inner clause had been written directly in the outer clause.
-```
-
-对于多个 GROUPING SETS 的组合列表，很多数据库认为是叉乘（cross product）的关系。
-
-例如：
-
-```
-GROUP BY a, CUBE (b, c), GROUPING SETS ((d), (e))
-
-等同于：
-
-GROUP BY GROUPING SETS (
-(a, b, c, d), (a, b, c, e),
-(a, b, d), (a, b, e),
-(a, c, d), (a, c, e),
-(a, d), (a, e)
-)
-```
-
-对于 GROUPING SETS 的组合与嵌套，各个数据库支持不太一样。例如 snowflake 不支持任何的组合和嵌套。
-（<https://docs.snowflake.net/manuals/sql-reference/constructs/group-by.html>）
-
-Oracle 既支持组合，也支持嵌套。
-（<https://docs.oracle.com/cd/B19306_01/server.102/b14223/aggreg.htm#i1006842>）
-
-Presto 支持组合，但不支持嵌套。
-（<https://prestodb.github.io/docs/current/sql/select.html>）
-
-## 2. 设计目标
-
-从语法上支持 GROUPING SETS， ROLLUP 和 CUBE。实现上述所述的1.1, 1.2, 1.3 1.4.
-
-对于1.6 GROUPING SETS 的组合与嵌套 先不实现。
-
-具体语法列出如下：
-
-### 2.1 GROUPING SETS 语法
-
-```
-SELECT ...
-FROM ...
-[ ... ]
-GROUP BY GROUPING SETS ( groupSet [ , groupSet [ , ... ] ] )
-[ ... ]
-
-groupSet ::= { ( expr [ , expr [ , ... ] ] )}
-
-<expr>
-各种表达式，包括列名.
-
-```
-
-### 2.2 ROLLUP 语法
-
-```
-SELECT ...
-FROM ...
-[ ... ]
-GROUP BY ROLLUP ( expr [ , expr [ , ... ] ] )
-[ ... ]
-
-<expr>
-各种表达式，包括列名.
-
-```
-
-### 2.3 CUBE 语法
-
-```
-SELECT ...
-FROM ...
-[ ... ]
-GROUP BY CUBE ( expr [ , expr [ , ... ] ] )
-[ ... ]
-
-<expr>
-各种表达式，包括列名.
-
-```
-
-## 3. 实现方案
-
-### 3.1 整体思路
-
-既然 GROUPING SET 子句逻辑上等价于多个相应 GROUP BY 子句的 UNION，可以通过扩展输入行(此输入行已经是通过下推条件过滤和投影后的), 在此基础上进行一个单一的 GROUP BY 操作来达到目的。
-
-关键是怎样扩展输入行呢？下面举例说明：
-
-例如，对应下面的语句：
-
-```
-SELECT a, b FROM src GROUP BY a, b GROUPING SETS ((a, b), (a), (b), ());
-
-```
-
-假定 src 表的数据如下：
-
-```
-1, 2
-3, 4
-
-```
-
-根据 GROUPING SETS 子句给出的列表，可以将输入行扩展为下面的 8 行 （GROUPING SETS集合数 * 行数, 同时为每行生成对应的 全列的GROUPING_ID: 和其他grouping 函数的值
-
-```
-1, 2 (GROUPING_ID: a, b -> 00->0)
-1, null (GUPING_ID: a, null -> 01 -> 1)
-null, 2 (GROUPING_ID: null, b -> 10 -> 2)
-null, null (GROUPING_ID: null, null -> 11 -> 3)
-
-3, 4 (GROUPING_ID: a, b -> 00 -> 0)
-3, null (GROUPING_ID: a, null -> 01 -> 1)
-null, 4 (GROUPING_ID: null, b -> 10 -> 2)
-null, null (GROUPING_ID: null, null -> 11 -> 3)
-
-```
-
-然后，将上面的 8 行数据作为输入，对 a, b, GROUPING_ID 进行 GROUP BY 操作即可。
-
-### 3.2 具体例子验证说明
-
-假设有一个 t 表，包含如下列和数据：
-
-```
-mysql> select * from t;
-+------+------+------+
-| k1 | k2 | k3 |
-+------+------+------+
-| a | A | 1 |
-| a | A | 2 |
-| a | B | 1 |
-| a | B | 3 |
-| b | A | 1 |
-| b | A | 4 |
-| b | B | 1 |
-| b | B | 5 |
-+------+------+------+
-8 rows in set (0.01 sec)
-
-```
-
-对于如下的查询：
-
-```
-SELECT k1, k2, GROUPING_ID(k1,k2), SUM(k3) FROM t GROUP BY GROUPING SETS ((k1, k2), (k1), (k2), ());
-
-```
-
-首先，对输入行进行扩展，每行数据扩展成 4 行 (GROUPING SETS子句的集合数目)，同时增加 GROUPING_ID() 列 ：
-
-例如 a, A, 1 扩展后变成下面的 4 行：
-
-```
-+------+------+------+-------------------------+
-| k1 | k2 | k3 | GROUPING_ID(`k1`, `k2`) |
-+------+------+------+-------------------------+
-| a | A | 1 | 0 |
-| a | NULL | 1 | 1 |
-| NULL | A | 1 | 2 |
-| NULL | NULL | 1 | 3 |
-+------+------+------+-------------------------+
-
-```
-
-最终， 全部扩展后的输入行如下（总共 32 行）：
-
-```
-+------+------+------+-------------------------+
-| k1 | k2 | k3 | GROUPING_ID(`k1`, `k2`) |
-+------+------+------+-------------------------+
-| a | A | 1 | 0 |
-| a | A | 2 | 0 |
-| a | B | 1 | 0 |
-| a | B | 3 | 0 |
-| b | A | 1 | 0 |
-| b | A | 4 | 0 |
-| b | B | 1 | 0 |
-| b | B | 5 | 0 |
-| a | NULL | 1 | 1 |
-| a | NULL | 1 | 1 |
-| a | NULL | 2 | 1 |
-| a | NULL | 3 | 1 |
-| b | NULL | 1 | 1 |
-| b | NULL | 1 | 1 |
-| b | NULL | 4 | 1 |
-| b | NULL | 5 | 1 |
-| NULL | A | 1 | 2 |
-| NULL | A | 1 | 2 |
-| NULL | A | 2 | 2 |
-| NULL | A | 4 | 2 |
-| NULL | B | 1 | 2 |
-| NULL | B | 1 | 2 |
-| NULL | B | 3 | 2 |
-| NULL | B | 5 | 2 |
-| NULL | NULL | 1 | 3 |
-| NULL | NULL | 1 | 3 |
-| NULL | NULL | 1 | 3 |
-| NULL | NULL | 1 | 3 |
-| NULL | NULL | 2 | 3 |
-| NULL | NULL | 3 | 3 |
-| NULL | NULL | 4 | 3 |
-| NULL | NULL | 5 | 3 |
-+------+------+------+-------------------------+
-32 rows in set.
-
-```
-
-现在对k1, k2, GROUPING_ID(`k1`, `k2`) 进行 GROUP BY：
-
-```
-+------+------+-------------------------+-----------+
-| k1 | k2 | grouping_id(`k1`, `k2`) | sum(`k3`) |
-+------+------+-------------------------+-----------+
-| a | A | 0 | 3 |
-| a | B | 0 | 4 |
-| a | NULL | 1 | 7 |
-| b | A | 0 | 5 |
-| b | B | 0 | 6 |
-| b | NULL | 1 | 11 |
-| NULL | A | 2 | 8 |
-| NULL | B | 2 | 10 |
-| NULL | NULL | 3 | 18 |
-+------+------+-------------------------+-----------+
-9 rows in set (0.02 sec)
-
-```
-
-可以看到，其结果与对 GROUPING SETS 子句后每个子集进行 GROUP BY 后再进行 UNION 的结果一致。
-
-```
-select k1, k2, sum(k3) from t group by k1, k2
-UNION ALL
-select NULL, k2, sum(k3) from t group by k2
-UNION ALL
-select k1, NULL, sum(k3) from t group by k1
-UNION ALL
-select NULL, NULL, sum(k3) from t;
-
-+------+------+-----------+
-| k1 | k2 | sum(`k3`) |
-+------+------+-----------+
-| b | B | 6 |
-| b | A | 5 |
-| a | A | 3 |
-| a | B | 4 |
-| a | NULL | 7 |
-| b | NULL | 11 |
-| NULL | B | 10 |
-| NULL | A | 8 |
-| NULL | NULL | 18 |
-+------+------+-----------+
-9 rows in set (0.06 sec)
-
-```
-
-### 3.3 FE 规划阶段
-
-#### 3.3.1 主要任务
-
-1. 引入 GroupByClause 类，封装 Group By 相关信息，替换原有的 groupingExprs.
-2. 增加 Grouping Sets, Cube 和 RollUp 的语法支持和语法检查、错误处理和错误信息；
-3. 在 SelectStmt 类中增加 GroupByClause 成员；
-4. 引入 GroupingFunctionCallExpr 类，封装grouping 和grouping_id 函数调用
-5. 引入 VirtualSlot 类，封装grouping，grouping_id 生成的虚拟列和实际列的对应关系
-6. 增加虚拟列 GROUPING_ID 和其他grouping，grouping_id 函数对应的虚拟列，并将此列加入到原有的 groupingExprs 表达式列表中；
-7. 增加一个 PlanNode，考虑更通用的功能，命名为 RepeatNode。对于 GroupingSets 的聚合，在执行计划中插入 RepeatNode。
-
-#### 3.3.2 Tuple
-
-在 GroupByClause 类中为了将 GROUPING_ID 加到 groupingExprs 表达式列表中，需要创建 virtual SlotRef, 相应的，需要对这个 slot 创建一个 tuple, 叫 GROUPING_ID Tuple。
-
-对于 RepeatNode 这个执行计划，其输入是子节点的所有 tuple， 输出的 tuple 除了 repeat 子节点的数据外，还需要填写 GROUPING_ID 和其他grouping，grouping_id 对应的虚拟列，因此。
-
-
-### 3.4 BE 查询执行阶段
-
-主要任务：
-
-1. 通过 RepeatNode 的执行类，增加扩展输入行的逻辑，其功能是在聚合之前将原有数据进行 repeat：对每行增加一列 GROUPING_ID， 然后按照 GroupingSets 中的集合数进行 repeat，并对对应列置为 null。根据grouping list设置新增虚拟列的值
-2. 实现 grouping_id() 和grouping() 函数。
-
-
-
-
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/metadata-design.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/metadata-design.md
deleted file mode 100644
index 58e2772c4a6..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/metadata-design.md
+++ /dev/null
@@ -1,126 +0,0 @@
----
-{
- "title": "元数据设计文档",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# 元数据设计文档
-
-## 名词解释
-
-* FE：Frontend，即 Doris 的前端节点。主要负责接收和返回客户端请求、元数据以及集群管理、查询计划生成等工作。
-* BE：Backend，即 Doris 的后端节点。主要负责数据存储与管理、查询计划执行等工作。
-* bdbje：[Oracle Berkeley DB Java Edition](http://www.oracle.com/technetwork/database/berkeleydb/overview/index-093405.html)。在 Doris 中，我们使用 bdbje 完成元数据操作日志的持久化、FE 高可用等功能。
-
-## 整体架构
-
-
-如上图，Doris 的整体架构分为两层。多个 FE 组成第一层，提供 FE 的横向扩展和高可用。多个 BE 组成第二层，负责数据存储与管理。本文主要介绍 FE 这一层中，元数据的设计与实现方式。
-
-1. FE 节点分为 follower 和 observer 两类。各个 FE 之间，通过 bdbje（[BerkeleyDB Java Edition](http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/overview/index-093405.html)）进行 leader 选举，数据同步等工作。
-
-2. follower 节点通过选举，其中一个 follower 成为 leader 节点，负责元数据的写入操作。当 leader 节点宕机后，其他 follower 节点会重新选举出一个 leader，保证服务的高可用。
-
-3. observer 节点仅从 leader 节点进行元数据同步，不参与选举。可以横向扩展以提供元数据的读服务的扩展性。
-
-> 注：follower 和 observer 对应 bdbje 中的概念为 replica 和 observer。下文可能会同时使用两种名称。
-
-## 元数据结构
-
-Doris 的元数据是全内存的。每个 FE 内存中，都维护一个完整的元数据镜像。在百度内部，一个包含2500张表，100万个分片（300万副本）的集群，元数据在内存中仅占用约 2GB。（当然，查询所使用的中间对象、各种作业信息等内存开销，需要根据实际情况估算。但总体依然维持在一个较低的内存开销范围内。）
-
-同时，元数据在内存中整体采用树状的层级结构存储，并且通过添加辅助结构，能够快速访问各个层级的元数据信息。
-
-下图是 Doris 元信息所存储的内容。
-
-
-
-如上图，Doris 的元数据主要存储4类数据：
-
-1. 用户数据信息。包括数据库、表的 Schema、分片信息等。
-2. 各类作业信息。如导入作业，Clone 作业、SchemaChange 作业等。
-3. 用户及权限信息。
-4. 集群及节点信息。
-
-## 数据流
-
-
-
-元数据的数据流具体过程如下：
-
-1. 只有 leader FE 可以对元数据进行写操作。写操作在修改 leader 的内存后，会序列化为一条log，按照 key-value 的形式写入 bdbje。其中 key 为连续的整型，作为 log id，value 即为序列化后的操作日志。
-
-2. 日志写入 bdbje 后，bdbje 会根据策略（写多数/全写），将日志复制到其他 non-leader 的 FE 节点。non-leader FE 节点通过对日志回放，修改自身的元数据内存镜像，完成与 leader 节点的元数据同步。
-
-3. leader 节点的日志条数达到阈值（默认 10w 条）并且满足checkpoint线程执行周期（默认六十秒）。checkpoint 会读取已有的 image 文件，和其之后的日志，重新在内存中回放出一份新的元数据镜像副本。然后将该副本写入到磁盘，形成一个新的 image。之所以是重新生成一份镜像副本，而不是将已有镜像写成 image，主要是考虑写 image 加读锁期间，会阻塞写操作。所以每次 checkpoint 会占用双倍内存空间。
-
-4. image 文件生成后，leader 节点会通知其他 non-leader 节点新的 image 已生成。non-leader 主动通过 http 拉取最新的 image 文件，来更换本地的旧文件。
-
-5. bdbje 中的日志，在 image 做完后，会定期删除旧的日志。
-
-## 实现细节
-
-### 元数据目录
-
-1. 元数据目录通过 FE 的配置项 `meta_dir` 指定。
-
-2. `bdb/` 目录下为 bdbje 的数据存放目录。
-
-3. `image/` 目录下为 image 文件的存放目录。
-
- * `image.[logid]` 是最新的 image 文件。后缀 `logid` 表明 image 所包含的最后一条日志的 id。
- * `image.ckpt` 是正在写入的 image 文件，如果写入成功，会重命名为 `image.[logid]`，并替换掉旧的 image 文件。
- * `VERSION` 文件中记录着 `cluster_id`。`cluster_id` 唯一标识一个 Doris 集群。是在 leader 第一次启动时随机生成的一个 32 位整型。也可以通过 fe 配置项 `cluster_id` 来指定一个 cluster id。
- * `ROLE` 文件中记录的 FE 自身的角色。只有 `FOLLOWER` 和 `OBSERVER` 两种。其中 `FOLLOWER` 表示 FE 为一个可选举的节点。（注意：即使是 leader 节点，其角色也为 `FOLLOWER`）
-
-### 启动流程
-
-1. FE 第一次启动，如果启动脚本不加任何参数，则会尝试以 leader 的身份启动。在 FE 启动日志中会最终看到 `transfer from UNKNOWN to MASTER`。
-
-2. FE 第一次启动，如果启动脚本中指定了 `-helper` 参数，并且指向了正确的 leader FE 节点，那么该 FE 首先会通过 http 向 leader 节点询问自身的角色（即 ROLE）和 cluster_id。然后拉取最新的 image 文件。读取 image 文件，生成元数据镜像后，启动 bdbje，开始进行 bdbje 日志同步。同步完成后，开始回放 bdbje 中，image 文件之后的日志，完成最终的元数据镜像生成。
-
- > 注1：使用 `-helper` 参数启动时，需要首先通过 mysql 命令，通过 leader 来添加该 FE，否则，启动时会报错。
-
- > 注2：`-helper` 可以指向任何一个 follower 节点，即使它不是 leader。
-
- > 注2：bdbje 在同步日志过程中，fe 日志会显示 `xxx detached`, 此时正在进行日志拉取，属于正常现象。
-
-3. FE 非第一次启动，如果启动脚本不加任何参数，则会根据本地存储的 ROLE 信息，来确定自己的身份。同时根据本地 bdbje 中存储的集群信息，获取 leader 的信息。然后读取本地的 image 文件，以及 bdbje 中的日志，完成元数据镜像生成。（如果本地 ROLE 中记录的角色和 bdbje 中记录的不一致，则会报错。）
-
-4. FE 非第一次启动，且启动脚本中指定了 `-helper` 参数。则和第一次启动的流程一样，也会先去询问 leader 角色。但是会和自身存储的 ROLE 进行比较。如果不一致，则会报错。
-
-#### 元数据读写与同步
-
-1. 用户可以使用 mysql 连接任意一个 FE 节点进行元数据的读写访问。如果连接的是 non-leader 节点，则该节点会将写操作转发给 leader 节点。leader 写成功后，会返回一个 leader 当前最新的 log id。之后，non-leader 节点会等待自身回放的 log id 大于回传的 log id 后，才将命令成功的消息返回给客户端。这种方式保证了任意 FE 节点的 Read-Your-Write 语义。
-
- > 注：一些非写操作，也会转发给 leader 执行。比如 `SHOW LOAD` 操作。因为这些命令通常需要读取一些作业的中间状态，而这些中间状态是不写 bdbje 的，因此 non-leader 节点的内存中，是没有这些中间状态的。（FE 之间的元数据同步完全依赖 bdbje 的日志回放，如果一个元数据修改操作不写 bdbje 日志，则在其他 non-leader 节点中是看不到该操作修改后的结果的。）
-
-2. leader 节点会启动一个 TimePrinter 线程。该线程会定期向 bdbje 中写入一个当前时间的 key-value 条目。其余 non-leader 节点通过回放这条日志，读取日志中记录的时间，和本地时间进行比较，如果发现和本地时间的落后大于指定的阈值（配置项：`meta_delay_toleration_second`。写入间隔为该配置项的一半），则该节点会处于**不可读**的状态。此机制解决了 non-leader 节点在长时间和 leader 失联后，仍然提供过期的元数据服务的问题。
-
-3. 各个 FE 的元数据只保证最终一致性。正常情况下，不一致的窗口期仅为毫秒级。我们保证同一 session 中，元数据访问的单调一致性。但是如果同一 client 连接不同 FE，则可能出现元数据回退的现象。（但对于批量更新系统，该问题影响很小。）
-
-### 宕机恢复
-
-1. leader 节点宕机后，其余 follower 会立即选举出一个新的 leader 节点提供服务。
-2. 当多数 follower 节点宕机时，元数据不可写入。当元数据处于不可写入状态下，如果这时发生写操作请求，目前的处理流程是 **FE 进程直接退出**。后续会优化这个逻辑，在不可写状态下，依然提供读服务。
-3. observer 节点宕机，不会影响任何其他节点的状态。也不会影响元数据在其他节点的读写。
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/spark_load.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/spark_load.md
deleted file mode 100644
index 654d415862f..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/design/spark_load.md
+++ /dev/null
@@ -1,212 +0,0 @@
----
-{
- "title": "Doris支持spark导入设计文档",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Doris支持spark导入设计文档
-
-## 背景
-
-Doris现在支持Broker load/routine load/stream load/mini batch load等多种导入方式。
-spark load主要用于解决初次迁移，大量数据迁移doris的场景，用于提升数据导入的速度。
-
-## 名词解释
-
-* FE：Frontend，即 Palo 的前端节点。主要负责接收和返回客户端请求、元数据以及集群管理、查询计划生成等工作。
-* BE：Backend，即 Palo 的后端节点。主要负责数据存储与管理、查询计划执行等工作。
-* Tablet： 一个palo table的水平分片称为tablet。
-* Dpp：Data preprocessing，数据预处理模块，通过外部计算资源（Hadoop、Spark）完成对导入数据预处理，包括转化、清洗、分区、排序和聚合等。
-
-## 设计
-
-### 目标
-
-Doris中现有的导入方式中，针对百G级别以上的数据的批量导入支持不是很好，功能上需要修改很多配置，而且可能无法完成导入，性能上会比较慢，并且由于没有读写分离，需要占用较多的cpu等资源。而这种大数据量导入会在用户迁移的时候遇到，所以需要实现基于spark集群的导入功能，利用spark集群的并发能力，完成导入时的ETL计算，排序、聚合等等，满足用户大数据量导入需求，降低用户导入时间和迁移成本。
-
-在Spark导入中，需要考虑支持多种spark部署模式，设计上需要兼容多种部署方式，可以考虑先实现yarn集群的部署模式；同时，由于用户数据格式多种多样，需要支持包括csv、parquet、orc等多种格式的数据文件。
-
-### 实现方案
-
-在将spark导入的设计实现的时候，有必要讲一下现有的导入框架。现在有的导入框架，可以参考《Doris Broker导入实现解析》。
-
-#### 方案1
-
-参考现有的导入框架和原有适用于百度内部hadoop集群的hadoop导入方式的实现，为了最大程度复用现有的导入框架，降低开发的难度，整体的方案如下：
-
-用户的导入语句经过语法和语意分析之后，生成LoadStmt，LoadStmt中增加一个isSparkLoad标识字段，如果为true，就会创建出SparkLoadJob，跟BrokerLoadJob类似，会通过状态机机制，实现Job的执行，在PENDING，会创建SparkLoadPendingTask，然后在LOADING阶段还是创建LoadLoadingTask，进行数据导入。在BE中，复用现有的计划执行框架，执行导入计划。
-
-实现Spark导入主要需要考虑以下几点：
-
-##### 语法
- 这块主要考虑用户习惯，导入语句格式上尽量保持跟broker导入语句相似。下面是一个方案：
-
-```
- LOAD LABEL example_db.label1
- (
- DATA INFILE("hdfs://hdfs_host:hdfs_port/user/palo/data/input/file")
- NEGATIVE
- INTO TABLE `my_table`
- PARTITION (p1, p2)
- COLUMNS TERMINATED BY ","
- columns(k1,k2,k3,v1,v2)
- set (
- v3 = v1 + v2,
- k4 = hll_hash(k2)
- )
- where k1 > 20
- )
- with spark.cluster_name
- PROPERTIES
- (
- "spark.master" = "yarn",
- "spark.executor.cores" = "5",
- "spark.executor.memory" = "10g",
- "yarn.resourcemanager.address" = "xxx.tc:8032",
- "max_filter_ratio" = "0.1",
- );
-```
-其中spark.cluster_name为用户导入使用的Spark集群名，可以通过SET PROPERTY来设置，可参考原来Hadoop集群的设置。
-property中的Spark集群设置会覆盖spark.cluster_name中对应的内容。
-各个property的含义如下:
-- spark.master是表示spark集群部署模式，支持包括yarn/standalone/local/k8s，预计先实现yarn的支持，并且使用yarn-cluster模式（yarn-client模式一般用于交互式的场景）。
-- spark.executor.cores: executor的cpu个数
-- spark.executor.memory: executor的内存大小
-- yarn.resourcemanager.address：指定yarn的resourcemanager地址
-- max_filter_ratio：指定最大过滤比例阈值
-
-##### SparkLoadJob
-
-用户发送spark load语句，经过parse之后，会创建SparkLoadJob，
-
-```
-SparkLoadJob:
- +-------+-------+
- | PENDING |-----------------|
- +-------+-------+ |
- | SparkLoadPendingTask |
- v |
- +-------+-------+ |
- | LOADING |-----------------|
- +-------+-------+ |
- | LoadLoadingTask |
- v |
- +-------+-------+ |
- | COMMITTED |-----------------|
- +-------+-------+ |
- | |
- v v
- +-------+-------+ +-------+-------+
- | FINISHED | | CANCELLED |
- +-------+-------+ +-------+-------+
- | Λ
- +-------------------------+
-```
-上图为SparkLoadJob的执行流程。
-
-##### SparkLoadPendingTask
-SparkLoadPendingTask主要用来提交spark etl作业到spark集群中。由于spark支持不同部署模型（localhost, standalone, yarn, k8s），所以需要抽象一个通用的接口SparkEtlJob，实现SparkEtl的功能，主要接口包括：
-- 提交spark etl任务
-- 取消spark etl的任务
-- 获取spark etl任务状态的接口
-
-大体接口如下：
-```
-class SparkEtlJob {
- // 提交spark etl作业
- // 返回JobId
- String submitJob(TBrokerScanRangeParams params);
-
- // 取消作业，用于支持用户cancel导入作业
- bool cancelJob(String jobId);
-
- // 获取作业状态，用于判断是否已经完成
- JobStatus getJobStatus(String jobId);
-private:
- std::list<DataDescription> data_descriptions;
-};
-```
-可以实现不同的子类，来实现对不同集群部署模式的支持。可以实现SparkEtlJobForYarn用于支持yarn集群的spark导入作业。具体来说上述接口中JobId就是Yarn集群的appid，如何获取appid？一个方案是通过spark-submit客户端提交spark job，然后分析标准错误中的输出，通过文本匹配获取appid。
-
-这里需要参考hadoop dpp作业的经验，就是需要考虑任务运行可能因为数据量、集群队列等原因，会达到并发导入作业个数限制，导致后续任务提交失败，这块需要考虑一下任务堆积的问题。一个方案是可以单独设置spark load job并发数限制，并且针对每个用户提供一个并发数的限制，这样各个用户之间的作业可以不用相互干扰，提升用户体验。
-
-spark任务执行的事情，包括以下几个关键点：
-1. 类型转化（extraction/Transformation）
-
- 将源文件字段转成具体列类型（判断字段是否合法，进行函数计算等等）
-2. 函数计算（Transformation），包括negative计算
-
- 完成用户指定的列函数的计算。函数列表："strftime","time_format","alignment_timestamp","default_value","md5sum","replace_value","now","hll_hash","substitute"
-3. Columns from path的提取
-4. 进行where条件的过滤
-5. 进行分区和分桶
-6. 排序和预聚合
-
- 因为在OlapTableSink过程中会进行排序和聚合，逻辑上可以不需要进行排序和聚合，但是因为排序和预聚合可以提升在BE端执行导入的效率。**如果在spark etl作业中进行排序和聚合，那么在BE执行导入的时候可以省略这个步骤。**这块可以依据后续测试的情况进行调整。目前看，可以先在etl作业中进行排序。
- 还有一个需要考虑的就是如何支持bitmap类型中的全局字典，string类型的bitmap列需要依赖全局字典。
- 为了告诉下游etl作业是否已经完成已经完成排序和聚合，可以在作业完成的时候生成一个job.json的描述文件，里面包含如下属性：
-
- ```
- {
- "is_segment_file" : "false",
- "is_sort" : "true",
- "is_agg" : "true",
- }
- ```
- 其中：
- is_sort表示是否排序
- is_agg表示是否聚合
- is_segment_file表示是否生成的是segment文件
-
-7. 现在rollup数据的计算都是基于base表，需要考虑能够根据index之间的层级关系，优化rollup数据的生成。
-
-这里面相对比较复杂一点就是列的表达式计算的支持。
-
-最后，spark load作业完成之后，产出的文件存储格式可以支持csv、parquet、orc，从存储效率上来说，建议默认为parquet。
-
-##### LoadLoadingTask
-
-LoadLoadingTask可以复现现在的逻辑，但是，有一个地方跟BrokerLoadJob不一样的地址就是，经过SparkEtlTask处理过后的数据文件已经完成列映射、函数计算、负导入、过滤、聚合等操作，这个时候LoadLoadingTask就不用进行这些操作，只需要进行简单的列映射和类型转化。
-
-##### BE导入任务执行
-
-这块可以完全复用现有的导入框架，应该不需要做改动。
-
-#### 方案2
-
-方案1可以最大限度的复用现有的导入框架，能够快速实现支持大数据量导入的功能。但是存在以下问题，就是经过spark etl处理之后的数据其实已经按照tablet划分好了，但是现有的Broker导入框架还是会对流式读取的数据进行分区和bucket计算，然后经过序列化通过rpc发送到对应的目标BE的机器，有一次序列化和网络IO的开销。 方案2是在SparkEtlJob生成数据的时候，直接生成doris的存储格式Segment文件，然后三个副本需要通过类似clone机制的方式，通过add_rowset接口，进行文件的导入。这种方案具体不一样的地方如下：
-
-1. 需要在生成的文件中添加tabletid后缀
-2. 在SparkLoadPendingTask类中增加一个接口protected Map<long, Pair<String, Long>> getFilePathMap()用于返回tabletid和文件之间的映射关系，
-3. 在BE rpc服务中增加一个spark_push接口，实现拉取源端etl转化之后的文件到本地（可以通过broker读取），然后通过add_rowset接口完成数据的导入，类似克隆的逻辑
-4. 生成新的导入任务SparkLoadLoadingTask,该SparkLoadLoadingTask主要功能就是读取job.json文件，解析其中的属性并且，将属性作为rpc参数，调用spark_push接口，向tablet所在的后端BE发送导入请求，进行数据的导入。BE中spark_push根据is_segment_file来决定如何处理，如果为true，则直接下载segment文件，进行add rowset；如果为false，则走pusher逻辑，实现数据导入。
-
-该方案将segment文件的生成也统一放到了spark集群中进行，能够极大的降低doris集群的负载，效率应该会比较高。但是方案2需要依赖于将底层rowset和segment v2的接口打包成独立的so文件，并且通过spark调用该接口来将数据转化成segment文件。
-
-## 总结
-
-综合以上两种方案，第一种方案的改动量比较小，但是BE做了重复的工作。第二种方案可以参考原有的Hadoop导入框架。所以，计划分两步完成spark load的工作。
-
-第一步，按照方案2，实现通过Spark完成导入数据的分区排序聚合，生成parquet格式文件。然后走Hadoop pusher的流程由BE转化格式。
-
-第二步，封装segment写入的库，直接生成Doris底层的格式，并且增加一个rpc接口，实现类似clone的导入逻辑。
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/be-vscode-dev.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/be-vscode-dev.md
deleted file mode 100644
index 4bcaf5d4073..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/be-vscode-dev.md
+++ /dev/null
@@ -1,294 +0,0 @@
----
-{ 'title': 'Doris BE开发调试环境 -- vscode', 'language': 'zh-CN' }
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Apache Doris Be 开发调试
-
-## 前期准备工作
-
-**本教程是在 Ubuntu 20.04 下进行的**
-
-**文中的出现的 BE 二进制文件名称 `doris_be`，在之前的版本中为 `palo_be`。**
-
-1. 下载 doris 源代码
-
- 下载地址为：[apache/doris: Apache Doris (github.com)](https://github.com/apache/doris)
-
-2. 安装 GCC 8.3.1+，Oracle JDK 1.8+，Python 2.7+，确认 gcc, java, python 命令指向正确版本, 设置 JAVA_HOME 环境变量
-
-3. 安装其他依赖包
-
-```
-sudo apt install build-essential openjdk-8-jdk maven cmake byacc flex automake libtool-bin bison binutils-dev libiberty-dev zip unzip libncurses5-dev curl git ninja-build python brotli
-sudo add-apt-repository ppa:ubuntu-toolchain-r/ppa
-sudo apt update
-sudo apt install gcc-10 g++-10
-sudo apt-get install autoconf automake libtool autopoint
-```
-
-4. 安装 openssl libssl-dev
-
-```
-sudo apt install -y openssl libssl-dev
-```
-
-## 编译
-
-以下操作步骤在 /home/workspace 目录下进行
-
-1. 下载源码
-
-```
-git clone https://github.com/apache/doris.git
-```
-
-2. 编译第三方依赖包
-
-```
- cd /home/workspace/doris/thirdparty
- ./build-thirdparty.sh
-```
-
-3. 编译 doris 产品代码
-
-```
-cd /home/workspace/doris
-./build.sh
-```
-
-注意：这个编译有以下几条指令：
-
-```
-./build.sh #同时编译be 和fe
-./build.sh --be #只编译be
-./build.sh --fe #只编译fe
-./build.sh --fe --be#同时编译be fe
-./build.sh --fe --be --clean#删除并同时编译be fe
-./build.sh --fe --clean#删除并编译fe
-./build.sh --be --clean#删除并编译be
-./build.sh --be --fe --clean#删除并同时编译be fe
-```
-
-如果不出意外，应该会编译成功，最终的部署文件将产出到 /home/workspace/doris/output/ 目录下。如果还遇到其他问题，可以参照 doris 的安装文档 http://doris.apache.org。
-
-## 部署调试(GDB)
-
-1. 给 be 编译结果文件授权
-
-```
-chmod /home/workspace/doris/output/be/lib/doris_be
-```
-
-注意： /home/workspace/doris/output/be/lib/doris_be 为 be 的执行文件。
-
-2. 创建数据存放目录
-
-通过查看/home/workspace/doris/output/be/conf/be.conf
-
-```
-# INFO, WARNING, ERROR, FATAL
-sys_log_level = INFO
-be_port = 9060
-be_rpc_port = 9070
-webserver_port = 8040
-heartbeat_service_port = 9050
-brpc_port = 8060
-
-# Note that there should at most one ip match this list.
-# If no ip match this rule, will choose one randomly.
-# use CIDR format, e.g. 10.10.10.0/
-# Default value is empty.
-priority_networks = 192.168.59.0/24 # data root path, separate by ';'
-storage_root_path = /soft/be/storage
-# sys_log_dir = ${PALO_HOME}/log
-# sys_log_roll_mode = SIZE-MB-
-# sys_log_roll_num =
-# sys_log_verbose_modules =
-# log_buffer_level = -
-# palo_cgroups
-```
-
-需要创建一个文件夹，这是 be 数据存放的地方
-
-```
-mkdir -p /soft/be/storage
-```
-
-3. 打开 vscode，并打开 be 源码所在目录，在本案例中打开目录为 **/home/workspace/doris/**
-
-4. 安装 vscode ms c++ 调试插件
-
-
-
-5. 创建 launch.json 文件，文件内容如下：
-
-```
-{
- "version": "0.2.0",
- "configurations": [
- {
- "name": "(gdb) Launch",
- "type": "cppdbg",
- "request": "launch",
- "program": "/home/workspace/doris/output/be/lib/doris_be",
- "args": [],
- "stopAtEntry": false,
- "cwd": "/home/workspace/doris/",
- "environment": [{"name":"PALO_HOME","value":"/home/workspace/doris/output/be/"},
- {"name":"UDF_RUNTIME_DIR","value":"/home/workspace/doris/output/be/lib/udf-runtime"},
- {"name":"LOG_DIR","value":"/home/workspace/doris/output/be/log"},
- {"name":"PID_DIR","value":"/home/workspace/doris/output/be/bin"}
- ],
- "externalConsole": true,
- "MIMode": "gdb",
- "setupCommands": [
- {
- "description": "Enable pretty-printing for gdb",
- "text": "-enable-pretty-printing",
- "ignoreFailures": true
- }
- ]
- }
- ]
-}
-```
-
-其中，environment 定义了几个环境变量 DORIS_HOME UDF_RUNTIME_DIR LOG_DIR PID_DIR，这是 doris_be 运行时需要的环境变量，如果没有设置，启动会失败。
-
-**注意：如果希望是 attach(附加进程）调试，配置代码如下：**
-
-```
-{
- "version": "0.2.0",
- "configurations": [
- {
- "name": "(gdb) Launch",
- "type": "cppdbg",
- "request": "attach",
- "program": "/home/workspace/doris/output/lib/doris_be",
- "processId":,
- "MIMode": "gdb",
- "internalConsoleOptions":"openOnSessionStart",
- "setupCommands": [
- {
- "description": "Enable pretty-printing for gdb",
- "text": "-enable-pretty-printing",
- "ignoreFailures": true
- }
- ]
- }
- ]
-}
-```
-
-配置中 **"request": "attach"， "processId":PID**，这两个配置是重点： 分别设置 gdb 的调试模式为 attach，附加进程的 processId，否则会失败。如何查找进程 id，可以在命令行中输入以下命令：
-
-```
-ps -ef | grep palo*
-```
-
-如图：
-
-
-
-其中的 15200 即为当前运行的 be 的进程 id.
-
-一个完整的 launch.json 的例子如下：
-
-```
- {
- "version": "0.2.0",
- "configurations": [
- {
- "name": "(gdb) Attach",
- "type": "cppdbg",
- "request": "attach",
- "program": "/home/workspace/doris/output/be/lib/doris_be",
- "processId": 17016,
- "MIMode": "gdb",
- "setupCommands": [
- {
- "description": "Enable pretty-printing for gdb",
- "text": "-enable-pretty-printing",
- "ignoreFailures": true
- }
- ]
- },
- {
- "name": "(gdb) Launch",
- "type": "cppdbg",
- "request": "launch",
- "program": "/home/workspace/doris/output/be/lib/doris_be",
- "args": [],
- "stopAtEntry": false,
- "cwd": "/home/workspace/doris/output/be",
- "environment": [
- {
- "name": "DORIS_HOME",
- "value": "/home/workspace/doris/output/be"
- },
- {
- "name": "UDF_RUNTIME_DIR",
- "value": "/home/workspace/doris/output/be/lib/udf-runtime"
- },
- {
- "name": "LOG_DIR",
- "value": "/home/workspace/doris/output/be/log"
- },
- {
- "name": "PID_DIR",
- "value": "/home/workspace/doris/output/be/bin"
- }
- ],
- "externalConsole": false,
- "MIMode": "gdb",
- "setupCommands": [
- {
- "description": "Enable pretty-printing for gdb",
- "text": "-enable-pretty-printing",
- "ignoreFailures": true
- }
- ]
- }
- ]
-}
-```
-
-6. 点击调试即可
-
- 下面就可以开始你的 Doris DEBUG 之旅了
-
-
-
-## 调试(LLDB)
-
-lldb的attach比gdb更快，使用方式和gdb类似。vscode需要安装的插件改为`CodeLLDB`，然后在launch中加入如下配置:
-```json
-{
- "name": "CodeLLDB attach",
- "type": "lldb",
- "request": "attach",
- "program": "${workspaceFolder}/output/be/lib/doris_be",
- "pid":"${command:pickMyProcess}"
-}
-```
-需要注意的是，此方式要求系统`glibc`版本为`2.18+`。如果没有则可以参考 [如何使CodeLLDB在CentOS7下工作](https://gist.github.com/JaySon-Huang/63dcc6c011feb5bd6deb1ef0cf1a9b96) 安装高版本glibc并将其链接到插件。
\ No newline at end of file
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/benchmark-tool.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/benchmark-tool.md
deleted file mode 100644
index 8bb45332c7a..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/benchmark-tool.md
+++ /dev/null
@@ -1,150 +0,0 @@
----
-{
- "title": "Doris BE存储层Benchmark工具",
- "language": "zh-CN"
-}
-
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Doris BE存储层Benchmark工具
-
-## 用途
-
- 可以用来测试BE存储层的一些部分(例如segment、page)的性能。根据输入数据构造出指定对象,利用google benchmark进行性能测试。
-
-## 编译
-
-1. 确保环境已经能顺利编译Doris本体,可以参考[编译与部署](/docs/install/source-install/compilation)。
-
-2. 运行目录下的`run-be-ut.sh`
-
-3. 编译出的可执行文件位于`./be/ut_build_ASAN/test/tools/benchmark_tool`
-
-## 使用
-
-#### 使用随机生成的数据集进行Segment读取测试
-
-会先利用数据集写入一个`segment`文件,然后对scan整个`segment`的耗时进行统计。
-
-> ./benchmark_tool --operation=SegmentScan --column_type=int,varchar --rows_number=10000 --iterations=0
-
-这里的`column_type`可以设置表结构,`segment`层的表结构类型目前支持`int、char、varchar、string`,`char`类型的长度为`8`,`varchar`和`string`类型长度限制都为最大值。默认值为`int,varchar`。
-
-数据集按以下规则生成。
->int: 在[1,1000000]内随机。
-
-字符串类型的数据字符集为大小写英文字母,长度根据类型不同。
-> char: 长度在[1,8]内随机。
-> varchar: 长度在[1,128]内随机。
-> string: 长度在[1,100000]内随机。
-
-`rows_number`表示数据的行数,默认值为`10000`。
-
-`iterations`表示迭代次数,benchmark会重复进行测试,然后计算平均耗时。如果`iterations`为`0`则表示由benchmark自动选择迭代次数。默认值为`10`。
-
-#### 使用随机生成的数据集进行Segment写入测试
-
-对将数据集添加进segment并写入磁盘的流程进行耗时统计。
-
-> ./benchmark_tool --operation=SegmentWrite
-
-#### 使用从文件导入的数据集进行Segment读取测试
-
-> ./benchmark_tool --operation=SegmentScanByFile --input_file=./sample.dat
-
-这里的`input_file`为导入的数据集文件。
-数据集文件第一行为表结构定义,之后每行分别对应一行数据,每个数据用`,`隔开。
-
-举例:
-```
-int,char,varchar
-123,hello,world
-321,good,bye
-```
-
-类型支持同样为`int`、`char`、`varchar`、`string`,注意`char`类型数据长度不能超过8。
-
-#### 使用从文件导入的数据集进行Segment写入测试
-
-> ./benchmark_tool --operation=SegmentWriteByFile --input_file=./sample.dat
-
-#### 使用随机生成的数据集进行page字典编码测试
-
-> ./benchmark_tool --operation=BinaryDictPageEncode --rows_number=10000 --iterations=0
-
-会随机生成长度在[1,8]之间的varchar,并对编码进行耗时统计。
-
-#### 使用随机生成的数据集进行page字典解码测试
-
-> ./benchmark_tool --operation=BinaryDictPageDecode
-
-会随机生成长度在[1,8]之间的varchar并编码,并对解码进行耗时统计。
-
-## Custom测试
-
-这里支持用户使用自己编写的函数进行性能测试,具体可以实现在`/be/test/tools/benchmark_tool.cpp`。
-例如实现有：
-```cpp
-void custom_run_plus() {
- int p = 100000;
- int q = 0;
- while (p--) {
- q++;
- if (UNLIKELY(q == 1024)) q = 0;
- }
-}
-void custom_run_mod() {
- int p = 100000;
- int q = 0;
- while (p--) {
- q++;
- if (q %= 1024) q = 0;
- }
-}
-```
-则可以通过注册`CustomBenchmark`来加入测试。
-```cpp
-benchmarks.emplace_back(
- new doris::CustomBenchmark("custom_run_plus", 0,
- custom_init, custom_run_plus));
-benchmarks.emplace_back(
- new doris::CustomBenchmark("custom_run_mod", 0,
- custom_init, custom_run_mod));
-```
-这里的`init`为每轮测试的初始化步骤(不会计入耗时),如果用户有需要初始化的对象则可以通过`CustomBenchmark`的派生类来实现。
-运行后有如下结果:
-```
-2021-08-30T10:29:35+08:00
-Running ./benchmark_tool
-Run on (96 X 3100.75 MHz CPU s)
-CPU Caches:
- L1 Data 32 KiB (x48)
- L1 Instruction 32 KiB (x48)
- L2 Unified 1024 KiB (x48)
- L3 Unified 33792 KiB (x2)
-Load Average: 0.55, 0.53, 0.39
-----------------------------------------------------------
-Benchmark Time CPU Iterations
-----------------------------------------------------------
-custom_run_plus 0.812 ms 0.812 ms 861
-custom_run_mod 1.30 ms 1.30 ms 539
-```
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/bitmap-hll-file-format.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/bitmap-hll-file-format.md
deleted file mode 100644
index 8e2aacd0c14..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/bitmap-hll-file-format.md
+++ /dev/null
@@ -1,64 +0,0 @@
----
-{
-"title": "Bitmap/HLL 数据格式",
-"language": "zh-CN"
-}
-
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-## Bitmap 格式
-### 格式说明
-Doris 中的bitmap 采用的是roaring bitmap 存储， be 端使用CRoaring，`Roaring` 的序列化格式在C++/java/go 等语言中兼容， 而C++ `Roaring64Map` 的格式序列化结果和Java中`Roaring64NavigableMap` 不兼容。Doris bimap 有5种类型， 分别用一个字节表示
-
-Doris 中的bitmap 序列化格式说明如下:
-
-```
- | flag | data .....|
- <--1Byte--><--n bytes-->
-```
-
-Flag 值说明如下：
-
-| 值 | 描述 |
-| ---- | ------------------------------------------------------------ |
-| 0 | EMPTY，空 bitmap, 后面data 部分为空，整个序列化结果只有一个字节 |
-| 1 | SINGLE32，bitmap 中只有一个32位无符号整型值， 后面4个字节表示32位无符号整型值 |
-| 2 | BITMAP32，32 位bitmap 对应java 中类型为 `org.roaringbitmap.RoaringBitmap` C++ 中类型为`roaring::Roaring`， 后面data 为roaring::Roaring 序列后的结构， 可以使用java 中的 `org.roaringbitmap.RoaringBitmap` 或c++中`roaring::Roaring` 直接反序列化 |
-| 3 | SINGLE64 ,bitmap 中只有一个64位无符号整型值，后面8个字节表示64位无符号整型值 |
-| 4 | BITMAP64, 64 位bitmap 对应java 中类型为 `org.roaringbitmap.RoaringBitmap;` c++ 中类型为doris 中的`Roaring64Map`，数据结构和 roaring 库中的结果一致，但是序列化和反序列话方法有所不同，后面会有1-8个字节的变长编码的uint64 的表述bitmap 中size。后面的数据是各式是多个 4个字节的高位表示和32位 roaring bitmap 序列化数据重复而成 |
-
-c++ 序列化和反序列化的示例 在 `be/src/util/bitmap_value.h` 的`BitmapValue::write()` `BitmapValue::deserialize()` 方法中, Java示例在 `fe/fe-common/src/main/java/org/apache/doris/common/io/BitmapValue.java` 中的`serialize()` `deserialize()` 方法中。
-
-## HLL 格式说明
-
-HLL 格式序列化数据在Doris 中自己实现的。同Bitmap 类型类似，HLL 格式是1个字节的flag 后面跟随多个字节数据组成，flag 含义如下
-
-
-
-| 值 | 描述 |
-| ---- | ------------------------------------------------------------ |
-| 0 | HLL_DATA_EMPTY，空 HLL, 后面data 部分为空，整个序列化结果只有一个字节 |
-| 1 | HLL_DATA_EXPLICIT， 后面1个字节 explicit 数据块个数，后面接多个数据块，每个数据块由8个字节长度和数据组成， |
-| 2 | HLL_DATA_SPARSE，只存非0 值，后面4个字节 表示 register 个数， 后面连续多个 register 结构，每个register 由前面2个字节的index 和1个字节的值组成 |
-| 3 | HLL_DATA_FULL ,表示所有16 * 1024个register都有值， 后面连续16 * 1024个字节的值数据 |
-
-c++ 序列化和反序列化的示例 在 `be/src/olap/hll.h` 的`serialize()` `deserialize()` 方法中, Java示例在 `fe/fe-common/src/main/java/org/apache/doris/common/io/hll.java` 中的`serialize()` `deserialize()` 方法中。
\ No newline at end of file
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/cpp-diagnostic-code.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/cpp-diagnostic-code.md
deleted file mode 100644
index 2b45a294496..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/cpp-diagnostic-code.md
+++ /dev/null
@@ -1,54 +0,0 @@
----
-{
- "title": "C++ 代码分析",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# C++ 代码分析
-
-Doris支持使用[Clangd](https://clangd.llvm.org/)和[Clang-Tidy](https://clang.llvm.org/extra/clang-tidy/)进行代码静态分析。Clangd和Clang-Tidy在 [LDB-toolchain](/docs/install/source-install/compilation-with-ldb-toolchain)
-中已经内置，另外也可以自己安装或者编译。
-
-### Clang-Tidy
-Clang-Tidy中可以做一些代码分析的配置,配置文件`.clang-tidy`在Doris根目录下。
-
-### 在VSCODE中配置Clangd
-
-首先需要安装clangd插件，然后在`settings.json`中编辑或者直接在首选项中更改插件配置。相比于vscode-cpptools，clangd可以为vscode提供更强大和准确的代码转跳，并且集成了clang-tidy的分析和快速修复功能。
-
-```json
- "clangd.path": "ldb_toolchain/bin/clangd", //clangd的路径
- "clangd.arguments": [
- "--background-index",
- "--clang-tidy", //开启clang-tidy
- "--compile-commands-dir=doris/be/build_RELEASE/", //会用到cmake生成的compile_commands.json,所以需要先编译一次生成该文件
- "--completion-style=detailed",
- "-j=5", //clangd分析文件的并行数
- "--all-scopes-completion",
- "--pch-storage=memory",
- "--pretty",
- "-log=verbose",
- "--query-driver=ldb_toolchain/bin/*" //编译器路径
- ],
- "clangd.trace": "/home/disk2/pxl/dev/baidu/bdg/doris/core/output/clangd-server.log" //clangd的日志路径,可以自己设定
-```
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/cpp-format-code.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/cpp-format-code.md
deleted file mode 100644
index 5a9a51d8934..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/cpp-format-code.md
+++ /dev/null
@@ -1,105 +0,0 @@
----
-{
- "title": "C++ 代码格式化",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# C++ 代码格式化
-
-Doris使用clang-format进行代码格式化，并在build-support目录下提供了封装脚本：
-
-* `clang-format.sh`.
-
- 格式化 `be/src` 和 `be/test` 目录下的 C/C++ 代码。
-
-* `check-format.sh`.
-
- 检查 `be/src` 和 `be/test` 目录下的 C/C++ 代码格式，并将 diff 输出，但不会修改文件内容。
-
-## 代码风格定制
-
-Doris的代码风格在Google Style的基础上稍有改动，定制为 `.clang-format` 文件，位于Doris根目录。
-
-目前，`.clang-format` 配置文件适配clang-format-13.0.1以上的版本。
-
-`.clang-format-ignore` 文件中记录了不希望被格式化的代码。这些代码通常来自第三方代码库，建议保持原有代码风格。
-
-## 环境准备
-
-需要下载安装clang-format，也可使用IDE或Editor提供的clang-format插件，下面分别介绍。
-
-### 下载安装clang-format
-
-推荐使用 NPM 安装 clang-format 14（不同版本的 clang-format 可能产生不同的代码格式，建议使用 14 版本）：
-
-`npm install clang-format@1.6.0`
-
-Ubuntu: `apt-get install clang-format`
-
-当前版本为10.0，也可指定旧版本，例如: `apt-get install clang-format-9`，建议源码编译14.0版本。
-
-Mac: `brew install clang-format`
-
-Centos 7:
-
-centos yum安装的clang-format版本过老，支持的StyleOption太少，建议源码编译14.0版本。
-
-LDB toolchain:
-
-如果使用 [LDB toolchain](/docs/install/source-install/compilation-with-ldb-toolchain)，
-最新版本的 [LDB toolchain](https://github.com/amosbird/ldb_toolchain_gen/releases)（>= v0.11）已经包含了预编译的clang-format
-13.0.1的二进制文件。
-
-### clang-format插件
-
-Clion IDE可使用插件"ClangFormat"，`File->Setting->Plugins`搜索下载。但版本无法和
-clang-format程序的版本匹配，从支持的StyleOption上看，应该是低于clang-format-9.0。
-
-## 使用方式
-
-### 命令行运行
-
-cd到Doris根目录下，然后执行如下命令:
-
-`build-support/clang-format.sh`
-
-> 注：`clang-format.sh`脚本要求您的机器上安装了python 3
-
-### 在IDE或Editor中使用clang-format
-
-#### Clion
-
-Clion如果使用插件，点击`Reformat Code`即可。
-
-#### VS Code
-
-VS Code需安装扩展程序Clang-Format，但需要自行提供clang-format执行程序的位置。
-
-打开VS Code配置页面，直接搜索"clang_format"，填上
-
-```
-"clang_format_path": "$clang-format path$",
-"clang_format_style": "file"
-```
-
-然后，右键点击`Format Document`即可。
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/debug-tool.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/debug-tool.md
deleted file mode 100644
index bf56879c286..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/debug-tool.md
+++ /dev/null
@@ -1,334 +0,0 @@
----
-{
- "title": "调试工具",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# 调试工具
-
-在Doris的使用、开发过程中，经常会遇到需要对Doris进行调试的场景，这里介绍一些常用的调试工具。
-
-**文中的出现的BE二进制文件名称 `doris_be`，在之前的版本中为 `palo_be`。**
-
-## FE 调试
-
-FE 是 Java 进程。这里只列举一下简单常用的 java 调试命令。
-
-1. 统计当前内存使用明细
-
- ```
- jmap -histo:live pid > 1.jmp
- ```
-
- 该命令可以列举存活的对象的内存占用并排序。（pid 换成 FE 进程 id）
-
- ```
- num #instances #bytes class name
- ----------------------------------------------
- 1: 33528 10822024 [B
- 2: 80106 8662200 [C
- 3: 143 4688112 [Ljava.util.concurrent.ForkJoinTask;
- 4: 80563 1933512 java.lang.String
- 5: 15295 1714968 java.lang.Class
- 6: 45546 1457472 java.util.concurrent.ConcurrentHashMap$Node
- 7: 15483 1057416 [Ljava.lang.Object;
- ```
-
- 可以通过这个方法查看目前存活对象占用的总内存（在文件最后），以及分析哪些对象占用了更多的内存。
-
- 注意，这个方法因指定了 `:live`，因此会触发 FullGC。
-
-2. 查看 JVM 内存使用
-
- ```
- jstat -gcutil pid 1000 1000
- ```
-
- 该命令可以滚动查看当前 JVM 各区域的内存使用情况。（pid 换成 FE 进程 id）
-
- ```
- S0 S1 E O M CCS YGC YGCT FGC FGCT GCT
- 0.00 0.00 22.61 3.03 95.74 92.77 68 1.249 5 0.794 2.043
- 0.00 0.00 22.61 3.03 95.74 92.77 68 1.249 5 0.794 2.043
- 0.00 0.00 22.61 3.03 95.74 92.77 68 1.249 5 0.794 2.043
- 0.00 0.00 22.92 3.03 95.74 92.77 68 1.249 5 0.794 2.043
- 0.00 0.00 22.92 3.03 95.74 92.77 68 1.249 5 0.794 2.043
- ```
-
- 其中主要关注 Old区（O）的占用百分比（如示例中为 3%）。如果占用过高，则可能出现 OOM 或 FullGC。
-
-3. 打印 FE 线程堆栈
-
- ```
- jstack -l pid > 1.js
- ```
-
- 该命令可以打印当前 FE 的线程堆栈。（pid 换成 FE 进程 id）。
-
- `-l` 参数会同时检测是否有死锁。该方法可以查看 FE 线程运行情况，是否有死锁，哪里卡住了等问题。
-
-## BE 调试
-
-### 环境准备
-
-[pprof](https://github.com/google/pprof): 来自gperftools，用于将gperftools所产生的内容转化成便于人可以阅读的格式，比如pdf, svg, text等.
-
-[graphviz](http://www.graphviz.org/): 在没有这个库的时候pprof只可以转化为text格式，但这种方式不易查看。那么安装这个库后，pprof可以转化为svg、pdf等格式，对于调用关系则更加清晰明了。
-
-[perf](https://perf.wiki.kernel.org/index.php/Main_Page): linux内核自带性能分析工具。[这里](http://www.brendangregg.com/perf.html)有一些perf的使用例子。
-
-[FlameGraph](https://github.com/brendangregg/FlameGraph): 可视化工具，用于将perf的输出以火焰图的形式展示出来。
-
-### 内存
-
-对于内存的调试一般分为两个方面。一个是内存使用的总量是否合理，内存使用量过大一方面可能是由于系统存在内存泄露，另一方面可能是因为程序内存使用不当。其次就是是否存在内存越界、非法访问的问题，比如程序访问一个非法地址的内存，使用了未初始化内存等。对于内存方面的调试我们一般使用如下几种方式来进行问题追踪。
-
-#### 查看日志
-
-当发现内存使用量过大的时候，我们可以先查看be.out日志，看看是否有大内存申请。由于Doris当前使用的TCMalloc管理内存，那么遇到大内存申请时，都会将申请的堆栈打印到be.out文件中，一般的表现形式如下：
-
-```
-tcmalloc: large alloc 1396277248 bytes == 0x3f3488000 @ 0x2af6f63 0x2c4095b 0x134d278 0x134bdcb 0x133d105 0x133d1d0 0x19930ed
-```
-
-这个表示在Doris BE在这个堆栈上尝试申请`1396277248 bytes`的内存。我们可以通过`addr2line`命令去把堆栈还原成我们能够看懂的信，具体的例子如下所示。
-
-```
-$ addr2line -e lib/doris_be 0x2af6f63 0x2c4095b 0x134d278 0x134bdcb 0x133d105 0x133d1d0 0x19930ed
-
-/home/ssd0/zc/palo/doris/core/thirdparty/src/gperftools-gperftools-2.7/src/tcmalloc.cc:1335
-/home/ssd0/zc/palo/doris/core/thirdparty/src/gperftools-gperftools-2.7/src/tcmalloc.cc:1357
-/home/disk0/baidu-doris/baidu/bdg/doris-baidu/core/be/src/exec/hash_table.cpp:267
-/home/disk0/baidu-doris/baidu/bdg/doris-baidu/core/be/src/exec/hash_table.hpp:86
-/home/disk0/baidu-doris/baidu/bdg/doris-baidu/core/be/src/exec/hash_join_node.cpp:239
-/home/disk0/baidu-doris/baidu/bdg/doris-baidu/core/be/src/exec/hash_join_node.cpp:213
-thread.cpp:?
-```
-
-#### HEAP PROFILE
-
-有时内存的申请并不是大内存的申请导致，而是通过小内存不断的堆积导致的。那么就没有办法通过查看日志定位到具体的申请信息，那么就需要通过其他方式来获得信息。
-
-这个时候我们可以利用TCMalloc的[HEAP PROFILE](https://gperftools.github.io/gperftools/heapprofile.html)的功能。如果设置了HEAPPROFILE功能，那么我们可以获得进程整体的内存申请使用情况。使用方式是在启动Doris BE前设置`HEAPPROFILE`环境变量。比如：
-
-```
-export HEAPPROFILE=/tmp/doris_be.hprof
-./bin/start_be.sh --daemon
-```
-
-这样，当满足HEAPPROFILE的dump条件时，就会将内存的整体使用情况写到指定路径的文件中。后续我们就可以通过使用`pprof`工具来对输出的内容进行分析。
-
-```
-$ pprof --text lib/doris_be /tmp/doris_be.hprof.0012.heap | head -30
-
-Using local file lib/doris_be.
-Using local file /tmp/doris_be.hprof.0012.heap.
-Total: 668.6 MB
- 610.6 91.3% 91.3% 610.6 91.3% doris::SystemAllocator::allocate_via_malloc (inline)
- 18.1 2.7% 94.0% 18.1 2.7% _objalloc_alloc
- 5.6 0.8% 94.9% 63.4 9.5% doris::RowBatch::RowBatch
- 5.1 0.8% 95.6% 7.1 1.1% butil::ResourcePool::add_block (inline)
- 3.7 0.5% 96.2% 3.7 0.5% butil::iobuf::create_block (inline)
- 3.4 0.5% 96.7% 3.4 0.5% butil::FlatMap::init
- 3.2 0.5% 97.2% 5.2 0.8% butil::ObjectPool::add_block (inline)
- 2.6 0.4% 97.6% 2.6 0.4% __gnu_cxx::new_allocator::allocate (inline)
- 2.0 0.3% 97.9% 2.0 0.3% butil::ObjectPool::add_block_group (inline)
- 2.0 0.3% 98.2% 2.0 0.3% butil::ResourcePool::add_block_group (inline)
- 1.7 0.3% 98.4% 1.7 0.3% doris::SegmentReader::_load_index
-```
-
-上述文件各个列的内容：
-
-* 第一列：函数直接申请的内存大小，单位MB
-* 第四列：函数以及函数所有调用的函数总共内存大小。
-* 第二列、第五列分别是第一列与第四列的比例值。
-* 第三列是个第二列的累积值。
-
-当然也可以生成调用关系图片，更加方便分析。比如下面的命令就能够生成SVG格式的调用关系图。
-
-```
-pprof --svg lib/doris_be /tmp/doris_be.hprof.0012.heap > heap.svg
-```
-
-**注意：开启这个选项是要影响程序的执行性能的，请慎重对线上的实例开启**
-
-#### pprof remote server
-
-HEAP PROFILE虽然能够获得全部的内存使用信息，但是也有比较受限的地方。1. 需要重启BE进行。2. 需要一直开启这个命令，导致对整个进程的性能造成影响。
-
-对Doris BE也可以使用动态开启、关闭heap profile的方式来对进程进行内存申请分析。Doris内部支持了GPerftools的[远程server调试](https://gperftools.github.io/gperftools/pprof_remote_servers.html)。那么可以通过`pprof`直接对远程运行的Doris BE进行动态的HEAP PROFILE。比如我们可以通过以下命令来查看Doris的内存的使用增量
-
-```
-$ pprof --text --seconds=60 http://be_host:be_webport/pprof/heap
-
-Total: 1296.4 MB
- 484.9 37.4% 37.4% 484.9 37.4% doris::StorageByteBuffer::create
- 272.2 21.0% 58.4% 273.3 21.1% doris::RowBlock::init
- 157.5 12.1% 70.5% 157.5 12.1% doris::RowBatch::RowBatch
- 90.7 7.0% 77.5% 90.7 7.0% doris::SystemAllocator::allocate_via_malloc
- 66.6 5.1% 82.7% 66.6 5.1% doris::IntegerColumnReader::init
- 47.9 3.7% 86.4% 47.9 3.7% __gnu_cxx::new_allocator::allocate
- 20.8 1.6% 88.0% 35.4 2.7% doris::SegmentReader::_load_index
- 12.7 1.0% 89.0% 12.7 1.0% doris::DecimalColumnReader::init
- 12.7 1.0% 89.9% 12.7 1.0% doris::LargeIntColumnReader::init
- 12.7 1.0% 90.9% 12.7 1.0% doris::StringColumnDirectReader::init
- 12.3 0.9% 91.9% 12.3 0.9% std::__cxx11::basic_string::_M_mutate
- 10.4 0.8% 92.7% 10.4 0.8% doris::VectorizedRowBatch::VectorizedRowBatch
- 10.0 0.8% 93.4% 10.0 0.8% doris::PlainTextLineReader::PlainTextLineReader
-```
-
-这个命令的输出与HEAP PROFILE的输出及查看方式一样，这里就不再详细说明。这个命令只有在执行的过程中才会开启统计，相比HEAP PROFILE对于进程性能的影响有限。
-
-#### LSAN
-
-[LSAN](https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer)是一个地址检查工具，GCC已经集成。在我们编译代码的时候开启相应的编译选项，就能够开启这个功能。当程序发生可以确定的内存泄露时，会将泄露堆栈打印。Doris BE已经集成了这个工具，只需要在编译的时候使用如下的命令进行编译就能够生成带有内存泄露检测版本的BE二进制
-
-```
-BUILD_TYPE=LSAN ./build.sh
-```
-
-当系统检测到内存泄露的时候，就会在be.out里面输出对应的信息。为了下面的演示，我们故意在代码中插入一段内存泄露代码。我们在`StorageEngine`的`open`函数中插入如下代码
-
-```
- char* leak_buf = new char[1024];
- strcpy(leak_buf, "hello world");
- LOG(INFO) << leak_buf;
-```
-
-我们就在be.out中获得了如下的输出
-
-```
-=================================================================
-==24732==ERROR: LeakSanitizer: detected memory leaks
-
-Direct leak of 1024 byte(s) in 1 object(s) allocated from:
- #0 0xd10586 in operator new[](unsigned long) ../../../../gcc-7.3.0/libsanitizer/lsan/lsan_interceptors.cc:164
- #1 0xe333a2 in doris::StorageEngine::open(doris::EngineOptions const&, doris::StorageEngine**) /home/ssd0/zc/palo/doris/core/be/src/olap/storage_engine.cpp:104
- #2 0xd3cc96 in main /home/ssd0/zc/palo/doris/core/be/src/service/doris_main.cpp:159
- #3 0x7f573b5eebd4 in __libc_start_main (/opt/compiler/gcc-4.8.2/lib64/libc.so.6+0x21bd4)
-
-SUMMARY: LeakSanitizer: 1024 byte(s) leaked in 1 allocation(s).
-```
-
-从上述的输出中，我们能看到有1024个字节被泄露了，并且打印出来了内存申请时的堆栈信息。
-
-**注意：开启这个选项是要影响程序的执行性能的，请慎重对线上的实例开启**
-
-**注意：如果开启了LSAN开关的话，tcmalloc就会被自动关闭**
-
-#### ASAN
-
-除了内存使用不合理、泄露以外。有的时候也会发生内存访问非法地址等错误。这个时候我们可以借助[ASAN](https://github.com/google/sanitizers/wiki/AddressSanitizer)来辅助我们找到问题的原因。与LSAN一样，ASAN也集成在了GCC中。Doris通过如下的方式进行编译就能够开启这个功能
-
-```
-BUILD_TYPE=ASAN ./build.sh
-```
-
-执行编译生成的二进制文件，当检测工具发现有异常访问时，就会立即退出，并将非法访问的堆栈输出在be.out中。对于ASAN的输出与LSAN是一样的分析方法。这里我们也主动注入一个地址访问错误，来展示下具体的内容输出。我们仍然在`StorageEngine`的`open`函数中注入一段非法内存访问，具体的错误代码如下
-
-```
- char* invalid_buf = new char[1024];
- for (int i = 0; i < 1025; ++i) {
- invalid_buf[i] = i;
- }
- LOG(INFO) << invalid_buf;
-```
-
-然后我们就会在be.out中获得如下的输出
-
-```
-=================================================================
-==23284==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61900008bf80 at pc 0x00000129f56a bp 0x7fff546eed90 sp 0x7fff546eed88
-WRITE of size 1 at 0x61900008bf80 thread T0
- #0 0x129f569 in doris::StorageEngine::open(doris::EngineOptions const&, doris::StorageEngine**) /home/ssd0/zc/palo/doris/core/be/src/olap/storage_engine.cpp:106
- #1 0xe2c1e3 in main /home/ssd0/zc/palo/doris/core/be/src/service/doris_main.cpp:159
- #2 0x7fa5580fbbd4 in __libc_start_main (/opt/compiler/gcc-4.8.2/lib64/libc.so.6+0x21bd4)
- #3 0xd30794 (/home/ssd0/zc/palo/doris/core/output3/be/lib/doris_be+0xd30794)
-
-0x61900008bf80 is located 0 bytes to the right of 1024-byte region [0x61900008bb80,0x61900008bf80)
-allocated by thread T0 here:
- #0 0xdeb040 in operator new[](unsigned long) ../../../../gcc-7.3.0/libsanitizer/asan/asan_new_delete.cc:82
- #1 0x129f50d in doris::StorageEngine::open(doris::EngineOptions const&, doris::StorageEngine**) /home/ssd0/zc/palo/doris/core/be/src/olap/storage_engine.cpp:104
- #2 0xe2c1e3 in main /home/ssd0/zc/palo/doris/core/be/src/service/doris_main.cpp:159
- #3 0x7fa5580fbbd4 in __libc_start_main (/opt/compiler/gcc-4.8.2/lib64/libc.so.6+0x21bd4)
-
-SUMMARY: AddressSanitizer: heap-buffer-overflow /home/ssd0/zc/palo/doris/core/be/src/olap/storage_engine.cpp:106 in doris::StorageEngine::open(doris::EngineOptions const&, doris::StorageEngine**)
-```
-
-从这段信息中该可以看到在`0x61900008bf80`这个地址我们尝试去写一个字节，但是这个地址是非法的。我们也可以看到 `[0x61900008bb80,0x61900008bf80)`这个地址的申请堆栈。
-
-**注意：开启这个选项是要影响程序的执行性能的，请慎重对线上的实例开启**
-
-**注意：如果开启了ASAN开关的话，tcmalloc就会被自动关闭**
-
-另外，如果be.out中输出了堆栈信息，但是并没有函数符号，那么这个时候需要我们手动的处理下才能获得可读的堆栈信息。具体的处理方法需要借助一个脚本来解析ASAN的输出。这个时候我们需要使用[asan_symbolize](https://llvm.org/svn/llvm-project/compiler-rt/trunk/lib/asan/scripts/asan_symbolize.py)来帮忙解析下。具体的使用方式如下：
-
-```
-cat be.out | python asan_symbolize.py | c++filt
-```
-
-通过上述的命令，我们就能够获得可读的堆栈信息了。
-
-### CPU
-
-当系统的CPU Idle很低的时候，说明系统的CPU已经成为了主要瓶颈，这个时候就需要分析一下当前的CPU使用情况。对于Doris的BE可以有如下两种方式来分析Doris的CPU瓶颈。
-
-#### pprof
-
-由于Doris内部已经集成了并兼容了GPerf的REST接口，那么用户可以通过`pprof`工具来分析远程的Doris BE。具体的使用方式如下：
-
-```
-pprof --svg --seconds=60 http://be_host:be_webport/pprof/profile > be.svg
-```
-
-这样就能够生成一张BE执行的CPU消耗图。
-
-
-
-#### perf + flamegragh
-
-这个是相当通用的一种CPU分析方式，相比于`pprof`，这种方式必须要求能够登陆到分析对象的物理机上。但是相比于pprof只能定时采点，perf是能够通过不同的事件来完成堆栈信息采集的。具体的使用方式如下：
-
-```
-perf record -g -p be_pid -- sleep 60
-```
-
-这条命令会统计60秒钟BE的CPU运行情况，并且生成perf.data。对于perf.data的分析，可以通过perf的命令来进行分析
-
-```
-perf report
-```
-
-分析得到如下的图片
-
-
-
-来对生成的内容进行分析。当然也可以使用flamegragh完成可视化展示。
-
-```
-perf script | ./FlameGraph/stackcollapse-perf.pl | ./FlameGraph/flamegraph.pl > be.svg
-```
-
-这样也会生成一张当时运行的CPU消耗图。
-
-
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/docker-dev.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/docker-dev.md
deleted file mode 100644
index 95991b23292..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/docker-dev.md
+++ /dev/null
@@ -1,162 +0,0 @@
----
-{
- "title": "Doris Docker 快速搭建开发环境",
- "language": "zh-CN"
-}
-
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Doris Docker 快速搭建开发环境
-
-## 相关详细文档导航
-
-- [使用 Docker 开发镜像编译](/docs/install/source-install/compilation)
-- [部署](/docs/install/install-deploy)
-- [VSCode Be 开发调试](./be-vscode-dev)
-
-## 环境准备
-
-- 安装 Docker
-- VSCode
- - Remote 插件
-
-## 构建镜像
-
-创建 dockerfile
-
-VSCode 中使用 Ctrl-d 替换掉所有的
-
-- <!!! your user !!!>
-- <!!! your user password !!!>
-- <!!! root password !!!>
-- <!!! your git email !!!>
-- <!!! your git username !!!>
-
-```dockerfile
-FROM apache/incubator-doris:build-env-latest
-
-USER root
-WORKDIR /root
-RUN echo '<!!! root password !!!>' | passwd root --stdin
-
-RUN yum install -y vim net-tools man wget git mysql lsof bash-completion \
- && cp /var/local/thirdparty/installed/bin/thrift /usr/bin
-
-# 更安全的使用，创建用户而不是使用 root
-RUN yum install -y sudo \
- && useradd -ms /bin/bash <!!! your user !!!> && echo <!!! your user password !!!> | passwd <!!! your user !!!> --stdin \
- && usermod -a -G wheel <!!! your user !!!>
-
-USER <!!! your user !!!>
-WORKDIR /home/<!!! your user !!!>
-RUN git config --global color.ui true \
- && git config --global user.email "<!!! your git email !!!>" \
- && git config --global user.name "<!!! your git username !!!>"
-
-# 按需安装 zsh and oh my zsh, 更易于使用，不需要的移除
-USER root
-RUN yum install -y zsh \
- && chsh -s /bin/zsh <!!! your user !!!>
-USER <!!! your user !!!>
-RUN wget https://github.com/robbyrussell/oh-my-zsh/raw/master/tools/install.sh -O - | zsh \
- && git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions \
- && git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting
-```
-
-运行构建命令
-
-```bash
-docker build -t doris .
-```
-
-运行镜像
-
-此处按需注意 [挂载的问题](../../docs/install/source-install/compilation/)
-
-> 见链接中：建议以挂载本地 Doris 源码目录的方式运行镜像 .....
-
-由于如果是使用 windows 开发，挂载会存在跨文件系统访问的问题，请自行斟酌设置
-
-```bash
-docker run -it doris:latest /bin/bash
-```
-
-如果选择安装了 zsh
-运行 容器后，在 ~/.zshrc 替换 plugins 为
-
-```
-plugins=(git zsh-autosuggestions zsh-syntax-highlighting)
-```
-
-创建目录并下载 doris
-
-```bash
-su <your user>
-mkdir code && cd code
-git clone https://github.com/apache/doris.git
-```
-
-## 编译
-
-注意:
-
-第一次编译的时候要使用如下命令
-
-```bash
-sh build.sh --clean --be --fe --ui
-```
-
-这是因为 build-env-for-0.15.0 版本镜像升级了 thrift(0.9 -> 0.13)，需要通过 --clean 命令强制使用新版本的 thrift 生成代码文件，否则会出现不兼容的代码。：
-
-编译 Doris
-
-```bash
-sh build.sh
-```
-
-## 运行
-
-手动创建 `meta_dir` 元数据存放位置, 默认值为 `${DORIS_HOME}/doris-meta`
-
-```bash
-mkdir meta_dir
-```
-
-启动FE
-
-```bash
-cd output/fe
-sh bin/start_fe.sh --daemon
-```
-
-启动BE
-
-```bash
-cd output/be
-sh bin/start_be.sh --daemon
-```
-
-mysql-client 连接
-
-```bash
-mysql -h 127.0.0.1 -P 9030 -u root
-```
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/fe-eclipse-dev.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/fe-eclipse-dev.md
deleted file mode 100644
index dc2b155982b..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/fe-eclipse-dev.md
+++ /dev/null
@@ -1,227 +0,0 @@
----
-{
- "title": "FE 开发环境搭建 - Eclipse",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# 使用 Eclipse 搭建 FE 开发环境
-
-## 环境准备
-
-* JDK 1.8+
-* Maven 3.x+
-* Eclipse，并已安装 [M2Eclipse](http://www.eclipse.org/m2e/)
-
-### 代码生成
-
-FE 模块需要部分生成代码，如 Thrift、Protobuf, jflex, cup 等框架的生成代码。这部分需要在 Linux 或者 Mac环境生成。
-
-#### windows下开发获取生成代码步骤
-
-1. 在 Linux 下， 进入 `fe 目录下执行以下命令：
-
- ```
- mvn generate-sources
- ```
-
-2. 如果使用window开发 需要将生成的 `fe/fe-core/target/generated-sources` 目录打包：
-
- `fe/fe-core/target/ && tar czf java.tar.gz generated-sources/`
-
-3. 将 `java.tar.gz` 拷贝到开发环境的 `fe/fe-core/target/` 目录下，并解压
-
- ```
- cp java.tar.gz /path/to/doris/fe/fe-core/target/
- cd /path/to/doris/fe/fe-core/target/ && tar xzf java.tar.gz
- ```
-
-#### mac下开发获取生成代码步骤
-
-mac下可以直接使用maven构建的步骤生成代码, 或者说可以直接编译.
-
-1. 安装thrift的解释器(0.13.0), 如果没有的话到官网下载源码进行编译安装或者直接时
- 使用`brew` 安装一个.
-
-2. 创建一个文件夹`thirdparty/installed/bin`, 然后将thrift 命令建立一个软链到这个
- 路径下(当然你可以copy二进制).
-
- ```
- mkdir -p thirdparty/installed/bin
- ln -s ${thrift_installed_full_path} thirdparty/installed/bin/thrift
- ```
-
-3. 调用maven直接进行构建, 如果出现一些错误请检查`$JAVA_HOME`路径以及java版本以及
- thrift是否能正常正确运行.
-
- ```
- cd fe && mvn package -DskipTests=true -Dos.arch=x86_64
- ```
-
-上述第3步中`-Dos.arch=x86_64` 是为了兼容苹果的m系列处理器(`os.arch=aarch64`),
-protobuf会使用x86_64架构的protoc二进制进行代码生成, 如果是使用m系列处理器的mac,
-有roseta做兼容所以不会有问题.
-
-Note:
-0. cup和jfex均使用java的jar包程序进行编译, 代码生成的流程可以平台无关
-1. protobuf文件使用了现成的开源插件`protoc-jar-maven-plugin`进行跨平台的生成,
- 本质上是下载已经编译好的对应平台二进制, 进行protobuf代码生成.
-2. thrift是目前(2022-06-26-Sun) FE在maven构建上唯一一个依赖
- `thirdparty/installed`的工具. 目前还没有使用类似protobuf的生成插件替换(TODO).
-
-## 导入 FE 工程
-
-### 使用eclipse工程导入
-
-1. 在开发环境的 `fe/` 目录下，执行以下命令生成 Eclipse 工程文件：
-
- `cd /path/to/doris/fe/ && mvn -npr eclipse:eclipse -Dskip.plugin=true`
-
- 执行完成后，会在 `fe/` 目录下生成 `.project` 和 `.classpath` 文件
-
-2. 导入 FE 工程
-
- * 打开 Eclipse，选择 `File -> Import`。
- * 选择 `General -> Existing Projects into Workspace`。
- * `Select root directory` 选择 `fe/` 目录，点击 `Finish` 完成导入。
- * 右击工程，选择 `Build Path -> Configure Build Path`。
- * 在 `Java Build Path` 对话框中，选择 `Source` 标签页，点击 `Add Folder`，勾选添加之前拷贝并解压的 `java/` 目录。
- * 点击 `Apply and Close` 完成。
-
-
-至此，FE 导入完成。Eclipse 中的工程目录大致如下：
-
-
-
-
-### 使用maven工程导入
-
-经过前边mac相关的操作之后, 我们应该能够直接本地maven构建了. 能够maven构建的项目
-是可以使用eclipse m2e 插件直接导入的.
-
-在eclipse File 菜单中依次选择`Import -> Maven -> Existing Maven Projects`
-然后选择doris fe文件夹即可完成导入. 导入时建议选择working set管理FE的多个module.
-
-至此, 我们已经可以使用eclipse进行FE的开发调试.
-
-## 运行单元测试
-
-在想要运行的单元测试文件上右击，选择 `Run As -> JUnit Test`。（如果要单步调试，则选择 `Debug As -> JUnit Test`）。
-
-如果出现以下错误：
-
-```
-java.lang.Exception: Method xxxx should have no parameters
-```
-
-则右击单元测试文件，选择 `Run As -> Run Configurations...`。（如果要单步调试，则选择 `Debug As -> Debug Configurations...`）。
-
-在 `Arguments` 标签页中的 `VM arguments` 中添加：
-
-```
--javaagent:${settings.localRepository}/org/jmockit/jmockit/1.48/jmockit-1.48.jar
-```
-
-其中 `${settings.localRepository}` 要换成 maven lib 库的路径，如：
-
-```
--javaagent:/Users/cmy/.m2/repository/org/jmockit/jmockit/1.48/jmockit-1.48.jar
-```
-
-之后在运行 `Run/Debug` 即可。
-
-FE的单元测试会首先启动一个FE服务，然后由测试用例作为客户端执行相应的测试逻辑。在UT报错时，UT的日志只会打印相应的客户端日志，如果需要**查看服务端日志**，可以在路径`${DORIS_HOME}/fe/mocked`下查看。
-
-## 运行 FE
-
-可以在 Eclipse 中直接启动一个 FE 进程，方便对代码进行调试。
-
-1. 创建一个运行目录：
-
- ```
- mkdir /path/to/doris/fe/run/
- cd /path/to/doris/fe/run/
- mkdir conf/ log/ palo-meta/
- ```
-
-2. 创建配置文件
-
- 在第一步创建的 `conf/` 目录下创建配置文件 `fe.conf`。你可以直接将源码目录下 `conf/fe.conf` 拷贝过来并做简单修改。
-
-3. 在 Eclipse 中找到 `src/main/java/org/apache/doris/PaloFe.java` 文件，右击选择 `Run As -> Run Configurations...`。在 `Environment` 标签页中添加如下环境变量：
-
- * `DORIS_HOME: /path/to/doris/fe/run/`
- * `PID_DIR: /path/to/doris/fe/run/`
- * `LOG_DIR: /path/to/doris/fe/run/log`
-
-4. 右击 `PaloFe.java`，选择 `Run As -> Java Application`，则可以启动 FE。
-
-## 代码更新
-
-### eclipse工程
-
-1. 更新词法、语法文件或者thrift 和proto 文件
-
- 如果修改了 `fe/fe-core/src/main/cup/sql_parser.cup` 或者 `fe/fe-core/src/main/jflex/sql_scanner.flex`文件或者proto 和thrift 文件。则需在 `fe` 目录下执行以下命令：
-
- ```
- mvn generate-sources
- ```
-
- 之后在 Eclipse 中刷新工程即可。
-
-2. 更新 maven 依赖
-
- 如果更新了 `fe/pom.xml` 中的依赖，则需在 `fe/` 目录下执行以下命令：
-
- `mvn -npr eclipse:eclipse -Dskip.plugin=true`
-
- 之后在 Eclipse 中刷新工程即可。如无法更新，建议删除工程，并按照该文档重新导入一遍即可。
-
-### maven工程
-
-1. 更新词法、语法文件或者thrift 和proto 文件 在fe目录下命令行执行一次
- ```
- cd fe && mvn package -DskipTests=true -Dos.arch=x86_64
- ```
-2. 更新maven依赖, 直接在eclipse里`Package Explorer` 右键选中maven项目
- `maven -> update project...`
-
-3. 在eclipse中刷新工程.
-
-## Import 顺序
-
-为了保持 Java 的 Import 顺序，请执行如下操作设定项目的 Import Order
-
-1. 创建文件 `fe_doris.importorder` 并写入以下内容：
-
- ```
- #Organize Import Order
- #Wed Jul 01 16:42:47 CST 2020
- 4=javax
- 3=java
- 2=org
- 1=com
- 0=org.apache.doris
- ```
-
-2. 打开 Eclipse 的偏好设置（Preferences），选择 `Java -> Code Style -> Organize Imports`。点击 `Import` 导入上述文件。
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/fe-idea-dev.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/fe-idea-dev.md
deleted file mode 100644
index b4f032e12ea..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/fe-idea-dev.md
+++ /dev/null
@@ -1,186 +0,0 @@
----
-{
- "title": "FE 开发环境搭建 - IntelliJ IDEA",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# 使用 IntelliJ IDEA 搭建 FE 开发环境
-
-## 1.环境准备
-
-JDK1.8+, IntelliJ IDEA
-
-1. 从 https://github.com/apache/doris.git 下载源码到本地
-
-2. 使用IntelliJ IDEA 打开代码根目录
-
-3. 如果仅进行fe开发而没有编译过thirdparty，则需要安装thrift，并将thrift 复制或者连接到 `thirdparty/installed/bin` 目录下
-
- 安装 `thrift` 0.13.0 版本(注意：`Doris` 0.15 和最新的版本基于 `thrift` 0.13.0 构建, 之前的版本依然使用`thrift` 0.9.3 构建)
-
- Windows:
- 1. 下载：`http://archive.apache.org/dist/thrift/0.13.0/thrift-0.13.0.exe`
- 2. 拷贝：将文件拷贝至 `./thirdparty/installed/bin`
-
- MacOS:
- 1. 下载：`brew install thrift@0.13.0`
- 2. 建立软链接：
- `mkdir -p ./thirdparty/installed/bin`
- # ARM架构macOS
- `ln -s /opt/homebrew/Cellar/thrift@0.13.0/0.13.0/bin/thrift ./thirdparty/installed/bin/thrift`
- # Intel架构macOS
- `ln -s /usr/local/Cellar/thrift@0.13.0/0.13.0/bin/thrift ./thirdparty/installed/bin/thrift`
-
- 注：MacOS执行 `brew install thrift@0.13.0` 可能会报找不到版本的错误，解决方法如下，在终端执行：
- 1. `brew tap-new $USER/local-tap`
- 2. `brew extract --version='0.13.0' thrift $USER/local-tap`
- 3. `brew install thrift@0.13.0`
- 参考链接: `https://gist.github.com/tonydeng/02e571f273d6cce4230dc8d5f394493c`
-
-4. 如果是Mac 或者 Linux 环境 可以通过 如下命令自动生成代码：
-
- ```
- cd fe
- mvn generate-sources
- ```
-
- 如果出现错误，则执行：
-
- ```
- cd fe && mvn clean install -DskipTests
- ```
-
-或者通过图形界面运行 maven 命令生成
-
-
-
-如果使用windows环境可能会有make命令和sh脚本无法执行的情况 可以通过拷贝linux上的 `fe/fe-core/target/generated-sources` 目录拷贝到相应的目录的方式实现，也可以通过docker 镜像挂载本地目录之后，在docker 内部生成自动生成代码，可以参照编译一节
-
-5. 如果还未生成过help文档，需要跳转到docs目录，执行`sh build_help_zip.sh`，
- 然后将build中的help-resource.zip拷贝到fe/fe-core/target/classes中
-
-## 2.调试
-
-1. 用idea导入fe工程
-
-2. 在fe目录下创建下面红框标出的目录（在新版本中该目录可能存在，如存在则跳过，否则创建）
-
-
-
-3. 编译`ui`项目，将 `ui/dist/`目录中的文件拷贝到`webroot`中（如果你不需要看`Doris` UI，这一步可以跳过）
-
-## 3.配置conf/fe.conf
-
-下面是我自己的配置，你可以根据自己的需要进行修改(注意：如果使用`Mac`开发，由于`docker for Mac`不支持`Host`模式，需要使用`-p`方式暴露`be`端口，同时`fe.conf`的`priority_networks`配置为容器内可访问的Ip，例如WIFI的Ip)
-
-```
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-#####################################################################
-## The uppercase properties are read and exported by bin/start_fe.sh.
-## To see all Frontend configurations,
-## see fe/src/org/apache/doris/common/Config.java
-#####################################################################
-
-# the output dir of stderr and stdout
-LOG_DIR = ${DORIS_HOME}/log
-
-DATE = `date +%Y%m%d-%H%M%S`
-JAVA_OPTS="-Xmx2048m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$DATE"
-
-# For jdk 9+, this JAVA_OPTS will be used as default JVM options
-JAVA_OPTS_FOR_JDK_9="-Xmx4096m -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xlog:gc*:$DORIS_HOME/log/fe.gc.log.$DATE:time"
-
-##
-## the lowercase properties are read by main program.
-##
-
-# INFO, WARN, ERROR, FATAL
-sys_log_level = INFO
-
-# store metadata, create it if it is not exist.
-# Default value is ${DORIS_HOME}/doris-meta
-# meta_dir = ${DORIS_HOME}/doris-meta
-
-http_port = 8030
-rpc_port = 9020
-query_port = 9030
-edit_log_port = 9010
-mysql_service_nio_enabled = true
-
-# Choose one if there are more than one ip except loopback address.
-# Note that there should at most one ip match this list.
-# If no ip match this rule, will choose one randomly.
-# use CIDR format, e.g. 10.10.10.0/24
-# Default value is empty.
-# priority_networks = 10.10.10.0/24;192.168.0.0/16
-
-# Advanced configurations
-# log_roll_size_mb = 1024
-# sys_log_dir = ${DORIS_HOME}/log
-# sys_log_roll_num = 10
-# sys_log_verbose_modules =
-# audit_log_dir = ${DORIS_HOME}/log
-# audit_log_modules = slow_query, query
-# audit_log_roll_num = 10
-# meta_delay_toleration_second = 10
-# qe_max_connection = 1024
-# max_conn_per_user = 100
-# qe_query_timeout_second = 300
-# qe_slow_log_ms = 5000
-
-```
-
-
-
-## 4.设置环境变量
-
-在IDEA中设置运行环境变量
-
-
-
-## 5.配置options
-
-由于部分依赖使用了`provided`，idea需要做下特殊配置，在`Run/Debug Configurations`设置中点击右侧`Modify options`，勾选`Add dependencies with "provided" scope to classpath`选项
-
-
-
-## 6.启动fe
-
-下面你就可以愉快的启动，调试你的FE了
-
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/fe-vscode-dev.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/fe-vscode-dev.md
deleted file mode 100644
index 727063747fc..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/fe-vscode-dev.md
+++ /dev/null
@@ -1,80 +0,0 @@
----
-{
- "title": "FE 开发环境搭建 - Visual Studio Code (VSCode)",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# 使用 VSCode 搭建 FE 开发环境
-
-有些开发者是在 开发机/WSL/docker 上搭建 FE 开发环境，但是这样的开发环境不能支持本地开发，有些习惯于使用 VSCode 的开发者可以配置远程开发调试
-
-## 环境准备
-
-* JDK11+ (Java 插件需要 JDK11+) (笔者是在 `home` 目录下建立了一个 lib 目录，分别安装了 [JDK11](https://github.com/adoptium/temurin11-binaries/releases/) 和 JDK8 ，分别用于插件和编译)
-* VSCode
- + Extension Pack for Java 插件
- + Remote 插件
-
-## 下载代码编译
-
-1. 从 https://github.com/apache/doris.git 下载源码到本地
-
-2. 使用 VSCode 打开代码 `/fe` 目录
-
-## 配置 VSCode
-
-在 `.vscode` 目录下创建 `settings.json` 文件, 分别配置
-
-* `"java.configuration.runtimes"`
-* `"java.jdt.ls.java.home"` -- 必须另外配置，指向 JDK11+ 的目录，用于配置 vscode-java 插件
-* `"maven.executable.path"` -- 指向 maven 的目录，用于配置 maven-language-server 插件
-
-example:
-
-```json
-{
- "java.configuration.runtimes": [
- {
- "name": "JavaSE-1.8",
- "path": "/!!!path!!!/jdk-1.8.0_191"
- },
- {
- "name": "JavaSE-11",
- "path": "/!!!path!!!/jdk-11.0.14.1+1",
- "default": true
- },
- ],
- "java.jdt.ls.java.home": "/!!!path!!!/jdk-11.0.14.1+1",
- "maven.executable.path": "/!!!path!!!/maven/bin/mvn"
-}
-```
-
-## 编译
-
-其他文章已经介绍的比较清楚了：
-* [使用 LDB toolchain 编译](/docs/install/source-install/compilation-with-ldb-toolchain)
-* ......
-
-为了进行调试，需要在 fe 启动时，加上调试的参数，比如 `-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005` 。
-
-具体是在 `incubator-doris/output/fe/bin/start_fe.sh` 里 `$JAVA $final_java_opt` 后面加上上面的参数。
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/github-checks.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/github-checks.md
deleted file mode 100644
index 5544d996949..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/github-checks.md
+++ /dev/null
@@ -1,31 +0,0 @@
----
-{
- "title": "Github 准入检查",
- "language": "zh-CN"
-}
-
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Github 准入检查
-
-请参阅：[Github Checks Guidance](https://cwiki.apache.org/confluence/display/DORIS/Github+Checks+Guidance)
-
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/how-to-share-blogs.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/how-to-share-blogs.md
deleted file mode 100644
index fc61dd9480b..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/how-to-share-blogs.md
+++ /dev/null
@@ -1,39 +0,0 @@
----
-{
- "title": "如何分享Blog",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# 如何分享 Blog
-
-Doris 社区欢迎大家分享 Doris 相关的文章。这些文章一经合入，将会出现在 Doris 官网网站上。
-
-文章内容包括但不限于：
-
-* Doris 使用技巧
-* Doris 功能介绍
-* Doris 系统调优
-* Doris 功能原理解读
-* Doris 业务场景实践
-
-具体说明，请前往：https://github.com/apache/doris-website 参阅 [README](https://github.com/apache/doris-website)
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/java-format-code.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/java-format-code.md
deleted file mode 100644
index 6fad1867ec7..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/java-format-code.md
+++ /dev/null
@@ -1,96 +0,0 @@
----
-{
- "title": "Java 代码格式化",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Java 代码格式化
-
-Doris 中 Java 部分代码的格式化工作通常有 IDE 来自动完成。这里仅列举通用格式规则，开发这需要根据格式规则，在不同 IDE 中设置对应的代码风格。
-
-## Import Order
-
-```
-org.apache.doris
-<blank line>
-third party package
-<blank line>
-standard java package
-<blank line>
-```
-
-* 禁止使用 `import *`
-* 禁止使用 `import static`
-
-## 编译时检查
-
-现在，在使用`maven`进行编译时，会默认进行`CheckStyle`检查。此检查会略微降低编译速度。如果想跳过此检查，请使用如下命令进行编译
-```
-mvn clean install -DskipTests -Dcheckstyle.skip
-```
-
-## Checkstyle 插件
-
-现在的 `CI` 之中会有 `formatter-check` 进行代码格式化检测。
-
-### IDEA
-
-如果使用 `IDEA` 进行 Java 开发，请在设置中安装 `Checkstyle-IDEA` 插件。
-
-在 `Tools->Checkstyle` 的 `Configuration File` 里点击 `Use a local Checkstyle file`，选择项目的 `fe/check/checkstyle/checkstyle.xml` 文件。
-
-**注意：** 保证`Checkstyle`的版本在9.3及以上（推荐使用最新版本）。
-
-
-
-**可以使用 `Checkstyle-IDEA` 插件来对代码进行 `Checkstyle` 检测**。
-
-
-
-### VS Code
-
-如果使用 VS Code 进行 Java 开发，请安装 `Checkstyle for Java` 插件，按照[文档](https://code.visualstudio.com/docs/java/java-linting)里的说明和动图进行配置。
-
-## IDEA
-
-### 自动格式化
-
-推荐使用 `IDEA` 的自动格式化功能。
-
-在 `Preferences->Editor->Code Style->Java` 的配置标识点击 `Import Scheme`，点击 `IntelliJ IDEA code style XML`，选择项目的 `build-support/IntelliJ-code-format.xml` 文件。
-
-### Rearrange Code
-
-Checkstyle 会按照 [Class and Interface Declarations](https://www.oracle.com/java/technologies/javase/codeconventions-fileorganization.html#1852) 检测代码的 declarations 顺序。
-
-在导入上面的 `build-support/IntelliJ-code-format.xml` 文件后，使用 `Code/Rearrange Code` 自动完成排序
-
-
-
-## Remove unused header
-
-默认快捷键 **CTRL + ALT + O --->** 仅仅删除未使用的导入。
-
-自动移除并且 Reorder ：
-
-点击 `Preferences->Editor->General->Auto Import->Optimize Imports on the Fly`
\ No newline at end of file
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/minidump.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/minidump.md
deleted file mode 100644
index dd15d8d7e91..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/minidump.md
+++ /dev/null
@@ -1,185 +0,0 @@
----
-{
- "title": "Minidump",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Minidump(removed)
-
-> Minidump 已经被移除，它在实际线上环境中没有用处，反而会引入额外的错误。
-
-Minidump 是微软定义的一种用于程序崩溃后错误上报的文件格式。其中包括了崩溃时的线程信息、寄存器信息、调用栈信息等等，这有助于开发人员快速定位问题。
-
-不同于 [Coredump](https://en.wikipedia.org/wiki/Core_dump)，Minidump 文件体积更小，更易于上报和网络传输。Coredump 文件会包含完整的内存镜像，因此体积可能有几十上百GB。而 Minidump 文件仅包含关键线程的调用栈和寄存器信息，因此体积通常只有 MB 级别。
-
-[Breakpad](https://github.com/google/breakpad) 是一个跨平台的崩溃转储和分析框架和工具集合。用户可以借助 Breakpad 来对 Minidump 文件进行自助分析。也可以收集 Minidump 文件并上报给 Doris 集群运维或开发人员。
-
-## 如何开启 Minidump
-
-Minidump 功能是在 Doris 0.15.0 之后的版本中引入的功能。该功能由 BE 的以下配置文件控制：
-
-* `disable_minidump`
-
- 是否开启 Minidump 功能。默认为 false，即开启。
-
-* `minidump_dir`
-
- Minidump 文件的存储目录。默认为 `${DORIS_HOME}/Minidump/`
-
-* `max_minidump_file_size_mb`
-
- Minidump 文件的大小限制。默认为 200MB。如果大小超过阈值，breakpad 会尝试减少文件中记录的信息，比如线程数量和寄存器数量来介绍 Minidump 的文件大小。但这只是一个期望值，实际文件大小可能比设定的大。
-
-* `max_minidump_file_number`
-
- 最多保留多少个 Minidump 文件。默认为 10，既保留最近的 10 个文件。
-
-## 如何生成 Minidump
-
-Minidump 的生成有两种方式：
-
-1. 程序崩溃
-
- 当程序遇到问题崩溃后，会自动生成 Minidump 文件。此时会在 be.out 中出现如下信息：
-
- ```
- Minidump created at: /doris/be/Minidump/4f8d4fe5-15f8-40a3-843109b3-d49993f3.dmp
- *** Aborted at 1636970042 (unix time) try "date -d @1636970042" if you are using GNU date ***
- PC: @ 0x1b184e4 doris::OlapScanNode::scanner_thread()
- *** SIGSEGV (@0x0) received by PID 71567 (TID 0x7f173a5df700) from PID 0; stack trace: ***
- @ 0x220c992 google::(anonymous namespace)::FailureSignalHandler()
- @ 0x7f174fb5e1d0 (unknown)
- @ 0x1b184e4 doris::OlapScanNode::scanner_thread()
- @ 0x15a19af doris::PriorityThreadPool::work_thread()
- @ 0x21d9107 thread_proxy
- @ 0x7f174fb53f84 start_thread
- @ 0x7f174f943ddf __GI___clone
- @ 0x0 (unknown)
- ```
-
- 其中 `/doris/be/Minidump/4f8d4fe5-15f8-40a3-843109b3-d49993f3.dmp` 为 Minidump 文件。而其后的堆栈是程序崩溃点所在的调用栈信息。
-
-2. 手动触发
-
- 用户可以主动地向 BE 进程发送 SIGUSR1 信号来触发 Minidump。如使用以下命令：
-
- ```
- kill -s SIGUSR1 71567
- ```
-
- 其中 71567 是 BE 的进程id（pid）。之后，会在 be.out 中出现如下信息：
-
- ```
- Receive signal: SIGUSR1
- Minidump created at: /doris/be/Minidump/1af8fe8f-3d5b-40ea-6b76ad8f-0cf6756f.dmp
- ```
-
- 其中 `Receive signal: SIGUSR1` 表示这是用户触发的 Minidump 操作。后面是 Minidump 文件位置。
-
- 用户手动触发的 Minidump 操作不会杀死 BE 进程，并且不会在 be.out 产生错误堆栈。
-
-## 如何分析 Minidump
-
-我们可以使用 breakpad 提供的各类工具来分析 Minidump，从而查看错误原因。
-
-### 获取 breakpad 工具
-
-用户可以自行前往 [Breakpad](https://github.com/google/breakpad) 代码库下载并编译 breakpad。编译方式可以参考 Doris 源码库中的 [thirdparty/vars.sh](https://github.com/apache/incubator-doris/blob/master/thirdparty/vars.sh) 中的 `build_breakpad()` 方法。
-
-也可以在 Doris 提供的 Docker 编译镜像 1.4.2 以上版本中，从镜像容器的 `/var/local/thirdparty/installed/bin` 目录下找到 breakpad 编译产出的各类工具。
-
-### 分析 Minidump
-
-我们可以使用以下两种方式来分析 Minidump 文件。
-
-1. 转储成 coredump 文件
-
- 使用 breakpad 提供的 `minidump-2-core` 工具将 Minidump 文件转储成 coredump 文件：
-
- ```
- ./minidump-2-core /doris/be/Minidump/1af8fe8f-3d5b-40ea-6b76ad8f-0cf6756f.dmp > 1.coredump
- ```
-
- 之后我们可以使用 gdb 工具来分析这个 coredump 文件了：
-
- ```
- gdb lib/doris_be -c 1.coredump
- ```
-
-2. 生成可读调用栈
-
- Minidump 文件中只包含调用栈的地址，我们需要把这些地址对应到实际的函数文件位置。因此，我们首先需要通过 `dump_syms ` 生成 BE 二进制文件的符号表 `doris_be.sym`：
-
- ```
- ./dump_syms ./lib/doris_be > doris_be.sym
- ```
-
- 接下来，我们需要符号表第一行的信息，构建一个对应的符号表目录。
-
- ```
- head -n1 doris_be.sym
- ```
-
- 以上命令会打印 doris_be.sym 的第一行内容如下：
-
- ```
- MODULE Linux x86_64 137706CC745F5EC3EABBF730D4B229370 doris_be
- ```
-
- 之后我们创建一个目录结构：
-
- ```
- mkdir -p ./symbols/doris_be/137706CC745F5EC3EABBF730D4B229370
- ```
-
- 目录路径中的 `doris_be` 和 `137706CC745F5EC3EABBF730D4B229370` 需和 doris_be.sym 文件的第一行内容一致。然后我们将 doris_be.sym 文件移动到该目录中：
-
- ```
- cp doris_be.sym ./symbols/doris_be/137706CC745F5EC3EABBF730D4B229370
- ```
-
- 最后，我们可以使用 `minidump_stackwalk` 来产出可读的调用栈信息：
-
- ```
- minidump_stackwalk 4f8d4fe5-15f8-40a3-843109b3-d49993f3.dmp ./symbols/ > readable.stack
- ```
-
- 其中 `4f8d4fe5-15f8-40a3-843109b3-d49993f3.dmp` 为 minidump 文件。`./symbols/` 为之前创建的包含 doris_be.sym 的目录。`readable.stack` 是将生成的结果重定向到这个文件中。同时，在执行这个命令时，屏幕上也会刷一些程序运行日志，可以不用理会。
-
- 至此，我们就获取了一个可读的线程调用栈文件：readable.stack。其中包含了 BE 程序在写 Minidump 文件时的所有线程的调用栈信息，以及对应的寄存器信息。
-
- 其中 `Crash reason` 说明了程序崩溃的原因。如果是 `DUMP_REQUESTED`，则表示这是一次用户主动触发的 Minidump。
-
- 我们可以通过以下命令过滤掉寄存器信息，从而获取一个比较清晰的调用栈视图：
-
- ```
- grep -v = readable.stack |grep -v "Found by" |vi -
- ```
-
- 其结果比较类似于通过 pstack 命令获取到的线程调用栈信息。
-
-
-
-
-
-
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/regression-testing.md b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/regression-testing.md
deleted file mode 100644
index 984115d7a30..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/regression-testing.md
+++ /dev/null
@@ -1,592 +0,0 @@
----
-{
- "title": "回归测试",
- "language": "zh-CN"
-}
-
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# 回归测试
-
-## 概念
-1. `Suite`: 一个测试用例，目前仅用来指代测试用例文件名
-2. `Group`: 一个测试集，目前仅用于指代测试用例所属的目录
-3. `Action`: 一个封装好的具体测试行为，比如用于执行sql的`sql` Action，用于校验结果的`test` Action，用于导入数据的`streamLoad` Action等
-
-## 测试步骤
-1. 需要预先安装好集群
-2. 修改配置文件`${DORIS_HOME}/regression-test/conf/regression-conf.groovy`，设置jdbc url、用户等配置项
-3. 创建测试用例文件并编写用例
-4. 如果用例文件包含`qt` Action，则需要创建关联的data文件，比如`suites/demo/qt_action.groovy`这个例子，需要用到`data/demo/qt_action.out`这个TSV文件来校验输出是否一致
-5. 运行`${DORIS_HOME}/run-regression-test.sh`测试全部用例,或运行`${DORIS_HOME}/run-regression-test.sh --run <suiteName>` 测试若干用例，更多例子见"启动脚本例子"章节
-
-## 目录结构
-开发时需要关注的重要文件/目录
-1. `run-regression-test.sh`: 启动脚本
-2. `regression-conf.groovy`: 回归测试的默认配置
-3. `data`: 存放输入数据和输出校验数据
-4. `suites`: 存放用例
-
-```
-./${DORIS_HOME}
- |-- **run-regression-test.sh** 回归测试启动脚本
- |-- regression-test
- | |-- plugins 插件目录
- | |-- conf
- | | |-- logback.xml 日志配置文件
- | | |-- **regression-conf.groovy** 默认配置文件
- | |
- | |-- framework 回归测试框架源码
- | |-- **data** 用例的输入输出文件
- | | |-- demo 存放demo的输入输出文件
- | | |-- correctness 存放正确性测试用例的输入输出文件
- | | |-- performance 存放性能测试用例的输入输出文件
- | | |-- utils 存放其他工具的输入输出文件
- | |
- | |-- **suites** 回归测试用例
- | |-- demo 存放测试用例的demo
- | |-- correctness 存放正确性测试用例
- | |-- performance 存放性能测试用例
- | |-- utils 其他工具
- |
- |-- output
- |-- regression-test
- |-- log 回归测试日志
-```
-
-
-## 框架默认配置
-测试时需要实际情况修改jdbc和fe的配置
-```groovy
-
-/* ============ 一般只需要关注下面这部分 ============ */
-// 默认DB,如果未创建,则会尝试创建这个db
-defaultDb = "regression_test"
-
-// Jdbc配置
-jdbcUrl = "jdbc:mysql://127.0.0.1:9030/?"
-jdbcUser = "root"
-jdbcPassword = ""
-
-// fe地址配置, 用于stream load
-feHttpAddress = "127.0.0.1:8030"
-feHttpUser = "root"
-feHttpPassword = ""
-
-/* ============ 一般不需要修改下面的部分 ============ */
-
-// DORIS_HOME变量是通过run-regression-test.sh传入的
-// 即 java -DDORIS_HOME=./
-
-// 设置回归测试用例的目录
-suitePath = "${DORIS_HOME}/regression-test/suites"
-// 设置输入输出数据的目录
-dataPath = "${DORIS_HOME}/regression-test/data"
-// 设置插件的目录
-pluginPath = "${DORIS_HOME}/regression-test/plugins"
-
-// 默认会读所有的组,读多个组可以用半角逗号隔开，如: "demo,performance"
-// 一般不需要在配置文件中修改，而是通过run-regression-test.sh --run -g来动态指定和覆盖
-testGroups = ""
-// 默认会读所有的用例, 同样可以使用run-regression-test.sh --run -s来动态指定和覆盖
-testSuites = ""
-// 默认会加载的用例目录, 可以通过run-regression-test.sh --run -d来动态指定和覆盖
-testDirectories = ""
-
-// 排除这些组的用例，可通过run-regression-test.sh --run -xg来动态指定和覆盖
-excludeGroups = ""
-// 排除这些suite，可通过run-regression-test.sh --run -xs来动态指定和覆盖
-excludeSuites = ""
-// 排除这些目录，可通过run-regression-test.sh --run -xd来动态指定和覆盖
-excludeDirectories = ""
-
-// 其他自定义配置
-customConf1 = "test_custom_conf_value"
-```
-
-## 编写用例的步骤
-1. 进入`${DORIS_HOME}/regression-test`目录
-2. 根据测试的目的来选择用例的目录，正确性测试存在`suites/correctness`，而性能测试存在`suites/performance`
-3. 新建一个groovy用例文件，增加若干`Action`用于测试，Action将在后续章节具体说明
-
-## Action
-Action是一个测试框架默认提供的测试行为，使用DSL来定义。
-
-### sql action
-sql action用于提交sql并获取结果，如果查询失败则会抛出异常
-
-参数如下
-- String sql: 输入的sql字符串
-- `return List<List<Object>>`: 查询结果，如果是DDL/DML，则返回一行一列，唯一的值是updateRowCount
-
-下面的样例代码存放于`${DORIS_HOME}/regression-test/suites/demo/sql_action.groovy`:
-```groovy
-suite("sql_action", "demo") {
- // execute sql and ignore result
- sql "show databases"
-
- // execute sql and get result, outer List denote rows, inner List denote columns in a single row
- List<List<Object>> tables = sql "show tables"
-
- // assertXxx() will invoke junit5's Assertions.assertXxx() dynamically
- assertTrue(tables.size() >= 0) // test rowCount >= 0
-
- // syntax error
- try {
- sql "a b c d e"
- throw new IllegalStateException("Should be syntax error")
- } catch (java.sql.SQLException t) {
- assertTrue(true)
- }
-
- def testTable = "test_sql_action1"
-
- try {
- sql "DROP TABLE IF EXISTS ${testTable}"
-
- // multi-line sql
- def result1 = sql """
- CREATE TABLE IF NOT EXISTS ${testTable} (
- id int
- )
- DISTRIBUTED BY HASH(id) BUCKETS 1
- PROPERTIES (
- "replication_num" = "1"
- )
- """
-
- // DDL/DML return 1 row and 1 column, the only value is update row count
- assertTrue(result1.size() == 1)
- assertTrue(result1[0].size() == 1)
- assertTrue(result1[0][0] == 0, "Create table should update 0 rows")
-
- def result2 = sql "INSERT INTO test_sql_action1 values(1), (2), (3)"
- assertTrue(result2.size() == 1)
- assertTrue(result2[0].size() == 1)
- assertTrue(result2[0][0] == 3, "Insert should update 3 rows")
- } finally {
- /**
- * try_xxx(args) means:
- *
- * try {
- * return xxx(args)
- * } catch (Throwable t) {
- * // do nothing
- * return null
- * }
- */
- try_sql("DROP TABLE IF EXISTS ${testTable}")
-
- // you can see the error sql will not throw exception and return
- try {
- def errorSqlResult = try_sql("a b c d e f g")
- assertTrue(errorSqlResult == null)
- } catch (Throwable t) {
- assertTrue(false, "Never catch exception")
- }
- }
-
- // order_sql(sqlStr) equals to sql(sqlStr, isOrder=true)
- // sort result by string dict
- def list = order_sql """
- select 2
- union all
- select 1
- union all
- select null
- union all
- select 15
- union all
- select 3
- """
-
- assertEquals(null, list[0][0])
- assertEquals(1, list[1][0])
- assertEquals(15, list[2][0])
- assertEquals(2, list[3][0])
- assertEquals(3, list[4][0])
-}
-```
-
-### qt action
-qt action用于提交sql，并使用对应的.out TSV文件来校验结果
-- String sql: 输入sql字符串
-- return void
-
-下面的样例代码存放于`${DORIS_HOME}/regression-test/suites/demo/qt_action.groovy`:
-```groovy
-suite("qt_action", "demo") {
- /**
- * qt_xxx sql equals to quickTest(xxx, sql) witch xxx is tag.
- * the result will be compare to the relate file: ${DORIS_HOME}/regression_test/data/qt_action.out.
- *
- * if you want to generate .out tsv file for real execute result. you can run with -genOut or -forceGenOut option.
- * e.g
- * ${DORIS_HOME}/run-regression-test.sh --run qt_action -genOut
- * ${DORIS_HOME}/run-regression-test.sh --run qt_action -forceGenOut
- */
- qt_select "select 1, 'beijing' union all select 2, 'shanghai'"
-
- qt_select2 "select 2"
-
- // order result by string dict then compare to .out file.
- // order_qt_xxx sql equals to quickTest(xxx, sql, true).
- order_qt_union_all """
- select 2
- union all
- select 1
- union all
- select null
- union all
- select 15
- union all
- select 3
- """
-}
-```
-
-### test action
-test action可以使用更复杂的校验规则来测试，比如验证行数、执行时间、是否抛出异常
-
-可用参数
-- String sql: 输入的sql字符串
-- `List<List<Object>> result`: 提供一个List对象，用于比较真实查询结果与List对象是否相等
-- `Iterator<Object> resultIterator`: 提供一个Iterator对象，用于比较真实查询结果与Iterator是否相等
-- String resultFile: 提供一个文件Uri(可以是本地文件相对路径，或http(s)路径)，用于比较真实查询结果与http响应流是否相等，格式与.out文件格式类似，但没有块头和注释
-- String exception: 校验抛出的异常是否包含某些字符串
-- long rowNum: 验证结果行数
-- long time: 验证执行时间是否小于这个值，单位是毫秒
-- `Closure<List<List<Object>>, Throwable, Long, Long> check`: 自定义回调校验，可传入结果、异常、时间。存在回调函数时，其他校验方式会失效。
-
-下面的样例代码存放于`${DORIS_HOME}/regression-test/suites/demo/test_action.groovy`:
-```groovy
... 3221 lines suppressed ...
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org