You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@amoro.apache.org by ji...@apache.org on 2025/01/22 02:24:00 UTC
(amoro) branch master updated: [AMORO-3404] Correct the description about MySQL usage (#3416)
This is an automated email from the ASF dual-hosted git repository.
jinsongzhou pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/amoro.git
The following commit(s) were added to refs/heads/master by this push:
new c69a2363a [AMORO-3404] Correct the description about MySQL usage (#3416)
c69a2363a is described below
commit c69a2363a6263297421ec6bece176b39a69a5c99
Author: Congxian Qiu <qc...@gmail.com>
AuthorDate: Wed Jan 22 10:23:51 2025 +0800
[AMORO-3404] Correct the description about MySQL usage (#3416)
This closes #3404
Co-authored-by: ZhouJinsong <zh...@163.com>
---
docs/admin-guides/deployment-on-kubernetes.md | 16 +++++------
docs/admin-guides/deployment.md | 40 ++++++++-------------------
docs/concepts/catalogs.md | 4 +--
3 files changed, 22 insertions(+), 38 deletions(-)
diff --git a/docs/admin-guides/deployment-on-kubernetes.md b/docs/admin-guides/deployment-on-kubernetes.md
index b5d75ef08..1d7b4b00a 100644
--- a/docs/admin-guides/deployment-on-kubernetes.md
+++ b/docs/admin-guides/deployment-on-kubernetes.md
@@ -156,17 +156,17 @@ ingress:
### Configure the database.
-AMS default is to use Derby database for storage. When the pod is destroyed, the data will also disappear.
-In production environments, we recommend using MySQL as the storage for system data.
+AMS uses embedded [Apache Derby](https://db.apache.org/derby/) as its backend storage by default.
+In production environments, we recommend using a RDBMS(Relational Database Management System) with higher availability guarantees as the storage for system data, you can ref to [Database Configuration](/deployment/#configure-system-database) for more detail.
```yaml
amoroConf:
database:
- type: mysql
- driver: com.mysql.cj.jdbc.Driver
- url: <jdbc-uri>
- username: <mysql-user>
- password: <mysql-password>
+ type: ${your_database_type}
+ driver: ${your_database_driver}
+ url: ${your_jdbc_url}
+ username: ${your_username}
+ password: ${your_password}
```
@@ -291,4 +291,4 @@ Notes:
By default, the `serviceAccount` will be used to create the flink-optimizer.
Therefore, if you need to schedule the flink-optimizer across namespaces,
-please create a `cluster-role` or use your own created `serviceAccount`.
\ No newline at end of file
+please create a `cluster-role` or use your own created `serviceAccount`.
diff --git a/docs/admin-guides/deployment.md b/docs/admin-guides/deployment.md
index 32ebe9fbc..b032c6844 100644
--- a/docs/admin-guides/deployment.md
+++ b/docs/admin-guides/deployment.md
@@ -100,42 +100,26 @@ Make sure the port is not used before configuring it.
### Configure system database
-You can use MySQL/PostgreSQL as the system database instead of the default Derby.
+AMS uses embedded [Apache Derby](https://db.apache.org/derby/) as the backend storage by default, so you can use `Derby` directly without any additional configuration.
-If you would like to use MySQL as the system database, you need to manually download the [MySQL JDBC Connector](https://repo1.maven.org/maven2/com/mysql/mysql-connector-j/8.1.0/mysql-connector-j-8.1.0.jar)
-and move it into the `{AMORO_HOME}/lib/` directory. You can use the following command to complete these operations:
-```shell
-$ cd ${AMORO_HOME}
-$ MYSQL_JDBC_DRIVER_VERSION=8.0.30
-$ wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/${MYSQL_JDBC_DRIVER_VERSION}/mysql-connector-java-${MYSQL_JDBC_DRIVER_VERSION}.jar
-$ mv mysql-connector-java-${MYSQL_JDBC_DRIVER_VERSION}.jar lib
-```
+You can also configure a relational backend storage as you needed.
-Create an empty database in MySQL/PostgreSQL, then AMS will automatically create tables in this MySQL/PostgreSQL database when it first started.
-If you want to create tables on yourself, set `ams.database.auto-create-tables` to false.
+> If you would like to use MySQL as the system database, you need to manually download the [MySQL JDBC Connector](https://repo1.maven.org/maven2/com/mysql/mysql-connector-j/8.1.0/mysql-connector-j-8.1.0.jar)
+and move it into the `${AMORO_HOME}/lib/` directory.
+You need to create an empty database in the RDBMS before to start the server, then AMS will automatically create tables in the database when it first started.
-One thing you need to do is Adding MySQL/PostgreSQL configuration under `config.yaml` of Ams:
+One thing you need to do is adding configuration under `config.yaml` of Ams:
```yaml
-# MySQL
ams:
database:
- type: mysql
- jdbc-driver-class: com.mysql.cj.jdbc.Driver
- url: jdbc:mysql://127.0.0.1:3306/amoro?useUnicode=true&characterEncoding=UTF8&autoReconnect=true&useAffectedRows=true&allowPublicKeyRetrieval=true&useSSL=false
- username: root
- password: root
+ type: ${database_type} # postgres or mysql
+ jdbc-driver-class: ${your_driver_name}
+ url: ${your_jdbc_url}
+ username: ${your_username}
+ password: ${your_password}
auto-create-tables: true
-# PostgreSQL
-#ams:
-# database:
-# type: postgres
-# jdbc-driver-class: org.postgresql.Driver
-# url: jdbc:postgresql://127.0.0.1:5432/amoro
-# auto-create-tables: false
-# username: user
-# password: passwd
```
### Configure high availability
@@ -301,7 +285,7 @@ $ bin/ams.sh stop
### Upgrade system databases
-You can find all the upgrade SQL scripts under `{AMORO_HOME}/conf/mysql/` with name pattern `upgrade-a.b.c-to-x.y.z.sql`.
+You can find all the upgrade SQL scripts under `${AMORO_HOME}/conf/${db_type}/` with name pattern `upgrade-a.b.c-to-x.y.z.sql`.
Execute the upgrade SQL scripts one by one to your system database based on your starting and target versions.
### Replace all libs and plugins
diff --git a/docs/concepts/catalogs.md b/docs/concepts/catalogs.md
index 02a64059b..e579957e5 100644
--- a/docs/concepts/catalogs.md
+++ b/docs/concepts/catalogs.md
@@ -34,7 +34,7 @@ the multi-catalog feature can be used to support SQL across data sources, such a
```SQL
SELECT c.ID, c.NAME, c.AGE, o.AMOUNT
-FROM MYSQL.ONLINE.CUSTOMERS c JOIN HIVE.OFFLINE.ORDERS o
+FROM ${CATALOG_A}.ONLINE.CUSTOMERS c JOIN ${CATALOG_B}.OFFLINE.ORDERS o
ON (c.ID = o.CUSTOMER_ID)
```
@@ -67,4 +67,4 @@ Through metadata center and SQL-based federated computing of the computing engin
such as DataOps and DataFabric
- Automatic catalog detection: In compute engines like Spark and Flink, it is possible to automatically detect the creation and changes of a
catalog, enabling a one-time configuration for permanent scalability.
-
\ No newline at end of file
+