You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@polaris.apache.org by yu...@apache.org on 2024/09/17 23:26:56 UTC
(polaris) branch main updated: Add the doc on how to configure Postgres as the backend database (#296)
This is an automated email from the ASF dual-hosted git repository.
yufei pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/polaris.git
The following commit(s) were added to refs/heads/main by this push:
new f374ab44 Add the doc on how to configure Postgres as the backend database (#296)
f374ab44 is described below
commit f374ab4442fbe0ab405905cd64b0b083f393d0ce
Author: Aihua Xu <ai...@gmail.com>
AuthorDate: Tue Sep 17 16:26:52 2024 -0700
Add the doc on how to configure Postgres as the backend database (#296)
---
docs/configuring-polaris-for-production.md | 4 +-
docs/metastores.md | 101 +++++++++++++++++++++
.../PolarisEclipseLinkMetaStoreSessionImpl.java | 2 +-
polaris-service/build.gradle.kts | 1 -
4 files changed, 104 insertions(+), 4 deletions(-)
diff --git a/docs/configuring-polaris-for-production.md b/docs/configuring-polaris-for-production.md
index 64dc3904..37f68281 100644
--- a/docs/configuring-polaris-for-production.md
+++ b/docs/configuring-polaris-for-production.md
@@ -20,7 +20,7 @@
# Configuring Apache Polaris (Incubating) for Production
-The default `polaris-server.yml` configuration is intended for develoment and testing. When deploying Polaris in production, there are several best practices to keep in mind.
+The default `polaris-server.yml` configuration is intended for development and testing. When deploying Polaris in production, there are several best practices to keep in mind.
## Security
@@ -56,7 +56,7 @@ Be sure to secure your metastore backend since it will be storing credentials an
### Configuring EclipseLink
-To use [EclipseLink](https://eclipse.dev/eclipselink/) for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an [EclipseLink `persistence.xml` file](https://eclipse.dev/eclipselink/documentation/2.5/solutions/testingjpa002.htm). This file, local to the Polaris service, will contain information on what database to use for metastore management and how to connect to it.
+To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to [metastore documentation](./metastores.md) for details.
### Bootstrapping
diff --git a/docs/metastores.md b/docs/metastores.md
new file mode 100644
index 00000000..26e8c14d
--- /dev/null
+++ b/docs/metastores.md
@@ -0,0 +1,101 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+This page documents the general configurations to connect to production database through [EclipseLink](https://eclipse.dev/eclipselink/).
+
+## Polaris Server Configuration
+Configure the `metaStoreManager` section in server configuration `polaris-server.yml` as follows:
+```
+metaStoreManager:
+ type: eclipse-link
+ conf-file: META-INF/persistence.xml
+ persistence-unit: polaris
+```
+`conf-file` points to the EclipseLink configuration file and `persistence-unit` tells which unit from the configuration file to use for database connection.
+
+E.g., `polaris-dev` and `polaris` persistence units are configured in `persistence.xml` to connect to development database and production database respectively. Updating `persistence-unit` from `polaris-dev` to `polaris` switch from the development to the production.
+
+`conf-file` by default points to the embedded resource file `META-INF/persistence.xml` in `polaris-eclipselink` module.
+To specify an external configuration file,
+1) Place `persistence.xml` into a jar file: `jar cvf /tmp/conf.jar persistence.xml`.
+2) Use `conf-file: /tmp/conf.jar!/persistence.xml`.
+
+## EclipseLink Configuration - persistence.xml
+The configuration file `persistence.xml` is used to set up the database connection properties, which can differ depending on the type of database and its configuration.
+
+Check out [default persistence.xml](https://github.com/apache/polaris/blob/main/extension/persistence/eclipselink/src/main/resources/META-INF/persistence.xml) for the complete sample and the following shows the database connection properties against file-based H2 database. Polaris creates and connects to a separate database for each realm. Specifically, `{realm}` placeholder in `jakarta.persistence.jdbc.url` is substituted with the actual realm name, allowing the Polaris server to connec [...]
+
+> Note: some database systems such as Postgres don't create databases automatically. Database admins need to create them manually before running Polaris server.
+```xml
+<persistence-unit name="polaris" transaction-type="RESOURCE_LOCAL">
+ <provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
+ <class>org.apache.polaris.core.persistence.models.ModelEntity</class>
+ <class>org.apache.polaris.core.persistence.models.ModelEntityActive</class>
+ <class>org.apache.polaris.core.persistence.models.ModelEntityChangeTracking</class>
+ <class>org.apache.polaris.core.persistence.models.ModelEntityDropped</class>
+ <class>org.apache.polaris.core.persistence.models.ModelGrantRecord</class>
+ <class>org.apache.polaris.core.persistence.models.ModelPrincipalSecrets</class>
+ <class>org.apache.polaris.core.persistence.models.ModelSequenceId</class>
+ <shared-cache-mode>NONE</shared-cache-mode>
+ <properties>
+ <property name="jakarta.persistence.jdbc.url"
+ value="jdbc:h2:file:tmp/polaris_test/filedb_{realm}"/>
+ <property name="jakarta.persistence.jdbc.user" value="sa"/>
+ <property name="jakarta.persistence.jdbc.password" value=""/>
+ <property name="jakarta.persistence.schema-generation.database.action" value="create"/>
+ </properties>
+</persistence-unit>
+```
+
+Build with H2 dependency and run Polaris service:
+```bash
+polaris> ./gradlew --no-daemon --info -PeclipseLink=true -PeclipseLinkDeps=com.h2database:h2:2.3.232 clean shadowJar
+polaris> java -jar polaris-service/build/libs/polaris-service-*.jar server ./polaris-server.yml
+```
+
+### Postgres
+
+The following shows a sample configuration for Postgres database.
+
+```xml
+<persistence-unit name="polaris" transaction-type="RESOURCE_LOCAL">
+ <provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
+ <class>org.apache.polaris.core.persistence.models.ModelEntity</class>
+ <class>org.apache.polaris.core.persistence.models.ModelEntityActive</class>
+ <class>org.apache.polaris.core.persistence.models.ModelEntityChangeTracking</class>
+ <class>org.apache.polaris.core.persistence.models.ModelEntityDropped</class>
+ <class>org.apache.polaris.core.persistence.models.ModelGrantRecord</class>
+ <class>org.apache.polaris.core.persistence.models.ModelPrincipalSecrets</class>
+ <class>org.apache.polaris.core.persistence.models.ModelSequenceId</class>
+ <shared-cache-mode>NONE</shared-cache-mode>
+ <properties>
+ <property name="jakarta.persistence.jdbc.url"
+ value="jdbc:postgresql://localhost:5432/{realm}"/>
+ <property name="jakarta.persistence.jdbc.user" value="postgres"/>
+ <property name="jakarta.persistence.jdbc.password" value="postgres"/>
+ <property name="jakarta.persistence.schema-generation.database.action" value="create"/>
+ <property name="eclipselink.persistence-context.flush-mode" value="auto"/>
+ </properties>
+</persistence-unit>
+```
+Build with Postgres dependency and run Polaris service:
+```bash
+polaris> ./gradlew --no-daemon --info -PeclipseLink=true -PeclipseLinkDeps=org.postgresql:postgresql:42.7.4 clean shadowJar
+polaris> java -jar polaris-service/build/libs/polaris-service-*.jar server ./polaris-server.yml
+```
\ No newline at end of file
diff --git a/extension/persistence/eclipselink/src/main/java/org/apache/polaris/extension/persistence/impl/eclipselink/PolarisEclipseLinkMetaStoreSessionImpl.java b/extension/persistence/eclipselink/src/main/java/org/apache/polaris/extension/persistence/impl/eclipselink/PolarisEclipseLinkMetaStoreSessionImpl.java
index e7c813a6..75d4530d 100644
--- a/extension/persistence/eclipselink/src/main/java/org/apache/polaris/extension/persistence/impl/eclipselink/PolarisEclipseLinkMetaStoreSessionImpl.java
+++ b/extension/persistence/eclipselink/src/main/java/org/apache/polaris/extension/persistence/impl/eclipselink/PolarisEclipseLinkMetaStoreSessionImpl.java
@@ -147,7 +147,7 @@ public class PolarisEclipseLinkMetaStoreSessionImpl implements PolarisMetaStoreS
// Currently eclipseLink can only support configuration as a resource inside a jar. To support
// external configuration, persistence.xml needs be placed inside a jar and here is to add the
// jar to the classpath.
- // Supported configuration file: META-INFO/persistence.xml, /tmp/conf.jar!/persistence.xml
+ // Supported configuration file: META-INF/persistence.xml, /tmp/conf.jar!/persistence.xml
int splitPosition = confFile.indexOf("!/");
if (splitPosition != -1) {
String jarPrefixPath = confFile.substring(0, splitPosition);
diff --git a/polaris-service/build.gradle.kts b/polaris-service/build.gradle.kts
index 31fd5707..4b0028aa 100644
--- a/polaris-service/build.gradle.kts
+++ b/polaris-service/build.gradle.kts
@@ -111,7 +111,6 @@ dependencies {
testRuntimeOnly("org.junit.platform:junit-platform-launcher")
testRuntimeOnly(project(":polaris-eclipselink"))
- testRuntimeOnly(libs.h2)
}
if (project.properties.get("eclipseLink") == "true") {