You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@submarine.apache.org by pi...@apache.org on 2022/05/18 04:29:58 UTC
[submarine] branch master updated: SUBMARINE-1270. add server-database's test into git hub action and extract the coverage analyze source from all jar with submarine prefix
This is an automated email from the ASF dual-hosted git repository.
pingsutw pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new 673c6b71 SUBMARINE-1270. add server-database's test into git hub action and extract the coverage analyze source from all jar with submarine prefix
673c6b71 is described below
commit 673c6b71012c25f80e4826aba40443cc72198e92
Author: FatalLin <fa...@gmail.com>
AuthorDate: Wed May 18 11:27:26 2022 +0800
SUBMARINE-1270. add server-database's test into git hub action and extract the coverage analyze source from all jar with submarine prefix
### What is this PR for?
The coverage is still not worked since last PR, so I dig the coverage script and try to find more information.
I found out the original script only extract the class form submarine-all* jar, but a huge numbers of module's jar is put under lib directory, so all of them won't be scanned.
In the mean time, I also noticed the github action didn't execute the unit tests of submaine-database, so I add them into flow as well.
### What type of PR is it?
Improvement
### Todos
* [ ] - Task
### What is the Jira issue?
<!-- * Open an issue on Jira https://issues.apache.org/jira/browse/SUBMARINE/
* Put link here, and add [SUBMARINE-*Jira number*] in PR title, eg. `SUBMARINE-23. PR title`
-->
https://issues.apache.org/jira/browse/SUBMARINE-1270
### How should this be tested?
should passed existed tests.
### Screenshots (if appropriate)
### Questions:
* Do the license files need updating?No
* Are there breaking changes for older versions?No
* Does this need new documentation?No
Author: FatalLin <fa...@gmail.com>
Signed-off-by: Kevin <pi...@apache.org>
Closes #958 from FatalLin/SUBMARINE-1270 and squashes the following commits:
d8f3bbae [FatalLin] extract all jar with submarine prefix instead of submarine-all
e96c2693 [FatalLin] add the waiting step for minio starting
32fadb4c [FatalLin] add missing dependency
95aa5a3f [FatalLin] add missing dependency
7d18dab4 [FatalLin] add missing config file for test
33df264d [FatalLin] add missing dependency
c4655cd1 [FatalLin] add submarine-server-database test into github action
31168265 [FatalLin] add surffire plugin in submarine-database module
4ab83129 [FatalLin] Merge branch 'master' of https://github.com/apache/submarine into SUBMARINE-1270
8b60db88 [FatalLin] move configruation to submodule
9e972c00 [FatalLin] Merge branch 'master' of https://github.com/apache/submarine into SUBMARINE-1270
9c8822f9 [FatalLin] move surffire module configuration to submodules
61365574 [FatalLin] remove debugging message
6f1b5739 [FatalLin] move surffire setting to root
ecfb0be9 [FatalLin] update pom
82442046 [FatalLin] rename test class
faf75ff0 [FatalLin] align test framework to junit and renmae test class
f446204c [FatalLin] try
e11b6080 [FatalLin] fix test name
be1ea607 [FatalLin] try
b92f3fc3 [FatalLin] remove unused dependency
625301e8 [FatalLin] updagte config
f22cb542 [FatalLin] update config
d3f63b23 [FatalLin] rename test class
3b07f02d [FatalLin] rename for test
b9ba5026 [FatalLin] update pom
f09e2416 [FatalLin] debug
e6a56a1e [FatalLin] fix
af1f41ab [FatalLin] debug
f88486a6 [FatalLin] execute test
103ef87a [FatalLin] add codes for tracking report issue
---
.github/workflows/master.yml | 35 ++++++++--
dev-support/cicd/coverage.sh | 4 +-
submarine-server/server-database/pom.xml | 21 ++++++
.../src/test/resources/mybatis-config.xml | 78 ++++++++++++++++++++++
4 files changed, 131 insertions(+), 7 deletions(-)
diff --git a/.github/workflows/master.yml b/.github/workflows/master.yml
index 4b81b021..006dde47 100644
--- a/.github/workflows/master.yml
+++ b/.github/workflows/master.yml
@@ -389,11 +389,20 @@ jobs:
MINIO_DEFAULT_BUCKETS: "submarine"
ports:
- 9000:9000
+ # wait until minio is health
+ options: --health-cmd "curl http://localhost:9000/minio/health/live" --health-interval 10s --health-timeout 10s --health-retries 10
+
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 50
- - name: Cache jacoco.exec
+ - name: Cache server-database jacoco.exec
+ uses: actions/cache@v2
+ with:
+ path: |
+ ./submarine-server/server-database/target/jacoco.exec
+ key: ${{ runner.os }}-docker-${{ github.sha }}
+ - name: Cache server-core jacoco.exec
uses: actions/cache@v2
with:
path: |
@@ -416,13 +425,25 @@ jobs:
python3 -V
sudo apt-get install python3-mysql.connector
python3 ./dev-support/database/init-database.py
- - name: Build
+ - name: Build submarine-server-database
+ env:
+ MODULES: "-pl :submarine-server-database"
+ run: |
+ echo ">>> mvn $BUILD_FLAG $MODULES -B"
+ mvn $BUILD_FLAG $MODULES -B
+ - name: Test submarine-server-database
+ env:
+ TEST_MODULES: "-pl :submarine-server-database"
+ run: |
+ echo ">>> mvn $TEST_FLAG $TEST_MODULES -B"
+ mvn $TEST_FLAG $TEST_MODULES -B
+ - name: Build submarine-server-database
env:
MODULES: "-pl :submarine-server-core"
run: |
echo ">>> mvn $BUILD_FLAG $MODULES -B"
mvn $BUILD_FLAG $MODULES -B
- - name: Test
+ - name: Test submarine-server-core
env:
TEST_MODULES: "-pl :submarine-server-core"
run: |
@@ -615,7 +636,13 @@ jobs:
with:
path: ./submarine-client/target/jacoco.exec
key: ${{ runner.os }}-docker-${{ github.sha }}
- - name: Cache submarine-server data
+ - name: Cache submarine-server-database data
+ uses: actions/cache@v2
+ with:
+ path: |
+ ./submarine-server/server-database/target/jacoco.exec
+ key: ${{ runner.os }}-docker-${{ github.sha }}
+ - name: Cache submarine-server-core data
uses: actions/cache@v2
with:
path: |
diff --git a/dev-support/cicd/coverage.sh b/dev-support/cicd/coverage.sh
index 6484dbe8..e9ccb050 100755
--- a/dev-support/cicd/coverage.sh
+++ b/dev-support/cicd/coverage.sh
@@ -45,7 +45,7 @@ rm -rf target/coverage-classes || true
mkdir -p target/coverage-classes
#Unzip all the classes from the last build
-find submarine-dist/target/* -name 'submarine-all*.jar' | \
+find submarine-dist/target/* -name 'submarine*.jar' | \
xargs -n1 unzip -o -q -d target/coverage-classes
#Exclude some classes from the coverage
@@ -54,5 +54,3 @@ find target/coverage-classes/javax/xml/bind -name 'ModuleUtil.class'| xargs rm -
#generate the reports
jacoco report "$REPORT_DIR/jacoco-all.exec" --classfiles target/coverage-classes --html "$REPORT_DIR/all" --xml "$REPORT_DIR/all.xml"
-echo '=====report====='
-cat $REPORT_DIR/all.xml
diff --git a/submarine-server/server-database/pom.xml b/submarine-server/server-database/pom.xml
index 72ed97e3..2c45f4bf 100644
--- a/submarine-server/server-database/pom.xml
+++ b/submarine-server/server-database/pom.xml
@@ -177,6 +177,27 @@
<version>${javax.ws.rsapi.version}</version>
</dependency>
+ <dependency>
+ <groupId>com.github.pagehelper</groupId>
+ <artifactId>pagehelper</artifactId>
+ <version>${pagehelper.version}</version>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>mysql</groupId>
+ <artifactId>mysql-connector-java</artifactId>
+ <version>${mysql-connector-java.version}</version>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>commons-logging</groupId>
+ <artifactId>commons-logging</artifactId>
+ <version>${commons.logging.version}</version>
+ <scope>test</scope>
+ </dependency>
+
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
diff --git a/submarine-server/server-database/src/test/resources/mybatis-config.xml b/submarine-server/server-database/src/test/resources/mybatis-config.xml
new file mode 100755
index 00000000..ecdc40ba
--- /dev/null
+++ b/submarine-server/server-database/src/test/resources/mybatis-config.xml
@@ -0,0 +1,78 @@
+<?xml version='1.0' encoding='UTF-8' ?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+<!DOCTYPE configuration PUBLIC '-//mybatis.org//DTD Config 3.0//EN'
+ 'http://mybatis.org/dtd/mybatis-3-config.dtd'>
+<configuration>
+ <settings>
+ <setting name="cacheEnabled" value="true"/>
+ <setting name="lazyLoadingEnabled" value="false"/>
+ <setting name="aggressiveLazyLoading" value="true"/>
+ <setting name="logImpl" value="STDOUT_LOGGING"/>
+ </settings>
+
+ <typeAliases>
+ <package name="com.github.pagehelper.model"/>
+ </typeAliases>
+
+ <plugins>
+ <plugin interceptor="com.github.pagehelper.PageInterceptor">
+ <property name="helperDialect" value="mysql"/>
+ <property name="offsetAsPageNum" value="true"/>
+ <property name="rowBoundsWithCount" value="true"/>
+ </plugin>
+ </plugins>
+
+ <environments default="development">
+ <environment id="development">
+ <transactionManager type="JDBC"/>
+ <dataSource type="POOLED">
+ <property name="driver" value="${jdbc.driverClassName}"/>
+ <property name="url" value="${jdbc.url}"/>
+ <property name="username" value="${jdbc.username}"/>
+ <property name="password" value="${jdbc.password}"/>
+ <property name="poolPingQuery" value="SELECT NOW()"/>
+ <property name="poolPingEnabled" value="true"/>
+ </dataSource>
+ </environment>
+ </environments>
+
+ <mappers>
+ <mapper resource='org/apache/submarine/database/mappers/SystemMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/SysUserMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/SysDictMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/SysDictItemMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/SysDeptMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/TeamMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/TeamMemberMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/ProjectMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/ProjectFilesMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/JobMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/MetricMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/ParamMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/EnvironmentMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/ExperimentTemplateMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/ExperimentMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/NotebookMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/RegisteredModelMapper.xml'/>
+ <mapper resource='org/apache/submarine/database/mappers/RegisteredModelTagMapper.xml'/>
+ <mapper resource="org/apache/submarine/database/mappers/ModelVersionMapper.xml" />
+ <mapper resource="org/apache/submarine/database/mappers/ModelVersionTagMapper.xml" />
+ </mappers>
+</configuration>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org