You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudberry.apache.org by "Mulily0513 (via GitHub)" <gi...@apache.org> on 2025/12/24 10:18:24 UTC
[PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Mulily0513 opened a new pull request, #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22
…pectations
- add bench_test and pxf_extension_test in run_tests.sh, plus matrix entries for bench and pxf_extension in CI
- bump surefire heap to 4G to avoid OOM
- update gpupgrade expected outputs to new PXF_HOME paths and JSON formatter error text
- make ProtocolUtils/HiveBaseTest/JdbcHiveTest/OrcWriteTest/ParquetWriteTest more robust to env defaults (protocol, creds, hive JDBC URL)
- keep MultiServerTest running under HDFS with a safe working directory fallback
- set distribution key and INSERT pattern for performance test data load
<!--Thank you for contributing! -->
<!--In case of an existing issue or discussions, please reference it-->
fix #ISSUE_Number
<!--Remove this section if no corresponding issue.-->
---
## Change logs
> Describe your change clearly, including what problem is being solved or what document is being added or updated.
## Contributor's checklist
Here are some reminders before you submit your pull request:
* Make sure that your Pull Request has a clear title and commit message. You can take the [Git commit template](https://github.com/cloudberrydb/cloudberrydb/blob/main/.gitmessage) as a reference.
* Sign the Contributor License Agreement as prompted for your first-time contribution (*One-time setup*).
* Learn the [code contribution](https://cloudberrydb.org/contribute/code) and [doc contribution](https://cloudberrydb.org/contribute/doc) guides for better collaboration.
* List your communications in the [GitHub Issues](https://github.com/cloudberrydb/cloudberrydb-site/issues) or [Discussions](https://github.com/orgs/cloudberrydb/discussions) (if has or needed).
* Feel free to ask for the cloudberrydb team to help review and approve.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "my-ship-it (via GitHub)" <gi...@apache.org>.
my-ship-it commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2652619883
##########
automation/sqlrepo/features/gpupgrade/extension2_1/step_1_before_running_pxf_pre_gpupgrade/expected/query01.ans:
##########
@@ -40,13 +40,13 @@ FROM pg_catalog.pg_extension AS e
INNER JOIN pg_catalog.pg_proc AS p ON (p.oid = d.objid)
WHERE d.deptype = 'e' AND e.extname = 'pxf'
ORDER BY 1;
- proname | prosrc | probin
----------------------+------------------------------+-------------
- pxf_read | pxfprotocol_import | $libdir/pxf
- pxf_validate | pxfprotocol_validate_urls | $libdir/pxf
- pxf_write | pxfprotocol_export | $libdir/pxf
- pxfdelimited_import | pxfdelimited_import | $libdir/pxf
- pxfwritable_export | gpdbwritableformatter_export | $libdir/pxf
- pxfwritable_import | gpdbwritableformatter_import | $libdir/pxf
+ proname | prosrc | probin
+---------------------+------------------------------+----------------------------------
+ pxf_read | pxfprotocol_import | $PXF_HOME/gpextable/pxf
Review Comment:
why we change here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "MisterRaindrop (via GitHub)" <gi...@apache.org>.
MisterRaindrop commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2646651518
##########
automation/src/main/java/org/greenplum/pxf/automation/utils/system/ProtocolUtils.java:
##########
@@ -23,15 +30,20 @@ public static ProtocolEnum getProtocol() {
}
public static String getSecret() {
- return System.getProperty(AWS_SECRET_ACCESS_KEY);
+ String secret = System.getProperty(AWS_SECRET_ACCESS_KEY);
+ return secret != null ? secret : System.getenv(AWS_SECRET_ACCESS_KEY);
}
public static String getAccess() {
- return System.getProperty(AWS_ACCESS_KEY_ID);
+ String access = System.getProperty(AWS_ACCESS_KEY_ID);
+ String result = access != null ? access : System.getenv(AWS_ACCESS_KEY_ID);
+ System.out.println("DEBUG: doInit() method called getAccess " + result);
Review Comment:
this is a debug print?
##########
concourse/singlecluster/conf/gphd-conf.sh:
##########
@@ -4,14 +4,7 @@ if [ -z "${JAVA_HOME:-}" ]; then
if [ -n "${JAVA_HADOOP:-}" ]; then
export JAVA_HOME="${JAVA_HADOOP}"
else
- # Auto-detect Java 8 path for different architectures
- if [ -d "/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)" ]; then
- export JAVA_HOME="/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)"
- elif [ -d "/usr/lib/jvm/java-8-openjdk" ]; then
- export JAVA_HOME="/usr/lib/jvm/java-8-openjdk"
- else
- export JAVA_HOME=$(readlink -f /usr/bin/java | sed 's:/bin/java::')
- fi
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk
Review Comment:
delete Auto-detect Java 8 path for different architectures？
##########
automation/src/test/java/org/greenplum/pxf/automation/features/hive/HiveBaseTest.java:
##########
@@ -371,25 +371,11 @@ void loadDataIntoHive(String fileName, HiveTable tableName) throws Exception {
void loadDataIntoHive(Hdfs hdfs, Hive hive, String fileName, HiveTable tableName) throws Exception {
- String localPath = localDataResourcesFolder + "/hive/" + fileName;
- String hdfsPath = hdfs.getWorkingDirectory() + "/" + fileName;
-
- // Verify local file exists
- java.io.File localFile = new java.io.File(localPath);
- if (!localFile.exists()) {
- throw new RuntimeException("Local file does not exist: " + localFile.getAbsolutePath());
- }
-
// copy data to hdfs
- hdfs.copyFromLocal(localPath, hdfsPath);
-
- // Verify file was copied to HDFS
- if (!hdfs.doesFileExist(hdfsPath)) {
- throw new RuntimeException("File was not copied to HDFS: " + hdfsPath);
- }
-
+ hdfs.copyFromLocal(localDataResourcesFolder + "/hive/" + fileName,
+ hdfs.getWorkingDirectory() + "/" + fileName);
// load to hive table
- hive.loadData(tableName, hdfsPath, false);
+ hive.loadData(tableName, hdfs.getWorkingDirectory() + "/" + fileName, false);
Review Comment:
Sometimes HDFS metadata loading isn't that fast. Would it be better to add hdfs.waitForFile? ensure data load successfully?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "Mulily0513 (via GitHub)" <gi...@apache.org>.
Mulily0513 commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2647431033
##########
.github/workflows/pxf-ci.yml:
##########
@@ -119,6 +119,8 @@ jobs:
- s3
- features
- gpdb
+ - bench
Review Comment:
I can try to further split the load tests and performance tests first and see how long each part takes. Then we can decide the best way to run them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "MisterRaindrop (via GitHub)" <gi...@apache.org>.
MisterRaindrop commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2647368612
##########
.github/workflows/pxf-ci.yml:
##########
@@ -119,6 +119,8 @@ jobs:
- s3
- features
- gpdb
+ - bench
Review Comment:
@tuhaihe bench runs for 5 hours, we need to distinguish between functional testing and performance testing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "Mulily0513 (via GitHub)" <gi...@apache.org>.
Mulily0513 commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2650357187
##########
.github/workflows/pxf-ci.yml:
##########
@@ -119,6 +119,8 @@ jobs:
- s3
- features
- gpdb
+ - bench
Review Comment:
Since the load test takes about 20 minutes and the performance tests take over 2 hours, we could keep just the load test for regular PR validation and move the performance tests to a separate schedule.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "MisterRaindrop (via GitHub)" <gi...@apache.org>.
MisterRaindrop commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2647434978
##########
concourse/singlecluster/conf/gphd-conf.sh:
##########
@@ -4,14 +4,7 @@ if [ -z "${JAVA_HOME:-}" ]; then
if [ -n "${JAVA_HADOOP:-}" ]; then
export JAVA_HOME="${JAVA_HADOOP}"
else
- # Auto-detect Java 8 path for different architectures
- if [ -d "/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)" ]; then
- export JAVA_HOME="/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)"
- elif [ -d "/usr/lib/jvm/java-8-openjdk" ]; then
- export JAVA_HOME="/usr/lib/jvm/java-8-openjdk"
- else
- export JAVA_HOME=$(readlink -f /usr/bin/java | sed 's:/bin/java::')
- fi
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk
Review Comment:
I mean this commit gphd-conf.sh is deleted automatically, set `export JAVA_HOME=/usr/lib/jvm/java-8-openjdk`. I mean it shouldn't be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "my-ship-it (via GitHub)" <gi...@apache.org>.
my-ship-it commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2652621398
##########
automation/src/test/java/org/greenplum/pxf/automation/features/jdbc/JdbcHiveTest.java:
##########
@@ -219,7 +219,7 @@ protected void createTablesForWriteTest(Hive hive, String hiverServerName, Strin
hiveReadable = TableFactory.getPxfJdbcReadableTable(
hiveReadableName, GPDB_WRITE_TYPES_TABLE_FIELDS, targetHiveTable.getFullName(), serverName);
} else {
- String jdbcUrl = String.format("%s%s:10000/default;auth=noSasl", HIVE_JDBC_URL_PREFIX, hive.getHost());
+ String jdbcUrl = String.format("%s%s:10000/default", HIVE_JDBC_URL_PREFIX, hive.getHost());
Review Comment:
Should we keep noSasl for security purpose?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "Mulily0513 (via GitHub)" <gi...@apache.org>.
Mulily0513 commented on PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#issuecomment-3698414660
> Hey @Mulily0513 @ostinru @MisterRaindrop is it ok to merge this PR for now?
There are still some issues with the four test cases in the features group. I'll fix them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "Mulily0513 (via GitHub)" <gi...@apache.org>.
Mulily0513 commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2647428936
##########
automation/src/main/java/org/greenplum/pxf/automation/utils/system/ProtocolUtils.java:
##########
@@ -23,15 +30,20 @@ public static ProtocolEnum getProtocol() {
}
public static String getSecret() {
- return System.getProperty(AWS_SECRET_ACCESS_KEY);
+ String secret = System.getProperty(AWS_SECRET_ACCESS_KEY);
+ return secret != null ? secret : System.getenv(AWS_SECRET_ACCESS_KEY);
}
public static String getAccess() {
- return System.getProperty(AWS_ACCESS_KEY_ID);
+ String access = System.getProperty(AWS_ACCESS_KEY_ID);
+ String result = access != null ? access : System.getenv(AWS_ACCESS_KEY_ID);
+ System.out.println("DEBUG: doInit() method called getAccess " + result);
Review Comment:
Yes, this is a debug print. It should be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "Mulily0513 (via GitHub)" <gi...@apache.org>.
Mulily0513 commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2647618562
##########
automation/src/test/java/org/greenplum/pxf/automation/features/hive/HiveBaseTest.java:
##########
@@ -371,25 +371,11 @@ void loadDataIntoHive(String fileName, HiveTable tableName) throws Exception {
void loadDataIntoHive(Hdfs hdfs, Hive hive, String fileName, HiveTable tableName) throws Exception {
- String localPath = localDataResourcesFolder + "/hive/" + fileName;
- String hdfsPath = hdfs.getWorkingDirectory() + "/" + fileName;
-
- // Verify local file exists
- java.io.File localFile = new java.io.File(localPath);
- if (!localFile.exists()) {
- throw new RuntimeException("Local file does not exist: " + localFile.getAbsolutePath());
- }
-
// copy data to hdfs
- hdfs.copyFromLocal(localPath, hdfsPath);
-
- // Verify file was copied to HDFS
- if (!hdfs.doesFileExist(hdfsPath)) {
- throw new RuntimeException("File was not copied to HDFS: " + hdfsPath);
- }
-
+ hdfs.copyFromLocal(localDataResourcesFolder + "/hive/" + fileName,
+ hdfs.getWorkingDirectory() + "/" + fileName);
// load to hive table
- hive.loadData(tableName, hdfsPath, false);
+ hive.loadData(tableName, hdfs.getWorkingDirectory() + "/" + fileName, false);
Review Comment:
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "Mulily0513 (via GitHub)" <gi...@apache.org>.
Mulily0513 commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2647426774
##########
concourse/singlecluster/conf/gphd-conf.sh:
##########
@@ -4,14 +4,7 @@ if [ -z "${JAVA_HOME:-}" ]; then
if [ -n "${JAVA_HADOOP:-}" ]; then
export JAVA_HOME="${JAVA_HADOOP}"
else
- # Auto-detect Java 8 path for different architectures
- if [ -d "/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)" ]; then
- export JAVA_HOME="/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)"
- elif [ -d "/usr/lib/jvm/java-8-openjdk" ]; then
- export JAVA_HOME="/usr/lib/jvm/java-8-openjdk"
- else
- export JAVA_HOME=$(readlink -f /usr/bin/java | sed 's:/bin/java::')
- fi
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk
Review Comment:
If the /usr/lib/jvm/java-8-openjdk/ directory indeed contains JDK versions for multiple architectures, then the JDK should automatically select the appropriate version based on the architecture.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "tuhaihe (via GitHub)" <gi...@apache.org>.
tuhaihe merged PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "Mulily0513 (via GitHub)" <gi...@apache.org>.
Mulily0513 commented on PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#issuecomment-3698415633
> Hey @Mulily0513 @ostinru @MisterRaindrop is it ok to merge this PR for now?
There are still some issues with the four test cases in the features group. I'll fix them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "Mulily0513 (via GitHub)" <gi...@apache.org>.
Mulily0513 commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2652659347
##########
automation/sqlrepo/features/gpupgrade/extension2_1/step_1_before_running_pxf_pre_gpupgrade/expected/query01.ans:
##########
@@ -40,13 +40,13 @@ FROM pg_catalog.pg_extension AS e
INNER JOIN pg_catalog.pg_proc AS p ON (p.oid = d.objid)
WHERE d.deptype = 'e' AND e.extname = 'pxf'
ORDER BY 1;
- proname | prosrc | probin
----------------------+------------------------------+-------------
- pxf_read | pxfprotocol_import | $libdir/pxf
- pxf_validate | pxfprotocol_validate_urls | $libdir/pxf
- pxf_write | pxfprotocol_export | $libdir/pxf
- pxfdelimited_import | pxfdelimited_import | $libdir/pxf
- pxfwritable_export | gpdbwritableformatter_export | $libdir/pxf
- pxfwritable_import | gpdbwritableformatter_import | $libdir/pxf
+ proname | prosrc | probin
+---------------------+------------------------------+----------------------------------
+ pxf_read | pxfprotocol_import | $PXF_HOME/gpextable/pxf
Review Comment:
Because in our packaged install the PXF extension library is shipped under ${PXF_HOME}/gpextable (and pxf.control module_pathname is rewritten accordingly), not in GPDB $libdir. So the expected output needs to match that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "ostinru (via GitHub)" <gi...@apache.org>.
ostinru commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2647230430
##########
.github/workflows/pxf-ci.yml:
##########
@@ -119,6 +119,8 @@ jobs:
- s3
- features
- gpdb
+ - bench
Review Comment:
`bench` runs for 5 hours. Not sure that we should run it on every PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "tuhaihe (via GitHub)" <gi...@apache.org>.
tuhaihe commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2647422383
##########
.github/workflows/pxf-ci.yml:
##########
@@ -119,6 +119,8 @@ jobs:
- s3
- features
- gpdb
+ - bench
Review Comment:
So we can remove it from this PR? How about it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "Mulily0513 (via GitHub)" <gi...@apache.org>.
Mulily0513 commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2652645704
##########
automation/src/test/java/org/greenplum/pxf/automation/features/jdbc/JdbcHiveTest.java:
##########
@@ -219,7 +219,7 @@ protected void createTablesForWriteTest(Hive hive, String hiverServerName, Strin
hiveReadable = TableFactory.getPxfJdbcReadableTable(
hiveReadableName, GPDB_WRITE_TYPES_TABLE_FIELDS, targetHiveTable.getFullName(), serverName);
} else {
- String jdbcUrl = String.format("%s%s:10000/default;auth=noSasl", HIVE_JDBC_URL_PREFIX, hive.getHost());
+ String jdbcUrl = String.format("%s%s:10000/default", HIVE_JDBC_URL_PREFIX, hive.getHost());
Review Comment:
auth=noSasl forces a no-auth connection mode. Removing it is better since it avoids explicitly disabling authentication.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "Mulily0513 (via GitHub)" <gi...@apache.org>.
Mulily0513 commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2650102266
##########
concourse/singlecluster/conf/gphd-conf.sh:
##########
@@ -4,14 +4,7 @@ if [ -z "${JAVA_HOME:-}" ]; then
if [ -n "${JAVA_HADOOP:-}" ]; then
export JAVA_HOME="${JAVA_HADOOP}"
else
- # Auto-detect Java 8 path for different architectures
- if [ -d "/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)" ]; then
- export JAVA_HOME="/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)"
- elif [ -d "/usr/lib/jvm/java-8-openjdk" ]; then
- export JAVA_HOME="/usr/lib/jvm/java-8-openjdk"
- else
- export JAVA_HOME=$(readlink -f /usr/bin/java | sed 's:/bin/java::')
- fi
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk
Review Comment:
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "tuhaihe (via GitHub)" <gi...@apache.org>.
tuhaihe commented on code in PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#discussion_r2647421969
##########
.github/workflows/pxf-ci.yml:
##########
@@ -119,6 +119,8 @@ jobs:
- s3
- features
- gpdb
+ - bench
Review Comment:
5 hours feels too long for regular PR validation. I don't think it’s necessary to run this on every PR.
It might make more sense to separate performance/benchmark testing from functional testing, and run the benchmark as a scheduled job (for example, weekly) instead of blocking day-to-day PRs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Fix: Add load/performance/extension test groups and align env/test ex… [cloudberry-pxf]
Posted by "tuhaihe (via GitHub)" <gi...@apache.org>.
tuhaihe commented on PR #22:
URL: https://github.com/apache/cloudberry-pxf/pull/22#issuecomment-3698248139
Hey @Mulily0513 @ostinru @MisterRaindrop is it ok to merge this PR for now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org