[jira] [Commented] (IMPALA-14546) TestPostgresJdbcTables.test_postgres_jdbc_tables fails on Ozone due to missing prefix

["Quanlong Huang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-14546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18045099#comment-18045099 ] 

Quanlong Huang commented on IMPALA-14546:
-----------------------------------------

FWIW, the exception is
{noformat}
E20251213 12:14:33.571976 2503334 FileSystemUtil.java:881] b04ce12f4aa82fee:f9f4198c00000000] Failed to copy /test-warehouse/data-sources/jdbc-drivers/postgresql-jdbc.jar to local path: /tmp/b1f3945e-f08b-44e1-85a8-baf1c669ed5a.jar
Java exception follows:
java.io.FileNotFoundException: jdbc-drivers/postgresql-jdbc.jar: No such file or directory!
        at org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.getFileStatusForKeyOrSnapshot(BasicRootedOzoneClientAdapterImpl.java:698)
        at org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.getFileStatus(BasicRootedOzoneClientAdapterImpl.java:672)
        at org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.getFileStatusAdapter(BasicRootedOzoneFileSystem.java:1079)
        at org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.lambda$getFileStatus$11(BasicRootedOzoneFileSystem.java:1063)
        at org.apache.hadoop.hdds.tracing.TracingUtil.executeInSpan(TracingUtil.java:169)
        at org.apache.hadoop.hdds.tracing.TracingUtil.executeInNewSpan(TracingUtil.java:159)
        at org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.getFileStatus(BasicRootedOzoneFileSystem.java:1062)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:393)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:344)
        at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:2556)
        at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:2525)
        at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:2501)
        at org.apache.impala.common.FileSystemUtil.copyToLocal(FileSystemUtil.java:856)
        at org.apache.impala.common.FileSystemUtil.copyFileFromUriToLocal(FileSystemUtil.java:878)
        at org.apache.impala.extdatasource.jdbc.dao.DataSourceObjectCache.get(DataSourceObjectCache.java:113)
        at org.apache.impala.extdatasource.jdbc.dao.GenericJdbcDatabaseAccessor.initializeDatabaseSource(GenericJdbcDatabaseAccessor.java:277)
        at org.apache.impala.extdatasource.jdbc.dao.GenericJdbcDatabaseAccessor.getTotalNumberOfRecords(GenericJdbcDatabaseAccessor.java:81)
        at org.apache.impala.extdatasource.jdbc.JdbcDataSource.prepare(JdbcDataSource.java:137)
        at org.apache.impala.extdatasource.ExternalDataSourceExecutor.prepare(ExternalDataSourceExecutor.java:264)
        at org.apache.impala.planner.DataSourceScanNode.prepareDataSource(DataSourceScanNode.java:230)
        at org.apache.impala.planner.DataSourceScanNode.init(DataSourceScanNode.java:106)
        at org.apache.impala.planner.SingleNodePlanner.createScanNode(SingleNodePlanner.java:1911)
        at org.apache.impala.planner.SingleNodePlanner.createTableRefNode(SingleNodePlanner.java:2243)
        at org.apache.impala.planner.SingleNodePlanner.createTableRefsPlan(SingleNodePlanner.java:938)
        at org.apache.impala.planner.SingleNodePlanner.createSelectPlan(SingleNodePlanner.java:753)
        at org.apache.impala.planner.SingleNodePlanner.createQueryPlan(SingleNodePlanner.java:285)
        at org.apache.impala.planner.SingleNodePlanner.createSingleNodePlan(SingleNodePlanner.java:177)
        at org.apache.impala.planner.Planner.createPlanFragments(Planner.java:137)
        at org.apache.impala.planner.Planner.createPlans(Planner.java:314)
        at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2051)
        at org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:3387)
        at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:3161)
        at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:2560)
        at org.apache.impala.service.Frontend.getTExecRequestWithFallback(Frontend.java:2426)
        at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2113)
        at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:175){noformat}

> TestPostgresJdbcTables.test_postgres_jdbc_tables fails on Ozone due to missing prefix
> -------------------------------------------------------------------------------------
>
>                 Key: IMPALA-14546
>                 URL: https://issues.apache.org/jira/browse/IMPALA-14546
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Test
>    Affects Versions: Impala 5.0.0
>            Reporter: Joe McDonnell
>            Priority: Blocker
>              Labels: broken-build
>
> custom_cluster/test_ext_data_sources.py's TestPostgresJdbcTables.test_postgres_jdbc_tables is consistently failing as it is unable to get the postgres JDBC driver:
> {noformat}
> E   InternalException: Data source DataSource{name=impalajdbcdatasource, location=, className=org.apache.impala.extdatasource.jdbc.JdbcDataSource, apiVersion=V1} returned an error from prepare(): Failed to retrieve total number of records: Failed to copy JDBC driver from '/test-warehouse/data-sources/jdbc-drivers/postgresql-jdbc.jar' to local filesystem.{noformat}
> For tests to work properly on Ozone, they need to use WAREHOUSE_LOCATION_PREFIX or FILESYSTEM_PREFIX when referring to filesystem locations. The code that loads the Postgres JDBC driver into Ozone uses the prefix, but this test doesn't. The test needs to be updated to use the prefix.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org