You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Qifan Chen (Jira)" <ji...@apache.org> on 2022/10/11 15:58:00 UTC
[jira] [Created] (IMPALA-11649) Null pointer exception seen in org.apache.impala.catalog.ParallelFileMetada in impala-asf-master-core-s3 testtaLoader
Qifan Chen created IMPALA-11649:
-----------------------------------
Summary: Null pointer exception seen in org.apache.impala.catalog.ParallelFileMetada in impala-asf-master-core-s3 testtaLoader
Key: IMPALA-11649
URL: https://issues.apache.org/jira/browse/IMPALA-11649
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Qifan Chen
https://master-03.jenkins.cloudera.com/job/impala-asf-master-core-s3/225/
Failed
generate_junitxml.buildall.create-load-data (from generate_junitxml.buildall.create-load-data)
Failing for the past 1 build (Since #225 )
Took 0 ms.
Error Message
Error in /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/testdata/bin/create-load-data.sh at line 95: -timeout)
SQL
{code:java}
06:06:05 ERROR: INSERT into TABLE functional_kudu.alltypes
06:06:05 SELECT id, bool_col, tinyint_col, smallint_col, int_col, bigint_col, float_col, double_col, date_string_col, string_col,
06:06:05 timestamp_col, year, month
06:06:05 FROM functional.alltypes
{code}
SQL error:
{code:java}
06:06:05 ImpalaBeeswaxException: ImpalaBeeswaxException:
06:06:05 INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
06:06:05 MESSAGE: AnalysisException: Failed to load metadata for table: ‘functional.alltypes'
06:06:05 CAUSED BY: TableLoadingException: Loading file and block metadata for 24 paths for table functional.alltypes: failed to load 2 paths. Check the catalog server log for more details.
{code}
Catalog server log:
{code:java}
Log file created at: 2022/10/09 04:03:47
Running on machine: impala-ec2-centos79-m6i-4xlarge-ondemand-005a.vpc.cloudera.com
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E1009 04:03:47.665833 14343 logging.cc:248] stderr will be logged to this file.
22/10/09 04:03:48 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
22/10/09 04:03:48 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
22/10/09 04:03:48 INFO impl.MetricsSystemImpl: s3a-file-system metrics system started
22/10/09 04:03:49 INFO Configuration.deprecation: No unit for fs.s3a.connection.request.timeout(0) assuming SECONDS
22/10/09 04:03:49 INFO impl.MetricsSystemImpl: Stopping s3a-file-system metrics system...
22/10/09 04:03:49 INFO impl.MetricsSystemImpl: s3a-file-system metrics system stopped.
22/10/09 04:03:49 INFO impl.MetricsSystemImpl: s3a-file-system metrics system shutdown complete.
22/10/09 04:03:49 INFO util.JvmPauseMonitor: Starting JVM pause monitor
E1009 04:06:05.058591 19943 ParallelFileMetadataLoader.java:171] Loading file and block metadata for 24 paths for table functional.alltypes encountered an error loading data for path s3a://impala-test-uswest2-2/test-warehouse/alltypes/year=2010/month=4
Java exception follows:
{code}
{code:java}
java.util.concurrent.ExecutionException: java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.impala.catalog.ParallelFileMetadataLoader.loadInternal(ParallelFileMetadataLoader.java:168)
at org.apache.impala.catalog.ParallelFileMetadataLoader.load(ParallelFileMetadataLoader.java:120)
at org.apache.impala.catalog.HdfsTable.loadFileMetadataForPartitions(HdfsTable.java:781)
at org.apache.impala.catalog.HdfsTable.loadFileMetadataForPartitions(HdfsTable.java:744)
at org.apache.impala.catalog.HdfsTable.loadAllPartitions(HdfsTable.java:719)
at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1268)
at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1162)
at org.apache.impala.catalog.TableLoader.load(TableLoader.java:144)
at org.apache.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:245)
at org.apache.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:242)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.fs.s3a.Listing$ObjectListingIterator.<init>(Listing.java:621)
at org.apache.hadoop.fs.s3a.Listing.createObjectListingIterator(Listing.java:163)
at org.apache.hadoop.fs.s3a.Listing.createFileStatusListingIterator(Listing.java:144)
at org.apache.hadoop.fs.s3a.Listing.getListFilesAssumingDir(Listing.java:212)
at org.apache.hadoop.fs.s3a.S3AFileSystem.innerListFiles(S3AFileSystem.java:4790)
at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$listFiles$37(S3AFileSystem.java:4732)
at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:543)
at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:524)
at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:445)
at org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2363)
at org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2382)
at org.apache.hadoop.fs.s3a.S3AFileSystem.listFiles(S3AFileSystem.java:4731)
at org.apache.impala.common.FileSystemUtil.listFiles(FileSystemUtil.java:765)
at org.apache.impala.common.FileSystemUtil.listStatus(FileSystemUtil.java:740)
at org.apache.impala.catalog.FileMetadataLoader.load(FileMetadataLoader.java:190)
at org.apache.impala.catalog.ParallelFileMetadataLoader.lambda$loadInternal$1(ParallelFileMetadataLoader.java:162)
{code}
... 4 more
E1009 04:06:05.059136 19943 ParallelFileMetadataLoader.java:171] Loading file and block metadata for 24 paths for table functional.alltypes encountered an error loading data for path s3a://impala-test-uswest2-2/test-warehouse/alltypes/year=2009/month=11
Java exception follows:
{code:java}
java.util.concurrent.ExecutionException: java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.impala.catalog.ParallelFileMetadataLoader.loadInternal(ParallelFileMetadataLoader.java:168)
at org.apache.impala.catalog.ParallelFileMetadataLoader.load(ParallelFileMetadataLoader.java:120)
at org.apache.impala.catalog.HdfsTable.loadFileMetadataForPartitions(HdfsTable.java:781)
at org.apache.impala.catalog.HdfsTable.loadFileMetadataForPartitions(HdfsTable.java:744)
at org.apache.impala.catalog.HdfsTable.loadAllPartitions(HdfsTable.java:719)
at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1268)
at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1162)
at org.apache.impala.catalog.TableLoader.load(TableLoader.java:144)
at org.apache.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:245)
at org.apache.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:242)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.fs.s3a.Listing$ObjectListingIterator.<init>(Listing.java:621)
at org.apache.hadoop.fs.s3a.Listing.createObjectListingIterator(Listing.java:163)
at org.apache.hadoop.fs.s3a.Listing.createFileStatusListingIterator(Listing.java:144)
at org.apache.hadoop.fs.s3a.Listing.getListFilesAssumingDir(Listing.java:212)
at org.apache.hadoop.fs.s3a.S3AFileSystem.innerListFiles(S3AFileSystem.java:4790)
at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$listFiles$37(S3AFileSystem.java:4732)
at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:543)
at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:524)
at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:445)
at org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2363)
at org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2382)
at org.apache.hadoop.fs.s3a.S3AFileSystem.listFiles(S3AFileSystem.java:4731)
at org.apache.impala.common.FileSystemUtil.listFiles(FileSystemUtil.java:765)
at org.apache.impala.common.FileSystemUtil.listStatus(FileSystemUtil.java:740)
at org.apache.impala.catalog.FileMetadataLoader.load(FileMetadataLoader.java:190)
at org.apache.impala.catalog.ParallelFileMetadataLoader.lambda$loadInternal$1(ParallelFileMetadataLoader.java:162)
... 4 more
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)