You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Alokh P <al...@dream11.com> on 2021/04/16 08:29:49 UTC
java.io.StreamCorruptedException: unexpected block data
Hi Community,
Facing this error when trying to query Parquet data using flink SQL Client
Create Table command
CREATE TABLE test(
`username` STRING,
`userid` INT) WITH ('connector' = 'filesystem', 'path' =
'/home/centos/test/0016_part_00.parquet', 'format' = 'parquet' );
Select command :
select * from test limit 10;
Getting the following exception
[ERROR] Could not execute SQL statement.
Reason:org.apache.flink.runtime.client.JobInitializationException:
Could not instantiate JobManager.
at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:494)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.runtime.JobException: Cannot instantiate
the coordinator for operator Source:
TableSourceScan(table=[[default_catalog, default_database, test]],
fields=[username, userid]) -> SinkConversionToTuple2 -> Sink: SQL
Client Stream Collect Sink
at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.<init>(ExecutionJobVertex.java:231)
at org.apache.flink.runtime.executiongraph.ExecutionGraph.attachJobGraph(ExecutionGraph.java:866)
at org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:257)
at org.apache.flink.runtime.scheduler.SchedulerBase.createExecutionGraph(SchedulerBase.java:318)
at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:272)
at org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:245)
at org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:133)
at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:111)
at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:345)
at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:330)
at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:95)
at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:39)
at org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.<init>(JobManagerRunnerImpl.java:162)
at org.apache.flink.runtime.dispatcher.DefaultJobManagerRunnerFactory.createJobManagerRunner(DefaultJobManagerRunnerFactory.java:86)
at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:478)
... 4 more
Caused by: java.io.StreamCorruptedException: unexpected block data
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1586)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2286)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2166)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2068)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1572)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2286)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2166)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2068)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1572)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2286)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2210)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2068)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1572)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2286)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2210)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2068)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1572)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:430)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:615)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:600)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:587)
at org.apache.flink.util.SerializedValue.deserializeValue(SerializedValue.java:62)
at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder.create(OperatorCoordinatorHolder.java:337)
at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.<init>(ExecutionJobVertex.java:225)
... 18 more
java.io.StreamCorruptedException: unexpected block data
Attaching below the parquet file that I am using as well.
Please point out what i'm missing here.
--
Disclaimer: This message and its attachments contain confidential and
legally privileged information. Dream Sports Inc. provides no warranties
with respect to the contents of this communication and disclaims any and
all liability for reliance thereon.
Re: java.io.StreamCorruptedException: unexpected block data
Posted by Jingsong Li <ji...@gmail.com>.
Hi Alokh,
Maybe this is related to https://issues.apache.org/jira/browse/FLINK-20241
We can improve `SerializableConfiguration` to throw better exceptions.
So the true reason may be "ClassNotFoundException"
Can you check your dependencies? Like Hadoop related dependencies?
Best,
Jingsong
On Fri, Apr 16, 2021 at 4:31 PM Alokh P <al...@dream11.com> wrote:
> The flink version is 1.12.1
>
> On Fri, Apr 16, 2021 at 1:59 PM Alokh P <al...@dream11.com> wrote:
>
>> Hi Community,
>> Facing this error when trying to query Parquet data using flink SQL
>> Client
>>
>> Create Table command
>>
>> CREATE TABLE test(
>> `username` STRING,
>> `userid` INT) WITH ('connector' = 'filesystem', 'path' =
>> '/home/centos/test/0016_part_00.parquet', 'format' = 'parquet' );
>>
>> Select command :
>>
>> select * from test limit 10;
>>
>> Getting the following exception
>>
>> [ERROR] Could not execute SQL statement. Reason:org.apache.flink.runtime.client.JobInitializationException:
>> Could not instantiate JobManager.
>>
>> at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:494)
>> at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused by: org.apache.flink.runtime.JobException: Cannot instantiate the coordinator for operator Source: TableSourceScan(table=[[default_catalog, default_database, test]], fields=[username, userid]) -> SinkConversionToTuple2 -> Sink: SQL Client Stream Collect Sink
>> at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.<init>(ExecutionJobVertex.java:231)
>> at org.apache.flink.runtime.executiongraph.ExecutionGraph.attachJobGraph(ExecutionGraph.java:866)
>> at org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:257)
>> at org.apache.flink.runtime.scheduler.SchedulerBase.createExecutionGraph(SchedulerBase.java:318)
>> at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:272)
>> at org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:245)
>> at org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:133)
>> at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:111)
>> at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:345)
>> at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:330)
>> at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:95)
>> at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:39)
>> at org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.<init>(JobManagerRunnerImpl.java:162)
>> at org.apache.flink.runtime.dispatcher.DefaultJobManagerRunnerFactory.createJobManagerRunner(DefaultJobManagerRunnerFactory.java:86)
>> at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:478)
>> ... 4 more
>> Caused by: java.io.StreamCorruptedException: unexpected block data
>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1586)
>> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2286)
>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2166)
>> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2068)
>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1572)
>> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2286)
>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2166)
>> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2068)
>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1572)
>> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2286)
>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2210)
>> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2068)
>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1572)
>> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2286)
>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2210)
>> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2068)
>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1572)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:430)
>> at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:615)
>> at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:600)
>> at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:587)
>> at org.apache.flink.util.SerializedValue.deserializeValue(SerializedValue.java:62)
>> at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder.create(OperatorCoordinatorHolder.java:337)
>> at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.<init>(ExecutionJobVertex.java:225)
>> ... 18 more
>>
>> java.io.StreamCorruptedException: unexpected block data
>>
>> Attaching below the parquet file that I am using as well.
>>
>> Please point out what i'm missing here.
>>
>>
>>
>>
> Disclaimer: This message and its attachments contain confidential and
> legally privileged information. Dream Sports Inc. provides no warranties
> with respect to the contents of this communication and disclaims any and
> all liability for reliance thereon.
--
Best, Jingsong Lee
Re: java.io.StreamCorruptedException: unexpected block data
Posted by Alokh P <al...@dream11.com>.
The flink version is 1.12.1
On Fri, Apr 16, 2021 at 1:59 PM Alokh P <al...@dream11.com> wrote:
> Hi Community,
> Facing this error when trying to query Parquet data using flink SQL Client
>
> Create Table command
>
> CREATE TABLE test(
> `username` STRING,
> `userid` INT) WITH ('connector' = 'filesystem', 'path' =
> '/home/centos/test/0016_part_00.parquet', 'format' = 'parquet' );
>
> Select command :
>
> select * from test limit 10;
>
> Getting the following exception
>
> [ERROR] Could not execute SQL statement. Reason:org.apache.flink.runtime.client.JobInitializationException:
> Could not instantiate JobManager.
>
> at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:494)
> at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.flink.runtime.JobException: Cannot instantiate the coordinator for operator Source: TableSourceScan(table=[[default_catalog, default_database, test]], fields=[username, userid]) -> SinkConversionToTuple2 -> Sink: SQL Client Stream Collect Sink
> at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.<init>(ExecutionJobVertex.java:231)
> at org.apache.flink.runtime.executiongraph.ExecutionGraph.attachJobGraph(ExecutionGraph.java:866)
> at org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:257)
> at org.apache.flink.runtime.scheduler.SchedulerBase.createExecutionGraph(SchedulerBase.java:318)
> at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:272)
> at org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:245)
> at org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:133)
> at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:111)
> at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:345)
> at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:330)
> at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:95)
> at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:39)
> at org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.<init>(JobManagerRunnerImpl.java:162)
> at org.apache.flink.runtime.dispatcher.DefaultJobManagerRunnerFactory.createJobManagerRunner(DefaultJobManagerRunnerFactory.java:86)
> at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:478)
> ... 4 more
> Caused by: java.io.StreamCorruptedException: unexpected block data
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1586)
> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2286)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2166)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2068)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1572)
> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2286)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2166)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2068)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1572)
> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2286)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2210)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2068)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1572)
> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2286)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2210)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2068)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1572)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:430)
> at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:615)
> at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:600)
> at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:587)
> at org.apache.flink.util.SerializedValue.deserializeValue(SerializedValue.java:62)
> at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder.create(OperatorCoordinatorHolder.java:337)
> at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.<init>(ExecutionJobVertex.java:225)
> ... 18 more
>
> java.io.StreamCorruptedException: unexpected block data
>
> Attaching below the parquet file that I am using as well.
>
> Please point out what i'm missing here.
>
>
>
>
--
Disclaimer: This message and its attachments contain confidential and
legally privileged information. Dream Sports Inc. provides no warranties
with respect to the contents of this communication and disclaims any and
all liability for reliance thereon.