You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by jinhwan choi <ch...@gmail.com> on 2019/04/15 11:09:40 UTC
[ISSUE] HADOOP3.1 & Erasure encode => BlockMissingException
Hi Drill Team
I upgrade my cluster HADOOP 3.1。
(apache drill is ver1.5)
And, I enabled Hadoop 3.X erasure encode feature on this cluster. (and
made some new parquet file)
But after upgrade, I faced this error.
--------------------------------------------------------------
SYSTEM ERROR: BlockMissingException: Could not obtain block:
BP-124827589-172.23.55.207-1483001524978:blk_-9223372036854744080_1954863
file=/drill/data/log_568803_R_521_2019040510190000.parquet
Please, refer to logs for more information.
[Error Id: 940a3b08-0629-400e-8cfb-fd297e1730c7 on tst-nss-hdn06:31010]
--------------------------------------------------------------
Can anyone please let us know the solution for this or if there are any
alternative.
Regards,
Choi
Re: [ISSUE] HADOOP3.1 & Erasure encode => BlockMissingException
Posted by Kunal Khatua <ku...@apache.org>.
There are multiple possibilities for this.
1. Apache Drill currently is supported for Hadoop 2.7.4 and not yet for Hadoop 3.x.
2. The Parquet file is of a later version. Drill supports Parquet 1.10.0
3. Your Apache Drill version is quoted as being 1.5 . I'm guessing that is a typo and you meant 1.15
From the error message (since it is a SYSTEM error), it is most likely #1. I'm not sure erasure coding can play a role, since the FS should be abstracting the erasure coding and the APIs for reading blocks from the file would (probably) be unchanged between the Hadoop 2.x and 3.x.
~ Kunal
On 4/15/2019 11:45:22 AM, jinhwan choi <ch...@gmail.com> wrote:
Hi Drill Team
I upgrade my cluster HADOOP 3.1。
(apache drill is ver1.5)
And, I enabled Hadoop 3.X erasure encode feature on this cluster. (and
made some new parquet file)
But after upgrade, I faced this error.
--------------------------------------------------------------
SYSTEM ERROR: BlockMissingException: Could not obtain block:
BP-124827589-172.23.55.207-1483001524978:blk_-9223372036854744080_1954863
file=/drill/data/log_568803_R_521_2019040510190000.parquet
Please, refer to logs for more information.
[Error Id: 940a3b08-0629-400e-8cfb-fd297e1730c7 on tst-nss-hdn06:31010]
--------------------------------------------------------------
Can anyone please let us know the solution for this or if there are any
alternative.
Regards,
Choi