You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Bharat Viswanadham (JIRA)" <ji...@apache.org> on 2019/02/16 03:32:00 UTC
[jira] [Created] (HDDS-1121) Key read failure when data is written
parallel in to Ozone
Bharat Viswanadham created HDDS-1121:
----------------------------------------
Summary: Key read failure when data is written parallel in to Ozone
Key: HDDS-1121
URL: https://issues.apache.org/jira/browse/HDDS-1121
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Bharat Viswanadham
Assignee: Bharat Viswanadham
When hive is run with multiple threads for data ingestion to ozone. After ingestion is done, during read we see this below error.
This issue is found during hive testing.
{code:java}
caused by: org.apache.hadoop.ozone.common.OzoneChecksumException: Checksum mismatch at index 0
at org.apache.hadoop.ozone.common.ChecksumData.verifyChecksumDataMatches(ChecksumData.java:143)
at org.apache.hadoop.ozone.common.Checksum.verifyChecksum(Checksum.java:239)
at org.apache.hadoop.ozone.common.Checksum.verifyChecksum(Checksum.java:217)
at org.apache.hadoop.hdds.scm.storage.BlockInputStream.readChunkFromContainer(BlockInputStream.java:227)
at org.apache.hadoop.hdds.scm.storage.BlockInputStream.seek(BlockInputStream.java:259)
at org.apache.hadoop.ozone.client.io.KeyInputStream$ChunkInputStreamEntry.seek(KeyInputStream.java:249)
at org.apache.hadoop.ozone.client.io.KeyInputStream.seek(KeyInputStream.java:180)
at org.apache.hadoop.fs.ozone.OzoneFSInputStream.seek(OzoneFSInputStream.java:62)
at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:82)
at org.apache.hadoop.fs.FSInputStream.readFully(FSInputStream.java:121)
at org.apache.hadoop.fs.FSDataInputStream.readFully(FSDataInputStream.java:111)
at org.apache.orc.impl.ReaderImpl.extractFileTail(ReaderImpl.java:555)
at org.apache.orc.impl.ReaderImpl.<init>(ReaderImpl.java:370)
at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.<init>(ReaderImpl.java:61)
at org.apache.hadoop.hive.ql.io.orc.OrcFile.createReader(OrcFile.java:105)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.populateAndCacheStripeDetails(OrcInputFormat.java:1647)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.callInternal(OrcInputFormat.java:1533)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.access$2700(OrcInputFormat.java:1329)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1513)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1510)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1688)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1510)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1329)
at java.util.concurrent.FutureTask.run(FutureTask.java:266){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org