You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Feilong He (Jira)" <ji...@apache.org> on 2020/01/08 06:08:00 UTC
[jira] [Updated] (HDFS-15080) Fix the issue in reading persistent
memory cache with an offset
[ https://issues.apache.org/jira/browse/HDFS-15080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Feilong He updated HDFS-15080:
------------------------------
Description:
Some applications can read a segment of pmem cache with an offset specified. The previous implementation for pmem cache read with DirectByteBuffer didn't cover this situation.
Let me explain further. In our test, we used spark SQL to run some TPC-DS workload to read the cache data and hits read exception. This was due to the missed seek offset arg, which is used in spark SQL to read data packet by packet.
was:Some applications can read a segment of pmem cache with an offset specified. The previous implementation for pmem cache read with DirectByteBuffer didn't cover this situation.
> Fix the issue in reading persistent memory cache with an offset
> ---------------------------------------------------------------
>
> Key: HDFS-15080
> URL: https://issues.apache.org/jira/browse/HDFS-15080
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: caching, datanode
> Reporter: Feilong He
> Assignee: Feilong He
> Priority: Major
> Fix For: 3.3.0, 3.1.4, 3.2.2
>
> Attachments: HDFS-15080-000.patch, HDFS-15080-branch-3.1-000.patch, HDFS-15080-branch-3.2-000.patch
>
>
> Some applications can read a segment of pmem cache with an offset specified. The previous implementation for pmem cache read with DirectByteBuffer didn't cover this situation.
> Let me explain further. In our test, we used spark SQL to run some TPC-DS workload to read the cache data and hits read exception. This was due to the missed seek offset arg, which is used in spark SQL to read data packet by packet.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org