You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kudu.apache.org by "Nah, Sammy" <sa...@intel.com> on 2023/02/25 01:20:22 UTC
[design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
Hi all,
I'm working on a new block cache implementation (KUDU-3342<https://issues.apache.org/jira/browse/KUDU-3342>) and I drafted a design doc[1] including motivation and background.
The patch that I am working on is WIP[2] but there is a PoC patch[3] that I've tested locally which shows that the implementation is feasible.
Please review and any feedback would be appreciated.
-sammy
[1] https://docs.google.com/document/d/12zzk7clZpJZtfKoPDL6LB13jRuBSq5LZP2rVfx9T-oc/edit?usp=sharing
[2] https://gerrit.cloudera.org/#/c/18686/
[3] https://gerrit.cloudera.org/#/c/19498/
RE: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
Posted by "Nah, Sammy" <sa...@intel.com>.
All,
Two patches for adding HBW Block cache has been submitted
https://gerrit.cloudera.org/#/c/19726/
https://gerrit.cloudera.org/#/c/19764/
Change 19726 is the first patch that creates a interface NVM and HBW.
Change 19764 implements the HBW cache.
Benchmarks are in progress using this implementation as the baseline.
-sammy
-----Original Message-----
From: Nah, Sammy <sa...@intel.com>
Sent: Friday, April 14, 2023 5:34 AM
To: dev@kudu.apache.org
Subject: RE: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
Wrong link to Gerrit change.
https://gerrit.cloudera.org/#/c/19726/
-sammy
-----Original Message-----
From: Nah, Sammy <sa...@intel.com>
Sent: Thursday, April 13, 2023 6:14 PM
To: dev@kudu.apache.org
Subject: RE: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
All,
First patch for KUDU-3342 has been submitted to https://gerrit.cloudera.org/#/c/19498/
Details commit steps available at https://github.com/apache/kudu/pull/53
-sammy
-----Original Message-----
From: Nah, Sammy <sa...@intel.com>
Sent: Tuesday, March 7, 2023 6:43 PM
To: dev@kudu.apache.org
Subject: RE: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
Alexey,
Benchmarks are in progress which will be followed by a review process before we can share the results. I will give an update when I have the details.
Thanks for your suggestions.
-sammy
-----Original Message-----
From: Alexey Serbin <as...@cloudera.com.INVALID>
Sent: Tuesday, March 7, 2023 6:10 PM
To: dev@kudu.apache.org
Subject: Re: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
Hi Sammy,
Thank you very much for working on this!
I took a quick look at the design draft and the patches.
I left a few comments on the design draft doc.
Implementation-wise, the Option1 approach looks reasonable to me.
There are few options how that might be done, and I guess we are about to converge on details in a few review/comment iterations.
Just curious: did you happen to run any benchmarks against a real hardware with HBW mamemory using the POC patch you posted at https://gerrit.cloudera.org/#/c/19498/ ?
Thanks!
Kind regards,
Alexey
On Fri, Feb 24, 2023 at 5:21 PM Nah, Sammy <sa...@intel.com> wrote:
> Hi all,
>
> I'm working on a new block cache implementation (KUDU-3342<
> https://issues.apache.org/jira/browse/KUDU-3342>) and I drafted a
> design doc[1] including motivation and background.
>
> The patch that I am working on is WIP[2] but there is a PoC patch[3]
> that I've tested locally which shows that the implementation is feasible.
>
> Please review and any feedback would be appreciated.
>
> -sammy
>
> [1]
> https://docs.google.com/document/d/12zzk7clZpJZtfKoPDL6LB13jRuBSq5LZP2
> rVfx9T-oc/edit?usp=sharing [2] https://gerrit.cloudera.org/#/c/18686/
> [3] https://gerrit.cloudera.org/#/c/19498/
>
>
RE: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
Posted by "Nah, Sammy" <sa...@intel.com>.
Wrong link to Gerrit change.
https://gerrit.cloudera.org/#/c/19726/
-sammy
-----Original Message-----
From: Nah, Sammy <sa...@intel.com>
Sent: Thursday, April 13, 2023 6:14 PM
To: dev@kudu.apache.org
Subject: RE: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
All,
First patch for KUDU-3342 has been submitted to https://gerrit.cloudera.org/#/c/19498/
Details commit steps available at https://github.com/apache/kudu/pull/53
-sammy
-----Original Message-----
From: Nah, Sammy <sa...@intel.com>
Sent: Tuesday, March 7, 2023 6:43 PM
To: dev@kudu.apache.org
Subject: RE: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
Alexey,
Benchmarks are in progress which will be followed by a review process before we can share the results. I will give an update when I have the details.
Thanks for your suggestions.
-sammy
-----Original Message-----
From: Alexey Serbin <as...@cloudera.com.INVALID>
Sent: Tuesday, March 7, 2023 6:10 PM
To: dev@kudu.apache.org
Subject: Re: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
Hi Sammy,
Thank you very much for working on this!
I took a quick look at the design draft and the patches.
I left a few comments on the design draft doc.
Implementation-wise, the Option1 approach looks reasonable to me.
There are few options how that might be done, and I guess we are about to converge on details in a few review/comment iterations.
Just curious: did you happen to run any benchmarks against a real hardware with HBW mamemory using the POC patch you posted at https://gerrit.cloudera.org/#/c/19498/ ?
Thanks!
Kind regards,
Alexey
On Fri, Feb 24, 2023 at 5:21 PM Nah, Sammy <sa...@intel.com> wrote:
> Hi all,
>
> I'm working on a new block cache implementation (KUDU-3342<
> https://issues.apache.org/jira/browse/KUDU-3342>) and I drafted a
> design doc[1] including motivation and background.
>
> The patch that I am working on is WIP[2] but there is a PoC patch[3]
> that I've tested locally which shows that the implementation is feasible.
>
> Please review and any feedback would be appreciated.
>
> -sammy
>
> [1]
> https://docs.google.com/document/d/12zzk7clZpJZtfKoPDL6LB13jRuBSq5LZP2
> rVfx9T-oc/edit?usp=sharing [2] https://gerrit.cloudera.org/#/c/18686/
> [3] https://gerrit.cloudera.org/#/c/19498/
>
>
RE: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
Posted by "Nah, Sammy" <sa...@intel.com>.
All,
First patch for KUDU-3342 has been submitted to https://gerrit.cloudera.org/#/c/19498/
Details commit steps available at https://github.com/apache/kudu/pull/53
-sammy
-----Original Message-----
From: Nah, Sammy <sa...@intel.com>
Sent: Tuesday, March 7, 2023 6:43 PM
To: dev@kudu.apache.org
Subject: RE: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
Alexey,
Benchmarks are in progress which will be followed by a review process before we can share the results. I will give an update when I have the details.
Thanks for your suggestions.
-sammy
-----Original Message-----
From: Alexey Serbin <as...@cloudera.com.INVALID>
Sent: Tuesday, March 7, 2023 6:10 PM
To: dev@kudu.apache.org
Subject: Re: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
Hi Sammy,
Thank you very much for working on this!
I took a quick look at the design draft and the patches.
I left a few comments on the design draft doc.
Implementation-wise, the Option1 approach looks reasonable to me.
There are few options how that might be done, and I guess we are about to converge on details in a few review/comment iterations.
Just curious: did you happen to run any benchmarks against a real hardware with HBW mamemory using the POC patch you posted at https://gerrit.cloudera.org/#/c/19498/ ?
Thanks!
Kind regards,
Alexey
On Fri, Feb 24, 2023 at 5:21 PM Nah, Sammy <sa...@intel.com> wrote:
> Hi all,
>
> I'm working on a new block cache implementation (KUDU-3342<
> https://issues.apache.org/jira/browse/KUDU-3342>) and I drafted a
> design doc[1] including motivation and background.
>
> The patch that I am working on is WIP[2] but there is a PoC patch[3]
> that I've tested locally which shows that the implementation is feasible.
>
> Please review and any feedback would be appreciated.
>
> -sammy
>
> [1]
> https://docs.google.com/document/d/12zzk7clZpJZtfKoPDL6LB13jRuBSq5LZP2
> rVfx9T-oc/edit?usp=sharing [2] https://gerrit.cloudera.org/#/c/18686/
> [3] https://gerrit.cloudera.org/#/c/19498/
>
>
RE: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
Posted by "Nah, Sammy" <sa...@intel.com>.
Alexey,
Benchmarks are in progress which will be followed by a review process before we can share the results. I will give an update when I have the details.
Thanks for your suggestions.
-sammy
-----Original Message-----
From: Alexey Serbin <as...@cloudera.com.INVALID>
Sent: Tuesday, March 7, 2023 6:10 PM
To: dev@kudu.apache.org
Subject: Re: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
Hi Sammy,
Thank you very much for working on this!
I took a quick look at the design draft and the patches.
I left a few comments on the design draft doc.
Implementation-wise, the Option1 approach looks reasonable to me.
There are few options how that might be done, and I guess we are about to converge on details in a few review/comment iterations.
Just curious: did you happen to run any benchmarks against a real hardware with HBW mamemory using the POC patch you posted at https://gerrit.cloudera.org/#/c/19498/ ?
Thanks!
Kind regards,
Alexey
On Fri, Feb 24, 2023 at 5:21 PM Nah, Sammy <sa...@intel.com> wrote:
> Hi all,
>
> I'm working on a new block cache implementation (KUDU-3342<
> https://issues.apache.org/jira/browse/KUDU-3342>) and I drafted a
> design doc[1] including motivation and background.
>
> The patch that I am working on is WIP[2] but there is a PoC patch[3]
> that I've tested locally which shows that the implementation is feasible.
>
> Please review and any feedback would be appreciated.
>
> -sammy
>
> [1]
> https://docs.google.com/document/d/12zzk7clZpJZtfKoPDL6LB13jRuBSq5LZP2
> rVfx9T-oc/edit?usp=sharing [2] https://gerrit.cloudera.org/#/c/18686/
> [3] https://gerrit.cloudera.org/#/c/19498/
>
>
Re: [design doc] KUDU-3342: Add an implementation of the block cache on high bandwidth memory
Posted by Alexey Serbin <as...@cloudera.com.INVALID>.
Hi Sammy,
Thank you very much for working on this!
I took a quick look at the design draft and the patches.
I left a few comments on the design draft doc.
Implementation-wise, the Option1 approach looks reasonable to me.
There are few options how that might be done, and I guess we are about
to converge on details in a few review/comment iterations.
Just curious: did you happen to run any benchmarks against a real hardware
with HBW mamemory using the POC patch you posted at
https://gerrit.cloudera.org/#/c/19498/ ?
Thanks!
Kind regards,
Alexey
On Fri, Feb 24, 2023 at 5:21 PM Nah, Sammy <sa...@intel.com> wrote:
> Hi all,
>
> I'm working on a new block cache implementation (KUDU-3342<
> https://issues.apache.org/jira/browse/KUDU-3342>) and I drafted a design
> doc[1] including motivation and background.
>
> The patch that I am working on is WIP[2] but there is a PoC patch[3] that
> I've tested locally which shows that the implementation is feasible.
>
> Please review and any feedback would be appreciated.
>
> -sammy
>
> [1]
> https://docs.google.com/document/d/12zzk7clZpJZtfKoPDL6LB13jRuBSq5LZP2rVfx9T-oc/edit?usp=sharing
> [2] https://gerrit.cloudera.org/#/c/18686/
> [3] https://gerrit.cloudera.org/#/c/19498/
>
>