You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Adar Dembo (Code Review)" <ge...@cloudera.org> on 2020/01/21 18:16:41 UTC

[kudu-CR] file cache: support alternate open modes

Hello Alexey Serbin, Andrew Wong,

I'd like you to do a code review. Please visit

    http://gerrit.cloudera.org:8080/15081

to review the following change.


Change subject: file cache: support alternate open modes
......................................................................

file cache: support alternate open modes

Previously, the file cache only supported opening existing files; any new
file creation happened out-of-band and then the file reopened via the cache.
If we're going to use the file cache for log index chunks, however, we need
to support CREATE_OR_OPEN style usage, and doing it in the log index itself
is somewhat hairy.

This patch modifies the file cache to support all of the modes defined in
Env::OpenMode. I tried to ensure that cache operations look and feel like a
standard POSIX filesystem, but it's tough to get this right, and I'm sure I
missed some corner cases, especially those involving the CREATE_OR_OPEN*
modes and concurrent deletes. Thankfully none of our use cases (block
managers, log segments, and log index chunks) do that.

Change-Id: Ie167302ef85b8e1a40fbb89a7742e2cbb43bcec3
---
M src/kudu/fs/file_block_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/util/file_cache-stress-test.cc
M src/kudu/util/file_cache-test.cc
M src/kudu/util/file_cache.cc
M src/kudu/util/file_cache.h
6 files changed, 240 insertions(+), 86 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/81/15081/1
-- 
To view, visit http://gerrit.cloudera.org:8080/15081
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie167302ef85b8e1a40fbb89a7742e2cbb43bcec3
Gerrit-Change-Number: 15081
Gerrit-PatchSet: 1
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>

[kudu-CR] file cache: support alternate open modes

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15081 )

Change subject: file cache: support alternate open modes
......................................................................


Patch Set 1:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.h
File src/kudu/util/file_cache.h:

http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.h@125
PS1, Line 125:  Every mode tries to behave as a POSIX filesystem would, but the
             :   // semantics aren't 100% right when using the more "interesting" modes (such
             :   // as CREATE_OR_OPEN_WITH_TRUNCATE) and deleting files concurrently. Beware.
It it worth enumerating examples of how it differs? Also if it's not used at all, would you be against punting on supporting that mode entirely? Especially if it's not used anywhere in our codebase. Eg by DCHECKing unsupported inputs.


http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.cc
File src/kudu/util/file_cache.cc:

http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.cc@340
PS1, Line 340: out
nit: maybe rename the variable to 'existing_file' then so that's a bit more obvious?


http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.cc@360
PS1, Line 360: // TODO(adar): this doesn't need to be a member as it's only used by Init, but
             :   // there's no easy way to plumb it through KuduOnceDynamic.
I wonder if it would help if we were to templatize ReopenFileIfNecessary with the OpenMode? And then call it with <initial_mode> in InitOnce(). Though punting is fine too.



-- 
To view, visit http://gerrit.cloudera.org:8080/15081
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie167302ef85b8e1a40fbb89a7742e2cbb43bcec3
Gerrit-Change-Number: 15081
Gerrit-PatchSet: 1
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 21 Jan 2020 21:42:14 +0000
Gerrit-HasComments: Yes

[kudu-CR] file cache: support alternate open modes

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15081 )

Change subject: file cache: support alternate open modes
......................................................................

file cache: support alternate open modes

Previously, the file cache only supported opening existing files; any new
file creation happened out-of-band and then the file reopened via the cache.
If we're going to use the file cache for log index chunks, however, we need
to support CREATE_OR_OPEN style usage, and doing it in the log index itself
is somewhat hairy.

This patch modifies the file cache to support most of the modes defined in
Env::OpenMode. I tried to ensure that cache operations look and feel like a
standard POSIX filesystem, but it's tough to get this right, and I'm sure I
missed some corner cases. I feel pretty good about our use cases (block
managers, log segments, and log index chunks) though.

Change-Id: Ie167302ef85b8e1a40fbb89a7742e2cbb43bcec3
Reviewed-on: http://gerrit.cloudera.org:8080/15081
Tested-by: Kudu Jenkins
Reviewed-by: Andrew Wong <aw...@cloudera.com>
---
M src/kudu/fs/file_block_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/util/file_cache-stress-test.cc
M src/kudu/util/file_cache-test.cc
M src/kudu/util/file_cache.cc
M src/kudu/util/file_cache.h
6 files changed, 266 insertions(+), 109 deletions(-)

Approvals:
  Kudu Jenkins: Verified
  Andrew Wong: Looks good to me, approved

-- 
To view, visit http://gerrit.cloudera.org:8080/15081
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ie167302ef85b8e1a40fbb89a7742e2cbb43bcec3
Gerrit-Change-Number: 15081
Gerrit-PatchSet: 4
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] file cache: support alternate open modes

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Kudu Jenkins, Andrew Wong, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15081

to look at the new patch set (#2).

Change subject: file cache: support alternate open modes
......................................................................

file cache: support alternate open modes

Previously, the file cache only supported opening existing files; any new
file creation happened out-of-band and then the file reopened via the cache.
If we're going to use the file cache for log index chunks, however, we need
to support CREATE_OR_OPEN style usage, and doing it in the log index itself
is somewhat hairy.

This patch modifies the file cache to support most of the modes defined in
Env::OpenMode. I tried to ensure that cache operations look and feel like a
standard POSIX filesystem, but it's tough to get this right, and I'm sure I
missed some corner cases. I feel pretty good about our use cases (block
managers, log segments, and log index chunks) though.

Change-Id: Ie167302ef85b8e1a40fbb89a7742e2cbb43bcec3
---
M src/kudu/fs/file_block_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/util/file_cache-stress-test.cc
M src/kudu/util/file_cache-test.cc
M src/kudu/util/file_cache.cc
M src/kudu/util/file_cache.h
6 files changed, 267 insertions(+), 109 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/81/15081/2
-- 
To view, visit http://gerrit.cloudera.org:8080/15081
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie167302ef85b8e1a40fbb89a7742e2cbb43bcec3
Gerrit-Change-Number: 15081
Gerrit-PatchSet: 2
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] file cache: support alternate open modes

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15081 )

Change subject: file cache: support alternate open modes
......................................................................


Patch Set 3: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.cc
File src/kudu/util/file_cache.cc:

http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.cc@360
PS1, Line 360: 
             : // A descriptor adhering to the RandomAccessFile interface (i
> Yeah I need to change this; the crappy hack I used was racy.
Thanks

Yeah, not the prettiest code, but I'm also not sure if there's a better way.



-- 
To view, visit http://gerrit.cloudera.org:8080/15081
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie167302ef85b8e1a40fbb89a7742e2cbb43bcec3
Gerrit-Change-Number: 15081
Gerrit-PatchSet: 3
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Wed, 22 Jan 2020 21:49:05 +0000
Gerrit-HasComments: Yes

[kudu-CR] file cache: support alternate open modes

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, Andrew Wong, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15081

to look at the new patch set (#3).

Change subject: file cache: support alternate open modes
......................................................................

file cache: support alternate open modes

Previously, the file cache only supported opening existing files; any new
file creation happened out-of-band and then the file reopened via the cache.
If we're going to use the file cache for log index chunks, however, we need
to support CREATE_OR_OPEN style usage, and doing it in the log index itself
is somewhat hairy.

This patch modifies the file cache to support most of the modes defined in
Env::OpenMode. I tried to ensure that cache operations look and feel like a
standard POSIX filesystem, but it's tough to get this right, and I'm sure I
missed some corner cases. I feel pretty good about our use cases (block
managers, log segments, and log index chunks) though.

Change-Id: Ie167302ef85b8e1a40fbb89a7742e2cbb43bcec3
---
M src/kudu/fs/file_block_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/util/file_cache-stress-test.cc
M src/kudu/util/file_cache-test.cc
M src/kudu/util/file_cache.cc
M src/kudu/util/file_cache.h
6 files changed, 266 insertions(+), 109 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/81/15081/3
-- 
To view, visit http://gerrit.cloudera.org:8080/15081
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie167302ef85b8e1a40fbb89a7742e2cbb43bcec3
Gerrit-Change-Number: 15081
Gerrit-PatchSet: 3
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] file cache: support alternate open modes

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/15081 )

Change subject: file cache: support alternate open modes
......................................................................


Patch Set 1:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/fs/log_block_manager.cc
File src/kudu/fs/log_block_manager.cc:

http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/fs/log_block_manager.cc@805
PS1, Line 805:   } while (PREDICT_FALSE(metadata_status.IsAlreadyPresent() ||
             :                          data_status.IsAlreadyPresent()));
> Is it possible this cycle run infinitely if the file got some sort of fs-le
It could loop forever if for some reason every new data/metadata file we tried to create already existed on the filesystem. That could happen if the ObjectIdGenerator stopped producing random UUIDs (i.e. if there was a bug in it). But I don't think a broken filesystem would yield such behavior.

In any case, this isn't any more or less likely to happen than it was before, right?


http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.h
File src/kudu/util/file_cache.h:

http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.h@125
PS1, Line 125:  Every mode tries to behave as a POSIX filesystem would, but the
             :   // semantics aren't 100% right when using the more "interesting" modes (such
             :   // as CREATE_OR_OPEN_WITH_TRUNCATE) and deleting files concurrently. Beware.
> It it worth enumerating examples of how it differs? Also if it's not used a
Yeah, I guess I can get rid of CREATE_OR_OPEN_WITH_TRUNCATE since we won't be using it.

There's still latent weirdness though. For example, open(CREATE_OR_OPEN) + delete + open(CREATE_OR_OPEN) (without a close() in between). Should we "undelete" the file? If so, we should probably also truncate it, as that's how it'd appear if done by the filesystem.

I'm fairly confident that for the use cases we care about, this will work. But I haven't thought through all of the combinations and how their semantics compare with that of a POSIX filesystem. Meaning, the comment I'm leaving here is intended to warn folks who think this is a general purpose tool with full POSIX filesystem semantics: if you want to use the file cache in some new way, make sure you think through the semantics you need and whether there are gaps.


http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.cc
File src/kudu/util/file_cache.cc:

http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.cc@340
PS1, Line 340: out
> nit: maybe rename the variable to 'existing_file' then so that's a bit more
The templating suggestion you offered makes this "double duty" unnecessary, so I'll leave the name the way it is.


http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.cc@360
PS1, Line 360: // TODO(adar): this doesn't need to be a member as it's only used by Init, but
             :   // there's no easy way to plumb it through KuduOnceDynamic.
> I wonder if it would help if we were to templatize ReopenFileIfNecessary wi
Yeah I need to change this; the crappy hack I used was racy.

Your suggestion generally works, but C++ doesn't allow partial template specialization for functions, so there's some code duplication. Take a look and let me know what you think.


http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.cc@557
PS1, Line 557:     DCHECK(!FindDescriptorUnlocked(file_name, FindMode::DONT_CREATE,
             :                                    &rwf_descs_, &ignored));
> nit: wrap into #ifndef NDEBUG ... #endif as for the OpenFile(..., shared_pt
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/15081
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie167302ef85b8e1a40fbb89a7742e2cbb43bcec3
Gerrit-Change-Number: 15081
Gerrit-PatchSet: 1
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 22 Jan 2020 17:02:48 +0000
Gerrit-HasComments: Yes

[kudu-CR] file cache: support alternate open modes

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15081 )

Change subject: file cache: support alternate open modes
......................................................................


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/fs/log_block_manager.cc
File src/kudu/fs/log_block_manager.cc:

http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/fs/log_block_manager.cc@805
PS1, Line 805:   } while (PREDICT_FALSE(metadata_status.IsAlreadyPresent() ||
             :                          data_status.IsAlreadyPresent()));
Is it possible this cycle run infinitely if the file got some sort of fs-level error or has chflags/etc. set?


http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.cc
File src/kudu/util/file_cache.cc:

http://gerrit.cloudera.org:8080/#/c/15081/1/src/kudu/util/file_cache.cc@557
PS1, Line 557:     DCHECK(!FindDescriptorUnlocked(file_name, FindMode::DONT_CREATE,
             :                                    &rwf_descs_, &ignored));
nit: wrap into #ifndef NDEBUG ... #endif as for the OpenFile(..., shared_ptr<RWFile>) method above?



-- 
To view, visit http://gerrit.cloudera.org:8080/15081
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie167302ef85b8e1a40fbb89a7742e2cbb43bcec3
Gerrit-Change-Number: 15081
Gerrit-PatchSet: 1
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 21 Jan 2020 23:35:22 +0000
Gerrit-HasComments: Yes