You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2020/01/17 17:06:24 UTC

[kudu-CR] schema: use dense hash map instead of std::unordered map

Hello Adar Dembo,

I'd like you to do a code review. Please visit

    http://gerrit.cloudera.org:8080/15064

to review the following change.


Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................

schema: use dense_hash_map instead of std::unordered_map

In a time series benchmark I'm working on, the client spent 12% of its
CPU in Schema::FindColumn. In particular, most of the CPU went to the
bucket calculation in std::unordered_map, which required a 'divq'
instruction that can take hundreds of cycles.

This switches Schema to use a dense_hash_map instead which performs
better. After this change, the percent of CPU used by my benchmark
worker thread in Schema::FindColumn dropped from ~12% to ~1.5% which
resulted in a few percent overall throughput increase.

Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
---
M src/kudu/common/schema.cc
M src/kudu/common/schema.h
2 files changed, 9 insertions(+), 5 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/64/15064/1
-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/15064 )

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/15064/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/15064/1//COMMIT_MSG@9
PS1, Line 9: In a time series benchmark I'm working on, the client spent 12% of its
           : CPU in Schema::FindColumn. In particular, most of the CPU went to the
           : bucket calculation in std::unordered_map, which required a 'divq'
           : instruction that can take hundreds of cycles.
> Any chance you can synthesize a microbenchmark that approximates this behav
if it's OK, I'll decline on this -- there are a bunch of microbenchmarks floating around the web for dense_hash_map vs std::unordered_map and everyone agrees it's better


http://gerrit.cloudera.org:8080/#/c/15064/1/src/kudu/common/schema.h
File src/kudu/common/schema.h:

http://gerrit.cloudera.org:8080/#/c/15064/1/src/kudu/common/schema.h@503
PS1, Line 503: :move(cols), ids, k
> If my understanding is correct, expected_max_items_in_table is for the expe
I rejiggered all this stuff in the next revision. Given the dense map is a simple table data structure, we no longer need the fancy allocator track memory usage, so I can use simple move constructors in cases like this instead of having to pick a random "expected number of elements" value as here.



-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 17 Jan 2020 21:05:22 +0000
Gerrit-HasComments: Yes

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15064

to look at the new patch set (#3).

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................

schema: use dense_hash_map instead of std::unordered_map

In a time series benchmark I'm working on, the client spent 12% of its
CPU in Schema::FindColumn. In particular, most of the CPU went to the
bucket calculation in std::unordered_map, which required a 'divq'
instruction that can take hundreds of cycles.

This switches Schema to use a dense_hash_map instead which performs
better. After this change, the percent of CPU used by my benchmark
worker thread in Schema::FindColumn dropped from ~12% to ~1.5% which
resulted in a few percent overall throughput increase.

This also made the fancy allocator which tried to count memory usage
unnecessary, since dense_hash_map is a simple enough data structure that
we can directly compute the memory usage. Now we can also simplify the
constructors since we no longer need to pass an allocator instance.

Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
---
M src/kudu/client/client-test.cc
M src/kudu/common/schema.cc
M src/kudu/common/schema.h
M src/kudu/common/wire_protocol.cc
4 files changed, 55 insertions(+), 83 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/64/15064/3
-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15064

to look at the new patch set (#4).

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................

schema: use dense_hash_map instead of std::unordered_map

In a time series benchmark I'm working on, the client spent 12% of its
CPU in Schema::FindColumn. In particular, most of the CPU went to the
bucket calculation in std::unordered_map, which required a 'divq'
instruction that can take hundreds of cycles.

This switches Schema to use a dense_hash_map instead which performs
better. After this change, the percent of CPU used by my benchmark
worker thread in Schema::FindColumn dropped from ~12% to ~1.5% which
resulted in a few percent overall throughput increase.

This also made the fancy allocator which tried to count memory usage
unnecessary, since dense_hash_map is a simple enough data structure that
we can directly compute the memory usage. Now we can also simplify the
constructors since we no longer need to pass an allocator instance.

Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
---
M src/kudu/client/client-test.cc
M src/kudu/common/schema.cc
M src/kudu/common/schema.h
M src/kudu/common/wire_protocol.cc
M thirdparty/download-thirdparty.sh
A thirdparty/patches/sparsehash-0002-Add-workaround-for-dense_hashtable-move-constructor-.patch
6 files changed, 87 insertions(+), 85 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/64/15064/4
-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/15064 )

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15064/4/thirdparty/patches/sparsehash-0002-Add-workaround-for-dense_hashtable-move-constructor-.patch
File thirdparty/patches/sparsehash-0002-Add-workaround-for-dense_hashtable-move-constructor-.patch:

PS4: 
> Did you also submit this patch upstream?
nope, last time I tried to submit a gcc 4.8 compatibility patch upstream they didn't seem keen on workarounds for gcc 4.8 (https://github.com/sparsehash/sparsehash-c11/pull/19)



-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 21 Jan 2020 22:50:11 +0000
Gerrit-HasComments: Yes

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/15064 )

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................


Patch Set 4: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15064/4/thirdparty/patches/sparsehash-0002-Add-workaround-for-dense_hashtable-move-constructor-.patch
File thirdparty/patches/sparsehash-0002-Add-workaround-for-dense_hashtable-move-constructor-.patch:

PS4: 
Did you also submit this patch upstream?



-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 21 Jan 2020 22:30:57 +0000
Gerrit-HasComments: Yes

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15064

to look at the new patch set (#2).

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................

schema: use dense_hash_map instead of std::unordered_map

In a time series benchmark I'm working on, the client spent 12% of its
CPU in Schema::FindColumn. In particular, most of the CPU went to the
bucket calculation in std::unordered_map, which required a 'divq'
instruction that can take hundreds of cycles.

This switches Schema to use a dense_hash_map instead which performs
better. After this change, the percent of CPU used by my benchmark
worker thread in Schema::FindColumn dropped from ~12% to ~1.5% which
resulted in a few percent overall throughput increase.

This also made the fancy allocator which tried to count memory usage
unnecessary, since dense_hash_map is a simple enough data structure that
we can directly compute the memory usage. Now we can also simplify the
constructors since we no longer need to pass an allocator instance.

Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
---
M src/kudu/common/schema.cc
M src/kudu/common/schema.h
M src/kudu/common/wire_protocol.cc
3 files changed, 42 insertions(+), 81 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/64/15064/2
-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15064

to look at the new patch set (#5).

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................

schema: use dense_hash_map instead of std::unordered_map

In a time series benchmark I'm working on, the client spent 12% of its
CPU in Schema::FindColumn. In particular, most of the CPU went to the
bucket calculation in std::unordered_map, which required a 'divq'
instruction that can take hundreds of cycles.

This switches Schema to use a dense_hash_map instead which performs
better. After this change, the percent of CPU used by my benchmark
worker thread in Schema::FindColumn dropped from ~12% to ~1.5% which
resulted in a few percent overall throughput increase.

This also made the fancy allocator which tried to count memory usage
unnecessary, since dense_hash_map is a simple enough data structure that
we can directly compute the memory usage. Now we can also simplify the
constructors since we no longer need to pass an allocator instance.

Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
---
M src/kudu/client/client-test.cc
M src/kudu/common/schema.cc
M src/kudu/common/schema.h
M src/kudu/common/wire_protocol.cc
M thirdparty/download-thirdparty.sh
A thirdparty/patches/sparsehash-0002-Add-workaround-for-dense_hashtable-move-constructor-.patch
6 files changed, 88 insertions(+), 86 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/64/15064/5
-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15064 )

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................


Patch Set 3: Code-Review+2

(1 comment)

LGTM, maybe Adar has more feedback

http://gerrit.cloudera.org:8080/#/c/15064/1/src/kudu/common/schema.h
File src/kudu/common/schema.h:

http://gerrit.cloudera.org:8080/#/c/15064/1/src/kudu/common/schema.h@503
PS1, Line 503: :move(cols), ids, k
> I rejiggered all this stuff in the next revision. Given the dense map is a 
yup, num_columns() seems to be the best choice here :)



-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 17 Jan 2020 22:04:11 +0000
Gerrit-HasComments: Yes

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/15064 )

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................


Patch Set 3:

Hit a gcc 4.8 compatibility bug. Added a workaround, hopefully will compile (I couldn't get gcc 4.8 to work on my ubuntu 18 box)


-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 21 Jan 2020 21:58:24 +0000
Gerrit-HasComments: No

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15064 )

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................

schema: use dense_hash_map instead of std::unordered_map

In a time series benchmark I'm working on, the client spent 12% of its
CPU in Schema::FindColumn. In particular, most of the CPU went to the
bucket calculation in std::unordered_map, which required a 'divq'
instruction that can take hundreds of cycles.

This switches Schema to use a dense_hash_map instead which performs
better. After this change, the percent of CPU used by my benchmark
worker thread in Schema::FindColumn dropped from ~12% to ~1.5% which
resulted in a few percent overall throughput increase.

This also made the fancy allocator which tried to count memory usage
unnecessary, since dense_hash_map is a simple enough data structure that
we can directly compute the memory usage. Now we can also simplify the
constructors since we no longer need to pass an allocator instance.

Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Reviewed-on: http://gerrit.cloudera.org:8080/15064
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Tested-by: Kudu Jenkins
---
M src/kudu/client/client-test.cc
M src/kudu/common/schema.cc
M src/kudu/common/schema.h
M src/kudu/common/wire_protocol.cc
M thirdparty/download-thirdparty.sh
A thirdparty/patches/sparsehash-0002-Add-workaround-for-dense_hashtable-move-constructor-.patch
6 files changed, 88 insertions(+), 86 deletions(-)

Approvals:
  Adar Dembo: Looks good to me, approved
  Kudu Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 6
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15064 )

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................


Patch Set 1:

> Build Failed
 > 
 > http://jenkins.kudu.apache.org/job/kudu-gerrit/20180/ : FAILURE

This test failure seems legit:

I0117 17:21:26.974736  1801 catalog_manager.cc:2397] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43500:
table { table_name: "client-testtb" } alter_schema_steps { type: RENAME_COLUMN rename_column { old_name: "int_val" new_name: "" } } modify_external_catalogs: true
client-test: /home/jenkins-slave/workspace/kudu-master/2/thirdparty/installed/common/include/sparsehash/internal/densehashtable.h:976: std::pair<iterator, bool> google::dense_hashtable<std::pair<const StringPiece, unsigned long>, StringPiece, GoodFastHash<StringPiece>, google::dense_hash_map<StringPiece, unsigned long, GoodFastHash<StringPiece>, std::equal_to<StringPiece>, STLCountingAllocator<std::pair<const StringPiece, unsigned long>, std::allocator<std::pair<const StringPiece, unsigned long> > > >::SelectKey, google::dense_hash_map<StringPiece, unsigned long, GoodFastHash<StringPiece>, std::equal_to<StringPiece>, STLCountingAllocator<std::pair<const StringPiece, unsigned long>, std::allocator<std::pair<const StringPiece, unsigned long> > > >::SetKey, std::equal_to<StringPiece>, STLCountingAllocator<std::pair<const StringPiece, unsigned long>, std::allocator<std::pair<const StringPiece, unsigned long> > > >::insert_noresize(K &&, Args &&...) [Value = std::pair<const StringPiece, unsigned long>, Key = StringPiece, HashFcn = GoodFastHash<StringPiece>, ExtractKey = google::dense_hash_map<StringPiece, unsigned long, GoodFastHash<StringPiece>, std::equal_to<StringPiece>, STLCountingAllocator<std::pair<const StringPiece, unsigned long>, std::allocator<std::pair<const StringPiece, unsigned long> > > >::SelectKey, SetKey = google::dense_hash_map<StringPiece, unsigned long, GoodFastHash<StringPiece>, std::equal_to<StringPiece>, STLCountingAllocator<std::pair<const StringPiece, unsigned long>, std::allocator<std::pair<const StringPiece, unsigned long> > > >::SetKey, EqualKey = std::equal_to<StringPiece>, Alloc = STLCountingAllocator<std::pair<const StringPiece, unsigned long>, std::allocator<std::pair<const StringPiece, unsigned long> > >, K = const StringPiece &, Args = <const std::pair<const StringPiece, unsigned long> &>]: Assertion `!equals(std::forward<K>(key), key_info.empty_key) && "Inserting the empty key"' failed.


-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 17 Jan 2020 18:14:53 +0000
Gerrit-HasComments: No

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15064 )

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15064/1/src/kudu/common/schema.h
File src/kudu/common/schema.h:

http://gerrit.cloudera.org:8080/#/c/15064/1/src/kudu/common/schema.h@503
PS1, Line 503: /*bucket_count=*/10
If my understanding is correct, expected_max_items_in_table is for the expected_max_items_in_table, not bucket count.  Maybe, set to 1000 or something related to the --max_num_columns flag setting (if the latter is relevant at all)?



-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 17 Jan 2020 18:09:34 +0000
Gerrit-HasComments: Yes

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/15064 )

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15064/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/15064/1//COMMIT_MSG@9
PS1, Line 9: In a time series benchmark I'm working on, the client spent 12% of its
           : CPU in Schema::FindColumn. In particular, most of the CPU went to the
           : bucket calculation in std::unordered_map, which required a 'divq'
           : instruction that can take hundreds of cycles.
Any chance you can synthesize a microbenchmark that approximates this behavior?



-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 17 Jan 2020 20:21:34 +0000
Gerrit-HasComments: Yes

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/15064 )

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15064/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/15064/1//COMMIT_MSG@9
PS1, Line 9: In a time series benchmark I'm working on, the client spent 12% of its
           : CPU in Schema::FindColumn. In particular, most of the CPU went to the
           : bucket calculation in std::unordered_map, which required a 'divq'
           : instruction that can take hundreds of cycles.
> if it's OK, I'll decline on this -- there are a bunch of microbenchmarks fl
Sure. I wasn't referring to that choice so much as an approximation of the particular Kudu workload you're experimenting with that happens to spend so much time in FindColumn.



-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Sat, 18 Jan 2020 00:11:52 +0000
Gerrit-HasComments: Yes

[kudu-CR] schema: use dense hash map instead of std::unordered map

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/15064 )

Change subject: schema: use dense_hash_map instead of std::unordered_map
......................................................................


Patch Set 5: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/15064
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8e8f80229b2dcfad05e204a6f6e50ce7dc3f4c73
Gerrit-Change-Number: 15064
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 21 Jan 2020 23:13:09 +0000
Gerrit-HasComments: No