You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Tim Armstrong (Code Review)" <ge...@cloudera.org> on 2019/01/24 00:51:02 UTC
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12262
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
IMPALA-7941: part 2/2: use cgroups memory limit
This uses the functionality from part 1 to detect the CGroups memory
limit and use it to set a lower process memory limit if needed.
min(system memory, cgroups memory limit) is used instead of
system memory to determine the memory limit. Behaviour of
processes without a memory limit set via CGroups is unchanged.
The default behaviour of using 80% of the memory limit detected
is still in effect. This seems like an OK default, but may
lead to some amount of wasted memory.
Modify containers to have a default JVM heap size of 2GB and
--mem_limit_includes_jvm, so that the automatically configured
memory limit makes more sense.
start-impala-cluster.py is modified to exercise all of this.
Testing:
Tested a containerised cluster manually on my system, which has
32GB of RAM. Here's the breakdown from the memz/ page showing
the JVM heap and auto-configured memory limit.
Process: Limit=7.31 GB Total=1.94 GB Peak=1.94 GB
JVM: max heap size: Total=1.78 GB
JVM: non-heap committed: Total=35.56 MB
Buffer Pool: Free Buffers: Total=0
Buffer Pool: Clean Pages: Total=0
Buffer Pool: Unused Reservation: Total=0
Control Service Queue: Limit=50.00 MB Total=0 Peak=0
Data Stream Service Queue: Limit=374.27 MB Total=0 Peak=0
Data Stream Manager Early RPCs: Total=0 Peak=0
TCMalloc Overhead: Total=12.20 MB
Untracked Memory: Total=121.31 MB
Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
---
M be/src/runtime/exec-env.cc
M bin/start-impala-cluster.py
M docker/coord_exec/Dockerfile
M docker/coordinator/Dockerfile
M docker/daemon_entrypoint.sh
M docker/executor/Dockerfile
6 files changed, 58 insertions(+), 35 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/12262/1
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 6: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 26 Jan 2019 00:03:13 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 5:
PS4 accidentally included a change I didn't mean to include.
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 25 Jan 2019 23:50:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 1: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 24 Jan 2019 05:09:28 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 2:
(4 comments)
Logic-wise, this looks good. A couple comments about logging and code comments.
http://gerrit.cloudera.org:8080/#/c/12262/2/be/src/runtime/exec-env.cc
File be/src/runtime/exec-env.cc:
http://gerrit.cloudera.org:8080/#/c/12262/2/be/src/runtime/exec-env.cc@233
PS2, Line 233: // Depending on the system configuration, we will have to calculate the process
: // memory limit either based on the available physical memory, or if overcommitting
: // is turned off, we use the memory commit limit from /proc/meminfo (see
: // IMPALA-1690).
Can you update this comment to describe how cgroup mem limit fits in?
http://gerrit.cloudera.org:8080/#/c/12262/2/be/src/runtime/exec-env.cc@358
PS2, Line 358: if (bytes_limit > MemInfo::physical_mem()) {
: LOG(WARNING) << "Memory limit "
: << PrettyPrinter::Print(bytes_limit, TUnit::BYTES)
: << " exceeds physical memory of "
: << PrettyPrinter::Print(MemInfo::physical_mem(), TUnit::BYTES);
: }
If there is a cgroup limit, an equivalent of this logging would be useful (i.e. bytes_limit > cgroup_mem_limit).
http://gerrit.cloudera.org:8080/#/c/12262/2/be/src/runtime/exec-env.cc@364
PS2, Line 364: LOG(INFO) << "Using global memory limit: "
: << PrettyPrinter::Print(bytes_limit, TUnit::BYTES);
We might consider adding some information to indicate how the value was set. i.e.
Using global memory limit: X (physical system memory)
Using global memory limit: X (cgroup memory limit)
Using global memory limit: X (constant mem_limit parameter)
Using global memory limit: X (percentage mem_limit parameter, Y% of physical system memory)
Using global memory limit: X (percentage mem_limit parameter, Y% of cgroup memory limit)
http://gerrit.cloudera.org:8080/#/c/12262/2/docker/daemon_entrypoint.sh
File docker/daemon_entrypoint.sh:
http://gerrit.cloudera.org:8080/#/c/12262/2/docker/daemon_entrypoint.sh@45
PS2, Line 45: export JAVA_TOOL_OPTIONS="-Xmx2g $JAVA_TOOL_OPTIONS"
You might find
https://github.com/apache/impala/blob/master/docker/entrypoint.sh#L350-L368
to be interesting. We might want to set the JVM size for the normal minicluster, but that shouldn't block this change.
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 24 Jan 2019 19:13:43 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/1872/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 24 Jan 2019 01:56:54 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 3:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/12262/3/be/src/runtime/exec-env.cc
File be/src/runtime/exec-env.cc:
http://gerrit.cloudera.org:8080/#/c/12262/3/be/src/runtime/exec-env.cc@357
PS3, Line 357: ot
> nit: to
Done
http://gerrit.cloudera.org:8080/#/c/12262/3/be/src/runtime/exec-env.cc@405
PS3, Line 405: --mem_limit="
: << PrettyPrinter::PrintBytes(*bytes_limit);
> this log line can be confusing to mean that the --mem_limit was set to byte
I changed it to be a bit more verbose - print the bytes limit and then the literal --mem_limit string in parentheses to explain where it came from.
I0125 15:48:19.728356 31957 exec-env.cc:405] Using process memory limit: 7.31 GB (--mem_limit=7848997273)
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 25 Jan 2019 23:49:27 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/1893/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 26 Jan 2019 00:44:19 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Bikramjeet Vig (Code Review)" <ge...@cloudera.org>.
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 5: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 25 Jan 2019 23:52:36 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Joe McDonnell, Bikramjeet Vig, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12262
to look at the new patch set (#5).
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
IMPALA-7941: part 2/2: use cgroups memory limit
This uses the functionality from part 1 to detect the CGroups memory
limit and use it to set a lower process memory limit if needed.
min(system memory, cgroups memory limit) is used instead of
system memory to determine the memory limit. Behaviour of
processes without a memory limit set via CGroups is unchanged.
The default behaviour of using 80% of the memory limit detected
is still in effect. This seems like an OK default, but may
lead to some amount of wasted memory.
Modify containers to have a default JVM heap size of 2GB and
--mem_limit_includes_jvm, so that the automatically configured
memory limit makes more sense.
start-impala-cluster.py is modified to exercise all of this.
Testing:
Tested a containerised cluster manually on my system, which has
32GB of RAM. Here's the breakdown from the memz/ page showing
the JVM heap and auto-configured memory limit.
Process: Limit=7.31 GB Total=1.94 GB Peak=1.94 GB
JVM: max heap size: Total=1.78 GB
JVM: non-heap committed: Total=35.56 MB
Buffer Pool: Free Buffers: Total=0
Buffer Pool: Clean Pages: Total=0
Buffer Pool: Unused Reservation: Total=0
Control Service Queue: Limit=50.00 MB Total=0 Peak=0
Data Stream Service Queue: Limit=374.27 MB Total=0 Peak=0
Data Stream Manager Early RPCs: Total=0 Peak=0
TCMalloc Overhead: Total=12.20 MB
Untracked Memory: Total=121.31 MB
Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
---
M be/src/runtime/exec-env.cc
M be/src/runtime/exec-env.h
M bin/start-impala-cluster.py
M docker/coord_exec/Dockerfile
M docker/coordinator/Dockerfile
M docker/daemon_entrypoint.sh
M docker/executor/Dockerfile
7 files changed, 126 insertions(+), 75 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/12262/5
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Bikramjeet Vig (Code Review)" <ge...@cloudera.org>.
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 3: Code-Review+1
(2 comments)
http://gerrit.cloudera.org:8080/#/c/12262/3/be/src/runtime/exec-env.cc
File be/src/runtime/exec-env.cc:
http://gerrit.cloudera.org:8080/#/c/12262/3/be/src/runtime/exec-env.cc@357
PS3, Line 357: ot
nit: to
http://gerrit.cloudera.org:8080/#/c/12262/3/be/src/runtime/exec-env.cc@405
PS3, Line 405: --mem_limit="
: << PrettyPrinter::PrintBytes(*bytes_limit);
this log line can be confusing to mean that the --mem_limit was set to bytes_limit.
eg. --mem_limit=10GB, avail_mem = 7GB, so bytes_limit will be 7GB but the log line will say:
"Using process memory limit from --mem_limit=7GB"
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 25 Jan 2019 21:09:40 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 1:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/12262/1/bin/start-impala-cluster.py
File bin/start-impala-cluster.py:
http://gerrit.cloudera.org:8080/#/c/12262/1/bin/start-impala-cluster.py@298
PS1, Line 298: .format(compute_impalad_mem_limit(cluster_size)
Need to remove this .format(), it's a noop
http://gerrit.cloudera.org:8080/#/c/12262/1/bin/start-impala-cluster.py@497
PS1, Line 497: mem_limit = compute_impalad_mem_limit(cluster_size)
Can move this outside loop
http://gerrit.cloudera.org:8080/#/c/12262/1/bin/start-impala-cluster.py@535
PS1, Line 535: # TODO: docker memory limit
remove
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 24 Jan 2019 00:55:15 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/1886/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 25 Jan 2019 08:33:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Joe McDonnell, Bikramjeet Vig, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12262
to look at the new patch set (#3).
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
IMPALA-7941: part 2/2: use cgroups memory limit
This uses the functionality from part 1 to detect the CGroups memory
limit and use it to set a lower process memory limit if needed.
min(system memory, cgroups memory limit) is used instead of
system memory to determine the memory limit. Behaviour of
processes without a memory limit set via CGroups is unchanged.
The default behaviour of using 80% of the memory limit detected
is still in effect. This seems like an OK default, but may
lead to some amount of wasted memory.
Modify containers to have a default JVM heap size of 2GB and
--mem_limit_includes_jvm, so that the automatically configured
memory limit makes more sense.
start-impala-cluster.py is modified to exercise all of this.
Testing:
Tested a containerised cluster manually on my system, which has
32GB of RAM. Here's the breakdown from the memz/ page showing
the JVM heap and auto-configured memory limit.
Process: Limit=7.31 GB Total=1.94 GB Peak=1.94 GB
JVM: max heap size: Total=1.78 GB
JVM: non-heap committed: Total=35.56 MB
Buffer Pool: Free Buffers: Total=0
Buffer Pool: Clean Pages: Total=0
Buffer Pool: Unused Reservation: Total=0
Control Service Queue: Limit=50.00 MB Total=0 Peak=0
Data Stream Service Queue: Limit=374.27 MB Total=0 Peak=0
Data Stream Manager Early RPCs: Total=0 Peak=0
TCMalloc Overhead: Total=12.20 MB
Untracked Memory: Total=121.31 MB
Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
---
M be/src/runtime/exec-env.cc
M be/src/runtime/exec-env.h
M bin/start-impala-cluster.py
M docker/coord_exec/Dockerfile
M docker/coordinator/Dockerfile
M docker/daemon_entrypoint.sh
M docker/executor/Dockerfile
7 files changed, 126 insertions(+), 75 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/12262/3
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 1:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3669/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 24 Jan 2019 00:51:17 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/1894/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 26 Jan 2019 01:13:07 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 3: Code-Review+1
(2 comments)
Looks good to me. I talked to Bikram, and he's going to finish the review.
http://gerrit.cloudera.org:8080/#/c/12262/2/be/src/runtime/exec-env.cc
File be/src/runtime/exec-env.cc:
http://gerrit.cloudera.org:8080/#/c/12262/2/be/src/runtime/exec-env.cc@364
PS2, Line 364: if (use_commit_limit) {
: avail_mem = MemInfo::commit_limit();
> I pulled out this logic into a function since it was getting unwieldy and a
Makes sense, that is much cleaner.
http://gerrit.cloudera.org:8080/#/c/12262/2/docker/daemon_entrypoint.sh
File docker/daemon_entrypoint.sh:
http://gerrit.cloudera.org:8080/#/c/12262/2/docker/daemon_entrypoint.sh@45
PS2, Line 45: export JAVA_TOOL_OPTIONS="-Xmx2g $JAVA_TOOL_OPTIONS"
> I did see that, but I didn't want to embed test-specific logic into the con
Yeah, I figured that this was already on our TODO list somewhere.
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 25 Jan 2019 23:08:27 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Joe McDonnell, Bikramjeet Vig, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12262
to look at the new patch set (#4).
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
IMPALA-7941: part 2/2: use cgroups memory limit
This uses the functionality from part 1 to detect the CGroups memory
limit and use it to set a lower process memory limit if needed.
min(system memory, cgroups memory limit) is used instead of
system memory to determine the memory limit. Behaviour of
processes without a memory limit set via CGroups is unchanged.
The default behaviour of using 80% of the memory limit detected
is still in effect. This seems like an OK default, but may
lead to some amount of wasted memory.
Modify containers to have a default JVM heap size of 2GB and
--mem_limit_includes_jvm, so that the automatically configured
memory limit makes more sense.
start-impala-cluster.py is modified to exercise all of this.
Testing:
Tested a containerised cluster manually on my system, which has
32GB of RAM. Here's the breakdown from the memz/ page showing
the JVM heap and auto-configured memory limit.
Process: Limit=7.31 GB Total=1.94 GB Peak=1.94 GB
JVM: max heap size: Total=1.78 GB
JVM: non-heap committed: Total=35.56 MB
Buffer Pool: Free Buffers: Total=0
Buffer Pool: Clean Pages: Total=0
Buffer Pool: Unused Reservation: Total=0
Control Service Queue: Limit=50.00 MB Total=0 Peak=0
Data Stream Service Queue: Limit=374.27 MB Total=0 Peak=0
Data Stream Manager Early RPCs: Total=0 Peak=0
TCMalloc Overhead: Total=12.20 MB
Untracked Memory: Total=121.31 MB
Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
---
M be/src/runtime/exec-env.cc
M be/src/runtime/exec-env.h
M bin/start-impala-cluster.py
M docker/coord_exec/Dockerfile
M docker/coordinator/Dockerfile
M docker/daemon_entrypoint.sh
M docker/executor/Dockerfile
M www/bootstrap/css/bootstrap.css
M www/bootstrap/css/bootstrap.min.css
9 files changed, 133 insertions(+), 77 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/12262/4
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 2:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/12262/2/be/src/runtime/exec-env.cc
File be/src/runtime/exec-env.cc:
http://gerrit.cloudera.org:8080/#/c/12262/2/be/src/runtime/exec-env.cc@233
PS2, Line 233: // Depending on the system configuration, we will have to calculate the process
: // memory limit either based on the available physical memory, or if overcommitting
: // is turned off, we use the memory commit limit from /proc/meminfo (see
: // IMPALA-1690).
> Can you update this comment to describe how cgroup mem limit fits in?
Done
http://gerrit.cloudera.org:8080/#/c/12262/2/be/src/runtime/exec-env.cc@358
PS2, Line 358: if (bytes_limit > MemInfo::physical_mem()) {
: LOG(WARNING) << "Memory limit "
: << PrettyPrinter::Print(bytes_limit, TUnit::BYTES)
: << " exceeds physical memory of "
: << PrettyPrinter::Print(MemInfo::physical_mem(), TUnit::BYTES);
: }
> If there is a cgroup limit, an equivalent of this logging would be useful (
Done
http://gerrit.cloudera.org:8080/#/c/12262/2/be/src/runtime/exec-env.cc@364
PS2, Line 364: LOG(INFO) << "Using global memory limit: "
: << PrettyPrinter::Print(bytes_limit, TUnit::BYTES);
> We might consider adding some information to indicate how the value was set
I pulled out this logic into a function since it was getting unwieldy and added some logging of the decisions. It's maybe not as nice as your suggested output but I think it's sufficient to understand the decision and the logic to produce the output is relatively simple.
http://gerrit.cloudera.org:8080/#/c/12262/2/docker/daemon_entrypoint.sh
File docker/daemon_entrypoint.sh:
http://gerrit.cloudera.org:8080/#/c/12262/2/docker/daemon_entrypoint.sh@45
PS2, Line 45: export JAVA_TOOL_OPTIONS="-Xmx2g $JAVA_TOOL_OPTIONS"
> You might find
I did see that, but I didn't want to embed test-specific logic into the containers.
I have a JIRA to do the minicluster change - it's a subtask of IMPALA-691 - but I think for it to work reasonably we need to fix some of the tests with unreasonably high memory requirements.
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 25 Jan 2019 07:38:26 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/1873/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 24 Jan 2019 02:09:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 6:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3677/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 26 Jan 2019 00:03:14 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Joe McDonnell, Bikramjeet Vig, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12262
to look at the new patch set (#2).
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
IMPALA-7941: part 2/2: use cgroups memory limit
This uses the functionality from part 1 to detect the CGroups memory
limit and use it to set a lower process memory limit if needed.
min(system memory, cgroups memory limit) is used instead of
system memory to determine the memory limit. Behaviour of
processes without a memory limit set via CGroups is unchanged.
The default behaviour of using 80% of the memory limit detected
is still in effect. This seems like an OK default, but may
lead to some amount of wasted memory.
Modify containers to have a default JVM heap size of 2GB and
--mem_limit_includes_jvm, so that the automatically configured
memory limit makes more sense.
start-impala-cluster.py is modified to exercise all of this.
Testing:
Tested a containerised cluster manually on my system, which has
32GB of RAM. Here's the breakdown from the memz/ page showing
the JVM heap and auto-configured memory limit.
Process: Limit=7.31 GB Total=1.94 GB Peak=1.94 GB
JVM: max heap size: Total=1.78 GB
JVM: non-heap committed: Total=35.56 MB
Buffer Pool: Free Buffers: Total=0
Buffer Pool: Clean Pages: Total=0
Buffer Pool: Unused Reservation: Total=0
Control Service Queue: Limit=50.00 MB Total=0 Peak=0
Data Stream Service Queue: Limit=374.27 MB Total=0 Peak=0
Data Stream Manager Early RPCs: Total=0 Peak=0
TCMalloc Overhead: Total=12.20 MB
Untracked Memory: Total=121.31 MB
Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
---
M be/src/runtime/exec-env.cc
M bin/start-impala-cluster.py
M docker/coord_exec/Dockerfile
M docker/coordinator/Dockerfile
M docker/daemon_entrypoint.sh
M docker/executor/Dockerfile
6 files changed, 59 insertions(+), 36 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/12262/2
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
IMPALA-7941: part 2/2: use cgroups memory limit
This uses the functionality from part 1 to detect the CGroups memory
limit and use it to set a lower process memory limit if needed.
min(system memory, cgroups memory limit) is used instead of
system memory to determine the memory limit. Behaviour of
processes without a memory limit set via CGroups is unchanged.
The default behaviour of using 80% of the memory limit detected
is still in effect. This seems like an OK default, but may
lead to some amount of wasted memory.
Modify containers to have a default JVM heap size of 2GB and
--mem_limit_includes_jvm, so that the automatically configured
memory limit makes more sense.
start-impala-cluster.py is modified to exercise all of this.
Testing:
Tested a containerised cluster manually on my system, which has
32GB of RAM. Here's the breakdown from the memz/ page showing
the JVM heap and auto-configured memory limit.
Process: Limit=7.31 GB Total=1.94 GB Peak=1.94 GB
JVM: max heap size: Total=1.78 GB
JVM: non-heap committed: Total=35.56 MB
Buffer Pool: Free Buffers: Total=0
Buffer Pool: Clean Pages: Total=0
Buffer Pool: Unused Reservation: Total=0
Control Service Queue: Limit=50.00 MB Total=0 Peak=0
Data Stream Service Queue: Limit=374.27 MB Total=0 Peak=0
Data Stream Manager Early RPCs: Total=0 Peak=0
TCMalloc Overhead: Total=12.20 MB
Untracked Memory: Total=121.31 MB
Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Reviewed-on: http://gerrit.cloudera.org:8080/12262
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/runtime/exec-env.cc
M be/src/runtime/exec-env.h
M bin/start-impala-cluster.py
M docker/coord_exec/Dockerfile
M docker/coordinator/Dockerfile
M docker/daemon_entrypoint.sh
M docker/executor/Dockerfile
7 files changed, 126 insertions(+), 75 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 7
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-7941: part 2/2: use cgroups memory limit
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12262 )
Change subject: IMPALA-7941: part 2/2: use cgroups memory limit
......................................................................
Patch Set 6: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/12262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9fb4fb936a46fc194a204391d03c07c8c7fba21
Gerrit-Change-Number: 12262
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 26 Jan 2019 04:40:06 +0000
Gerrit-HasComments: No